You can switch directly to the correct "Displacement Texture" by clicking on the outer right button (shaped like 2 capsules) to the right of the "Texture Field" in the "Displacement Modifier":
Wow, that would've been real handy if I had known that... :D I'm learning something new everyday, thanks for sharing Ingmar! -Chunck :)