Currently, there is no way to make a vector displacement map in Blender. There are a few other softwares that do support vector displacement baking though, one of which is ZBrush. Hopefully the Blender dev team will add this feature soon!
Vector displacement maps are similar to normal maps, but they are not inherently normal maps. Normal maps are used to alter the direction of shading normals, whereas vector displacement actually displaces geometry.
