How can I be sure having found the correct camera angle and position relative to the ground plane with a manual adaption by rotating the ground? Could an addon like FSpy help here?
Yes, FSpy would definitely come in handy in this case.
If you stick with the manual route, set your cameras lens, and scale of the door, then using your best eye judgment, it should line up well.
If I rotate the camera around "Global X" in my scene, the horizon aligns well and the camera is pointing slightly upwards (around 96 degrees) as I would have estimated from the footage where a house seems to be standing on a hill with a slight slope. I accidentally first grabbed the plane and aligned the horizion with the footage which led to a strange result:
So, I think, for a manual alignment you should best know the real location well (since you've filmed it) because there're otherwise several possible settings that allow an alignment without correctly reconstructing the real life location and camera settings.
I've tried to use fSpy for automatically calculating the camera angle and position by tracing real world parallel lines that converge due to the perspective distortion in a vanishing point. Unfortunately the only parallel real world lines also appear parallel in the image so that a reconstruction with fSpy fails:
