Thank you very much for your reply. I have calibrated the camera twice and I assume its correct. Also, the problem occurs earlier than helperFind3Dto2DCorrespondences comes into play.
I have now set up a ground truth set which contains a perfect circle within the x-y-Plane. Both the first estimated pose as well as the actual pose of the ground truth start from [0 0 0]. However, the second relative pose (the first calculated one) is majorly directed along the z axis (something like t = [0.015 -0.03 0.988]).
I have checked all the essential matrices and its corresponding R and t results, all of them rather tend to show a significant translation in z-direction than in x- or y-direction.
Considering this issue, the problem can only be related to one of the following:
- calibration (as you mentioned, which I, however, assume to be correct. But is there any possiblity to verify them results?)
- coplanar feature points. I calculate the essential matrix (not the fundamental matrix), so I reckon all feature points being in one single plane is not an issue, correct?
Do you have an idea how to get to the bottom of this?