In stereocalibration, is the relationship between the 'R and T output as PoseCamera2' and the actual camera position the same, or does the sign of x in T reverse?

I am currently calibrating four cameras (Camera1, Camera2, Camera3, Camera4). To do this, I have created pairs (Camera1 & Camera2, Camera2 & Camera3, Camera1 & Camera4) and performed calibration to determine the relative positions of all cameras in the coordinate system of Camera1. For Camera1 and Camera2, I added about 80 images of a checkerboard taken using the stereocalibration feature of the calibration app for calibration.

As a result, I obtained the following:

R = [0.794, -0.0318, 0.605; 0.0226, 0.999, 0.0228; -0.606, -0.00446, 0.795]

T = [-2793, 44.86, 483.2] (units in [mm]).

The visual output, which I have attached as an image, shows that rotating Camera2 by R and translating it by T to align with the coordinate system of Camera1 makes it coincide with Camera1. Therefore, it can be seen that R and T correspond with the visual output.

However, the actual relative position of Camera2 to Camera1 in the coordinate system of Camera1 should be [2793, 44.86, 482.3]. Thus, I am considering that the sign of the x component of T obtained through stereocalibration might be reversed compared to the actual T. Is my understanding incorrect?

Answer 1

[Qu Cao]

Sorry for the confusion. We will update our documenation to be more specific about the meaning of PoseCamera2.

PoseCamera2 is the relative pose that transforms camera 2 pose into camera 1 pose. If you represent the pose of camera 1 as [R1, t1; 0 1], pose of camera 2 as [R2, t2; 0 1], and PoseCamera2 as [R, t; 0 1]. Then you will get the following relationship:

[R1, t1; 0 1] = [R, t; 0 1] * [R2, t2; 0 1]

which gives you

R1 = R*R2

and

t1 = R*t2+t

In the plot, the x component of t2 is positive, if you add R*t2 with t, which is negative, you get t1, which is equal to zero.

[Taisei]

Thank you for your response.

I understand that the relationship R1=R⋅R2 and t1=R⋅t2+t holds true if the pose of Camera 1 is represented as [R1,t1;0,1], the pose of Camera 2 as [R2,t2;0,1], and PoseCamera2 as [R,t;0,1].

I have a question regarding this. Does each parameter R1, t1, R2, t2, R, t depend on each coordiante system (camera 1, 2, and posecamera2)? In short, I would like to know where is the origin for each parameter.

Additionally, I have a related question. When reconstructing a checkerboard in 3D using the intrinsic and extrinsic parameters obtained from the stereo calibration, I believe we will use the functions rectifyStereoImages, disparityMap, reconstructScene, and detectCheckerboardPoints. In this process, should we use the extrinsic parameters R and T as they are without modification? Is there a need to reverse the x component of T or make any other adjustments?

I appreciate your assistance with these two questions.

[Qu Cao]

• R and t doens't depend on the coordinate system in which camera 1 pose and camera 2 pose are represented. The coordiante system used in stereo camera calibration is described here. Essentially, In a stereo system, the origin is located at the optical center of camera 1.
• If you want to do 3-D reconstruction from disparity, I would recommend using the reprojection matrix returned by rectifyStereoImages. Once you rectify the stereo image, the cameras will be "rotated" so that their Z-axis are in parallal. Please also check the second syntax of rectifyStereoImages which might be helpful for you to do cooridnate transformation between unrectified and rectified camera coordinate systems.

[Taisei]

I understood that the origin is located at the optical center of camera 1 in stereo system. But, I think that this means R and t depend on the corrdinate system in which camera 1 pose. Is it wrong? If it is wrong, which corrdinate system do R and t depend on? Do they depend on world corrdinate system?

Here is a different issue:

Currently, I am facing an issue, referred to as 'After performing stereo calibration using a checkerboard, the same checkerboard is reconstructed in 3D. I have several questions regarding the results.' related to the 3D reconstruction of a checkerboard from two cameras. If possible, I would appreciate it if you could also respond to my already posted question about this issue.

Here is the link: https://jp.mathworks.com/matlabcentral/answers/2129111-after-performing-stereo-calibration-using-a-checkerboard-the-same-checkerboard-is-reconstructed-in?s_tid=srchtitle