How to create stereoParameters with known rotation and translation matrices, for rectification

I have a translation and rotation matrix for two identical cameras. The translation matrix is in units of mm. The left and right camera images are rendered using these orientations.

How do I figure out the translation and rotation expected by stereoParameters? I have tried:

% Rotation and position of each camera
camera_left.sensor_rotation = eul2rotmat(deg2rad([0,-5,0]));
camera_right.sensor_rotation = eul2rotmat(deg2rad([0,5,0]));
camera_left.sensor_position = [-30,0,0];
camera_right.sensor_position = [30,0,0];

focalLength    = focal_length / pixel_pitch;  % focal length in pixels
principalPoint = [1024/2, 1280/2];
imageSize      = [1280, 1024];
c = cameraIntrinsics(focalLength,principalPoint,imageSize);
cameraParams = cameraParameters('IntrinsicMatrix',c.IntrinsicMatrix);

% This is the part that seems to be incorrect
rot = camera_left.sensor_rotation * camera_right.sensor_rotation';
t = camera_left.sensor_position - camera_right.sensor_position * camera_left.sensor_rotation';

stereoParams = stereoParameters(cameraParams,cameraParams,rot, t);
[left, right] = rectifyStereoImages(image_left,image_right,stereoParams);

But the rectified images do not return valid disparities (and depending on the camera rotations, they look completely wrong).

Answer 1

[William Rose]

@C.S.,

I am surprised your code runs, because lines 2 and 3 call a function by the wrong name - unless you wrote two functions of your own. You call eul2rotmat() and eurl2rotmat(). The Matlab function is eul2rotm().

[William Rose]

ALso,

cameraParameters= cameraParameters(...)

should be

cameraParams= cameraParameters(...)

and then you should pass cameraParams (twice) to stereoParameters().

[William Rose]

Also, argument 3 (=rot) and 4 (=t) to stereoParameters() should be rotation of camera 2 and translation of camera 2. I don't understand the logic for defining rot and t the way you are doing. For rot, you multiply two roation matrices, which are -5d rotaiton about y and +5d rotation about y. The product of those two rotations will bo no rotation, since the second one cancels out the first. I doubt that is what you want.

As for the t argument, why not just take the differnce of the two camera positions?

If you bring the cameras very close together (1 mm) and pass the identity matrix for rot and pass a vector of zeros for t, do yuo get close to reasonable results? I would hope so.

[C.S.]

Thank you William!

Apologies for the function errors. I typed this up to remove confusing variable names etc, but made some errors in the function names. Those are not in the code I'm running that is not working. I updated the opening post.

'The product of those two rotations will bo no rotation, since the second one cancels out the first. I doubt that is what you want.'

I'm multiplying by the inverse of rotation matrix 2 (to 'undo' the orientation of camera 2), then by the rotation matrix 1 (to apply the orientation of camera 1). This doesn't equal the identity, in this example it is essentially a 10 degree rotation.

With the translation -- I tried what you suggested, but that also does not work. The version in the original post is assuming the translation is supposed to be in the coordinate system of camera 1, but it appears that is not the case either...

Using a tiny translation and identity matrix for both cameras, the output looks fine. The issue arises when I introduce the rotations.

Thank you for your help! I'd appreciate any other suggestions.

[William Rose]

Aha! Now I see the prime after the second rotation matrix, to transpose it, which is the same as inverting it, since rotation matrices are orthonormal.

I see in the Help for stereoParameters, and I know you have seen the same, that rot and t are the rotation and translaiton of camera 2 "relative to camera 1". Therefore I assume the rot and t should be expressed in "camera 1 coordinates". I have not worked twith the computer vision toolbox, but when working with rotations and translations I always have to double check my work, especially with rotations, to correcly determine whether I want, and whether I am correctly expressing, a rotation of the coordinate frame, or a rotation of the objects in a frame - which are the inverse of each other. So the first thing I would check is to see if things work better if I invert the rotation, or the translation, or both.

[William Rose]

@C.S.,

I would also try not rotating and not translating camera 1 at all. Only translate and rotate camera 2. Then does stereoParameters() give expected results? That would be a step in the right direction...

I previously wrote "I assume the rot and t should be expressed in camera 1 coordinates." Maybe that is wrong, especially for the translation part. The Help says the translation argument to stereoParameters() shoud be the position of cam2 relative to cam1, but maybe that means cam2position-cam1position, in "absolute" coordinates. If true, your formula for t would need to be changed.