How to map the disparity image values to the original image?

14 views (last 30 days)
Shida on 8 Feb 2016
Commented: Dima Lisin on 22 Feb 2016
Using stereo camera calibration and Matlab's instructions I produce the disparity map. Now I want to create a Kinect-like depth map for which I need to map the depth image to the original image (e.g., Camera 1). The problem is that the "disparity" and "reconstructScene" output matrices with the dimensions of the image produced by the rectifyStereoImages function and not the original image. I wonder how I can map the disparity values/image to the pixels of the original image. Below is an example of the code (a simplified version and not the actual one I am running, but mostly the same).
[J1,J2]=rectifyStereoImages(I1,I2,stereoParams, 'OutputView', 'full');
disparityMap = disparity(rgb2gray(J1), rgb2gray(J2), 'DisparityRange', ...
disparityRange);
point3D = reconstructScene(disparity, stereoParams);
% Where:
% size of I1 is 5208x3476x3
% size of disparity is 3672x7112
% size of J2 is 3672x7112x3
And now I want to get a depth image or at least a disparity image with the size 5208x3476 which corresponds pixel-wise to the original image (I1)
I appreciate if anyone has an idea/suggestion.
Best wishes,

Dima Lisin on 21 Feb 2016
You would have "un-rectify" the image, and there is no easy way to do that because rectification is a non-linear transformation.
Dima Lisin on 22 Feb 2016
Does it have to be the original RGB image, or can it be the undistorted RGB image? If it is the latter, you can undistort your stereo images, instead of rectifying them. Then compute dense optical flow from one into the other. Then use triangulate to get a dense 3D reconstruction, and compute the depth from that.
Out of curiosity, why not just compute the depth map for the rectified image?