(question)The coordinates of the reconstructed 3D points are different after the virtual camera intrinsic K has also changed proportionally after image resize?

As far as I know, after image resize, the corresponding intrinsic parameter K also changes proportionally, but why the coordinates of the 3D reconstruction of the same point are not the same?

The following program is a simple experiment, the original image size is 1080*1920, after resize it becomes 480*640, the intrinsic parameter K1 corresponds to the original image, the intrinsic parameter K2 corresponds to the resize, RT1, RT2 are the extrinsic projection matrix of the camera (should remain unchanged?,[R,T],3*4 size), why is there a difference in the reconstructed 3D points?

%% 验证图像resize后，内参是否成比例缩放的正确性，使用重建3D点来建立

fx = 1040;
fy = 1040;
cx = 1920/2;
cy = 1080/2;

K1 = [fx,0,cx;
    0,fy,cy;
    0,0,1];

RT1 = [eye(3),[4;5;6]];% just random set
RT2 = [rotz(30),[40;50;60]];% just random set
p1 = K1*RT1;% extrinsic projection matrix
p2 = K1*RT2;% extrinsic projection matrix

pt1 = [100,200]; 
pt2 = [300,400];

point3d1 = triangulate(pt1, pt2, p1,p2)

point3d1 = 1×3

-260.0716 -27.3955 273.9519

% resize 3D construct test,from (1080,1920)to (480,640)
rx = 640/1920;
ry = 480/1080;
fx = fx*rx;
fy = fy*ry;
cx = cx*rx;
cy = cy*ry;

K2 = [fx,0,cx;
    0,fy,cy;
    0,0,1];
p1 = K2*RT1;
p2 = K2*RT2;

pt1 = [pt1(1)*rx,pt1(2)*ry];
pt2 = [pt2(1)*rx,pt2(2)*ry];
point3d2 = triangulate(pt1, pt2, p1,p2)

point3d2 = 1×3

-193.0397 -26.7213 189.1251

you see, point3d1 and point3d2 is not same,why?

your answer would be greatly appreciate!

Answer 1

[cui,xingxing]

After careful consideration, I was once again fortunate enough to obtain a more plausible explanation, which I now state as follows to help others.

In a short conclusion:

Image scaling must specify a uniform (fx=fy) scaling factor in order to derive the correct intrinsic parameter K, otherwise inconsistencies in the x,y axis focal lengths with respect to the original image directly lead to deviations in the calculated 3D points!

Returning to the problem at the beginning, the given image size is 1080*1920, and its focal length is 1040 pixels, i.e. fx=fy=1040, because by definition fx=f/dx,fy=f/dy, where dx, dy are the number of pixels per unit length, and f is the actual physical size of the focal length; thus the a priori dx=dy can be introduced, which is constant This "convention" should also be followed for later image scaling.

Imagine if the scaled image fx,fy were obtained in different proportions, dx,dy would not be the same, causing distortion of the image, and in addition, according to the external projection matrix P = K*[R,t], fx,fy in K would vary disproportionately leading to a deviation in the calculated P!

%% 验证图像resize后，内参是否成比例缩放的正确性，使用重建3D点来建立

fx = 1040;
fy = 1040;
cx = 1920/2;
cy = 1080/2;

K1 = [fx,0,cx;
    0,fy,cy;
    0,0,1];

RT1 = [eye(3),[4;5;6]];% just random set
RT2 = [rotz(30),[40;50;60]];% just random set
p1 = K1*RT1;% extrinsic projection matrix
p2 = K1*RT2;% extrinsic projection matrix

pt1 = [100,200]; 
pt2 = [300,400];

point3d1 = triangulate(pt1, pt2, p1,p2)

point3d1 = 1×3

-260.0716 -27.3955 273.9519

So you can't simply scale a 1080*1920 image to a 480*640 size image because they are scaled differently on the x,y axis. To use the presumed K, you should get your target image size by scaling fx=fy=1040 in the same proportion.

scalar = 0.6 % assume any value, just make sure that fx = fy in the same proportion!

scalar = 0.6000

rx = scalar;
ry = scalar;

fx = fx*rx;
fy = fy*ry;
cx = cx*rx;
cy = cy*ry;

K2 = [fx,0,cx;
    0,fy,cy;
    0,0,1];
p1 = K2*RT1;
p2 = K2*RT2;

pt1 = [pt1(1)*rx,pt1(2)*ry];
pt2 = [pt2(1)*rx,pt2(2)*ry];
point3d2 = triangulate(pt1, pt2, p1,p2)

point3d2 = 1×3

-260.0716 -27.3955 273.9519

Now they look the same, officially because the initial pixel focal lengths are both 1040.