Using Optical Flow to warp an image

Hi guys. I have computed the optical flow between images A and B. I wish to use this to warp image C to D. The problem is: the flow is a velocity vector with decimal values. Images are in the form of matrices for which rows and columns are integer values. Hence, C(i,j)+ optical flow(i,j) won't give me D(i,j) Please tell me how I can convert a matrix to a graph or how I can handle the decimal values. Thanks !!!

[David Young]

Do you have a single optic flow vector for the whole image, or a sparse set of optic flow vectors for some features in the image, or a dense set of optic flow vectors for each pixel in the image?

[QEWE]

I've got a dense set of optic flow vectors for each pixel in the image.

Answer 1

[David Young]

Given a vector for every pixel, you can use interp2 to do the warping and to handle the non-integer lookup. Suppose that vx represents the x-component of the flow, such that the rightwards component of velocity at D(i,j) is x(i,j). Likewise vy is the y-component. Then you can do something similar to this:

C= imread('pout.tif'); % test image
[x, y] = meshgrid(1:size(C,2), 1:size(C,1));

% generate synthetic test data, for experimenting
vx = 0.1*y;   % an arbitrary flow field, in this case
vy = 0.1*x;   % representing shear

% compute the warped image - the subtractions are because we're specifying
% where in the original image each pixel in the new image comes from
D = interp2(double(C), x-vx, y-vy);

% display the result
imshow(D, []);

The bit that matters is the line in the middle with the call to interp2.

By the way, if you had a sparse set of flow vectors instead, you'd need to look at using imtransform with maketform.

[QEWE]

Thanks for the solution.

I've tried something similar before.

When I write:

D = interp2(double(C), x+vx, y+vy);

I get the same image with minor shifts (i.e, I don't get the rotated/warped image.

When I write:

D = interp2(double(C), x-vx, y-vy);

I see what's more like a mirror image of the original image, but again, its not the rotated or warped image.

Given that vx and vy represent the optical flow, do you reckon its fine to use interp2 ? Does interp2 ensure that every pixel moves in the direction and by the magnitude specified by vx and vy for that pixel ?

[QEWE]

ok. I am getting the mirror image because the meshgrid had been defined as [y,x] and the interp2 used [x,y].

Now, all the 3 images look alike: original image, warped image with '-' sign and the warped image with '+' sign in the interp2 function.

Only very minor differences noted like haziness or minor shift. But the image has not been warped.

[David Young]

The example I gave works, so I'm not sure why you are not getting the results you expect with your data. interp2 does indeed ensure that every pixel moves as specified by vx and vy for that pixel. Are you sure that your vx and vy arrays are correct?

[QEWE]

Ok. I guess there was a problem with my optical flow code and the datatypes of matrices used. It works now. Thanks a lot for your solution !!! :)

[Natesh Srinivasan]

@David

I think this is incorrect for non uniform flow fields. For Example, consider the following code, where A is the input image and B is the 'warped' image

function testinterpolation()

A = rand(5,5)

[X, Y] = meshgrid([1:5],[1:5]);
Mx = X/10;
My = Y/10;

xplusMx = X - Mx;
yplusMy = Y - My;

B = interp2 (A,xplusMx,yplusMy,'linear',0)

end

we cannot predict where a particular pixel has come from if by simply taking a minus sign because this could have been from a completely different pixel.