image stitching: ideas to solve singular or badly scaled matrix

I'm trying to do image mosaicing according to this example of MATLAB:

Feature Based Panoramic Image Stitching

My images are captured in an approximate vertical column. What the code does, is to find all the transformation with respect to the first image. i.e. :

tforms(n).T = tforms(n-1).T * tforms(n).T;

Then, it starts stitching (warping) the images based on the obtained transformations above. The code works well when stitching 3-4 images. But starts to give this popular warning after that! I'm trying to stitch about 25 images and with the start of this warning the images are not stitched properly:

Warning: Matrix is close to singular or badly scaled. Results may be inaccurate. RCOND =  2.679189e-08. 
> In affine2d>affine2d.get.Tinv at 339
  In affine2d>affine2d.transformPointsInverse at 214
  In imwarp>remapAndResampleGeneric2d at 248
  In imwarp>remapPointsAndResample at 188
  In imwarp at 173

Could anyone suggest how to solve this problem? From physical point of view, this procedure must work, but how to fix this mathematical shortcoming? To give you an idea of the resulting error, here you can see one ill-conditioned transformation matrix and the final mosaic:

1.0e+03 *

      0.0010   -0.0000         0
      0.0000    0.0010         0
     -0.0592   -5.8776    0.0010

Answer 1

[Dima Lisin]

Hi Mona,

the problem here is that each time the transformation is estimated, errors are introduced. These errors accumulate, which is why the stitching works for a few images, but not for 25.

One thing you can try to improve the mosaic, is to start from the middle image, rather than from the first one. Let's say you have 7 images. First build a mosaic using images 4, 5, 6, and 7. Then build a mosaic using images 4, 3, 2, and 1. Then stitch the two mosaics. This way, in both mosaics you are not moving two far from the starting image, so you should have a smaller error.

The other alternative is to use the Optimization Toolbox to do a global optimization over the transformations to minimize the geometric errors.

[Mona Mahboob Kanafi]

Thanks a lot for your explanation and hints! :)

[Dima Lisin]

You are most welcome. And here is another hint, as long as there is enough overlap, try using fewer images. What I mean is that if you can throw out every other image in your sequence, and there is still enough overlap between the images, you may get a better result. There will be fewer transformations, which should reduce the accumulated error.