Here are some tips which you can try:
• Camera Calibration: Make sure you have good stereo calibration so that your calibration images are in focus, and that you have enough images to cover most of the field of view. Use the showReprojectionErrors to check the reprojection errors. Ideally, they should be less than .5 of a pixel. If you see that any calibration image pairs produce reprojection errors much higher than the others, try excluding them and re-calibrating. Also, look at the result of rectfyStereoImages to make sure that the rectification makes sense. Corresponding points should be on the same horizontal lines.
• Pre-processing Rectified Images: Look carefully at your disparity map. Are the shapes of the objects of interest visible or is there too much noise? If the disparity map looks bad, try pre-processing the rectified images. Applying histogram equalization using the histeq function can help. It may also help to low-pass filter the rectified images.
• Tuning Disparity Parameters: Try tuning the parameters of the disparity function. Specifically, 'Disparity Range' and 'Block Size'. To figure out what the disparity range should be create an anaglyph of the rectified images using the stereoAnaglyph function, and display it using imtool. In imtool, measure the distances in pixels between a few pairs of corresponding points in the two images. That should give you an idea of what the disparity range should be. Then tune 'Block Size' to get the object silhouettes to appear in the disparity map without too much noise. If your disparity values are very high (e. g. greater than 256 pixels), then you can try moving the cameras closer together, or moving the cameras further away from the objects of interest. If your disparity values are close to 0, then you should either move the cameras further apart from each other or move them closer to the objects of interest.
• Post-processing Disparity Map: Once you get a decent disparity map, you can try to clean up the noise some more by applying the median filter to it using the medfilt2 function.
• Restricting the 3D Volume: Once you get the 3-D points from reconstructScene, you can get rid of much of the noise by "cropping" the 3-D volume. Simply eliminate the points for which the Z coordinate is either too small (e.g. less than 0) or too large, by setting their coordinates to NaN. You can similarly limit the range of X and Y.