To find out the X, Y and Z co-ordinates from 3D point cloud.

Hello I'm using Stereo Vision in MATLAB. In that i have to find out the X, Y and Z co-ordinates of region of interest(in my case white color object) from 3D point cloud. I able to create 3D scene reconstruction but how to proceed further I don't know. Can anyone suggest some code examples or method to find out these co-ordinates. I'm attaching my 3d point cloud and disparity map with this. Thanks in Advance.

Answer 1

[Dima Lisin]

Hi Naseeb,

If you trying to find the 3D coordinates of a particular object in the scene, then you need to be able to detect it either in the image, or in the point cloud. reconstructScene returns an M-by-N-by-3 array, where [M,N] is the size of the rectified image. So the 3D coordinates are in correspondence with image pixels. Thus, if you can find the object in image 1 of your stereo pair, then you can simply look up the [X,Y,Z] coordinates of the pixels belonging to this object in the 3D point array returned by reconstructScene.

How to detect an object in the image is a whole separate question, and the answer depends on the type of object. If the object has a distinctive color, you can try using a segmentation algorithm. Otherwise, you can use local feature matching, or a detection algorithm trained for a specific kind of object, such as vision.CascadeObjectDetector or vision.PeopleDetector.

Alternatively, if your object has a particular 3D shape, such as plane, sphere, or cylinder, you can try detecting it in the point cloud directly. There are functions for fitting primitive shapes to the point cloud: pcfitplane, pcfitshpere, and pcfitcylinder.

[Naseeb Gill]

Thanks Dima Sir, I'm working as you suggest me and tell you if I got struck somewhere.

[Naseeb Gill]

Hi Dima Sir, As you suggest me I find out the centroids (mx2 double) i.e X and Y in terms of pixels of my Region of Interest which is white in color and 3-4 in numbers. I stored these values into array called centroids. I have also another array called point3D (437x540x3 single) in which [X,Y,Z] values in mm unit is stored with corresponding to X and Y values of pixels. I tried to write a code to extract those [X,Y,Z]values and store into another array named co_ord. But it's not working. I'm not able to store exract and store [X,Y,Z] values in co_ord array.

My code is as

 load('C:\Users\Naseeb\Desktop\centroids.mat'); % load centroids file
load('C:\Users\Naseeb\Desktop\point3D.mat'); %load point3D file
centroids = ceil(centroids); % change values into integer
x = cell(size(centroids,1),1); % create array X for storing first column values
y = cell(size(centroids,1),1); % create array Y for storing second column values
 i = cell(size(centroids,1),1);
for i = 1:size(centroids,1) 
   for row = 1:size(centroids,1)
    for column = 1:size(centroids,2)
        x{i} = centroids(size(centroids,1),1); % store values of first column into array X
        y{i} = centroids(size(centroids,1),2); % store values of second column into array Y
        co_ord = point3D(x{i},y{i},:); % store values into co_ord file
    end
   end 
end

I'm attaching my files to this.

[Uzair UlHaq]

If you want to extract x,y,z values from points3D then simply use: x=points3D(:,:,1) y=points3D(:,:,2) z=points3D(:,:,3)