Main Content

View Skeletal Data from Kinect for Windows V2

This example uses:

Open Live Script

This example shows how to view an RGB image from the Kinect® for Windows® V2 with the skeleton joint locations overlaid on the image.

Requirements

  • MATLAB® and Image Acquisition Toolbox™

  • Kinect for Windows V2 sensor

  • Minimum PC configuration: Windows 10 64-bit with a dedicated USB3 controller

Set Up Kinect V2 for Color and Depth Acquisition

Create color and depth videoinput objects.

colorVid = videoinput("kinect",1)
Summary of Video Input Object Using 'Kinect V2 Color Sensor'.

   Acquisition Source(s):  Kinect V2 Color Source is available.

  Acquisition Parameters:  'Kinect V2 Color Source' is the current selected source.
                           10 frames per trigger using the selected source.
                           'BGR_1920x1080' video data to be logged upon START.
                           Grabbing first of every 1 frame(s).
                           Log data to 'memory' on trigger.

      Trigger Parameters:  1 'immediate' trigger(s) on START.

                  Status:  Waiting for START.
                           0 frames acquired since starting.
                           0 frames available for GETDATA.

depthVid = videoinput("kinect",2)
Summary of Video Input Object Using 'Kinect V2 Depth Sensor'.

   Acquisition Source(s):  Kinect V2 Depth Source is available.

  Acquisition Parameters:  'Kinect V2 Depth Source' is the current selected source.
                           10 frames per trigger using the selected source.
                           'Depth_512x424' video data to be logged upon START.
                           Grabbing first of every 1 frame(s).
                           Log data to 'memory' on trigger.

      Trigger Parameters:  1 'immediate' trigger(s) on START.

                  Status:  Waiting for START.
                           0 frames acquired since starting.
                           0 frames available for GETDATA.

Look at the device-specific properties on the depth source device, which is the depth sensor on the Kinect V2.

Set 'EnableBodyTracking' to on, so that the depth sensor returns body tracking metadata along with the depth frame.

depthSource = getselectedsource(depthVid);
depthSource.EnableBodyTracking = "on";

Acquire 100 color and depth frames.

framesPerTrig = 100;
colorVid.FramesPerTrigger = framesPerTrig;
depthVid.FramesPerTrigger = framesPerTrig;

Start the depth and color acquisition objects. This begins acquisition, but does not start logging of acquired data.

pause(5);
start([depthVid colorVid]);

Access Image and Skeletal Data

Get images and metadata from the color and depth device objects.

[colorImg] = getdata(colorVid);
[~,~,metadata] = getdata(depthVid);

This is the order of joints returned by the Kinect.

  • SpineBase = 1;

  • SpineMid = 2;

  • Neck = 3;

  • Head = 4;

  • ShoulderLeft = 5;

  • ElbowLeft = 6;

  • WristLeft = 7;

  • HandLeft = 8;

  • ShoulderRight = 9;

  • ElbowRight = 10;

  • WristRight = 11;

  • HandRight = 12;

  • HipLeft = 13;

  • KneeLeft = 14;

  • AnkleLeft = 15;

  • FootLeft = 16;

  • HipRight = 17;

  • KneeRight = 18;

  • AnkleRight = 19;

  • FootRight = 20;

  • SpineShoulder = 21;

  • HandTipLeft = 22;

  • ThumbLeft = 23;

  • HandTipRight = 24;

  • ThumbRight = 25;

Create a skeleton connection map to link the joints.

SkeletonConnectionMap = [
    [4 3];  % Neck
    [3 21]; % Head
    [21 2]; % Right Leg
    [2 1];
    [21 9];
    [9 10];  % Hip
    [10 11];
    [11 12]; % Left Leg
    [12 24];
    [12 25];
    [21 5];  % Spine
    [5 6];
    [6 7];   % Left Hand
    [7 8];
    [8 22];
    [8 23];
    [1 17];
    [17 18];
    [18 19];  % Right Hand
    [19 20];
    [1 13];
    [13 14];
    [14 15];
    [15 16];
    ];

Extract the 90th frame and tracked body information.

imageFrame = framesPerTrig-10;
imageframeMetadata = metadata(imageFrame);

Find the indexes of the tracked bodies.

anyBodiesTracked = any(imageframeMetadata.IsBodyTracked ~= 0);
trackedBodies = find(imageframeMetadata.IsBodyTracked);

Find number of skeletons tracked.

nBodies = length(trackedBodies);

Get the joint indices of the tracked bodies with respect to the color image.

colorJointIndices = imageframeMetadata.ColorJointIndices(:,:,trackedBodies);

Extract the 90th color frame.

colorImage = colorImg(:,:,:,imageFrame);

View RGB Image with Skeletal Overlay

Marker colors for up to six bodies.

colors = ["r";"g";"b";"c";"y";"m"];

Display the RGB image.

imshow(colorImage);

Overlay the skeleton on this RGB frame.

for i = 1:24
    for body = 1:nBodies
        X1 = [colorJointIndices(SkeletonConnectionMap(i,1),1,body) colorJointIndices(SkeletonConnectionMap(i,2),1,body)];
        Y1 = [colorJointIndices(SkeletonConnectionMap(i,1),2,body) colorJointIndices(SkeletonConnectionMap(i,2),2,body)];
        line(X1,Y1,"LineWidth",1.5,"LineStyle","-","Marker","+","Color",colors(body));
    end    
    hold on;
end
hold off;

Figure contains an axes object. The axes object contains 49 objects of type image, line.

Related Topics