@skeleton attribute

dp.kinect dp.kinect2 dp.kinect3 dp.oak

signature

skeleton INT

values

0: off default
1: joints
2: joints and orientation

examples

@skeleton 1

Skeleton, joint, and user tracking output on the fifth outlet. Output format (native Max messages, OSC protocol, etc.) is determined by @skeletonformat.

Some sensors may not support orientation output @skeleton 2.

User Identification

User identification messages are generated by enabling @skeleton. The data provided is an integer playerID which is a unique identifier representing the currently tracked/lost person. This playerID is unique within a given point of time and for a person seen by the sensor. Its minimum value is 1 and often 6 is the most people a sensor can detect. Keep in mind that some sensors support detecting more than six people. This same playerID is used for matching/connecting with other sensor data like the person’s location, joints, depthmap pixels, and sound location/association.

The three messages output are

OSC                        Max native                
-------------------------------------------------
/new_user playerID         new_user playerID
/calib_success playerID    calib_success playerID
/lost_user playerID        lost_user playerID

# Examples
/new_user 2                new_user 2
/calib_success 2           calib_success 2
/lost_user 2               lost_user 2

new_user Indicates a new person has been identified but full skeleton tracking is not yet available. This is the first message in the possible progression of events for a given playerId.

calib_success Indicates the person’s skeleton joints are now being fully tracked. This can only be received after a new_user message.

lost_user Indicates the previously identified/tracked person has been lost. This can only be received after a new_user message. It is possible to receive this event and not have received a calib_success message. This would indicate that a person was seen but the sensor was not able to fully track them before they were lost. This lack of full skeletal tracking could be because of interference or because the sensor was already skeletal tracking its maximum number of people. After receiving this message, the playerId may be reused by the sensor for a new person.

The following legacy messages are not supported: calib_pose, calib_start, calib_fail, exit_user, and reenter_user.

User Location

User location messages, also known as center of mass, are generated when @skeleton is enabled. Once a person is identified, you will begin receiving user location messages. It is not necessary to have full skeletal tracking as indicated by a calib_success message. This enables you to roughly track a person’s location when there is interference or because the sensor was already skeletal tracking its maximum number of people.

OSC                                       Max native
-----------------------------------------------------------------------------
/user/playerID x y z q                    user playerID x y z q

# Example
/user/2 42.1323 -100.8237 1984.348 0      user 2 42.1323 -100.8237 1984.348 0

playerId: same integer used for a tracked skeleton
x, y, z: floats of the real-world XYZ location of the person’s center of mass relative to the sensor. Their values are influenced by the attributes @distmeter, @flipx, @rotatemethod, and @translate.
q: integer bit field that indicates the quality of the person’s location. It is possible to have no, one, or multiple quality bit flags at the same time. The bit flags are summed together. Quality bit flag values are:
1 = Part of the player’s body is out of frame to the camera’s right.
2 = Part of the player’s body is out of frame to the camera’s left.
4 = Part of the player’s body is out of frame above the camera’s field of view.
8 = Part of the player’s body is out of frame below the camera’s field of view.
Example: quality = 5 indicates the player’s body is out of frame to the camera’s right (1) and above (4). 1 + 4 = 5.

Some sensors may not support user quality output

Skeleton Joints

Skeleton joint data is output when @skeleton, @skeldepth, or @skelcolor are enabled. A sensor may be able to locate more users than it can track joints of those users. For example, the Kinect v1 sensor can locate up to six users yet can only track the skeleton joints for two of them.

Joint data is affected by the other attributes @distmeter, @flipx, @align, @position, @quat, @rotate, @rotatexyz, and @scale.

Joint names, diagrams, output order

Most plugins support the same base list of 20 joints. Some plugin ML models provide additional joints. Some of those additional joints are output by those specific plugins.

Joints have consistent names with all plugins. l_ means left. r_ means right.

	dp.kinect diagram	dp.kinect2 diagram	dp.kinect3 diagram	dp.oak (MoveNet) diagram
l_thumb, r_thumb
l_hand_tip, r_hand_tip
l_hand, r_hand
l_wrist, r_wrist
l_elbow, r_elbow
l_shoulder, r_shoulder
l_foot, r_foot
l_ankle, r_ankle
l_knee, r_knee
l_hip, r_hip
l_ear, r_ear
l_eye, r_eye
nose
head
neck
c_shoulder
torso
waist

Joint output order

Plugins v1.3.20210818 and newer output joints in a reverse hierarchy. This order clarifies joint hierarchy, facilitates puppet animation, and supports additional joints for some ML models.

l_thumb  l_hand_tip  l_hand      l_wrist  l_elbow l_shoulder
r_thumb  r_hand_tip  r_hand      r_wrist  r_elbow r_shoulder
l_foot   l_ankle     l_knee      l_hip
r_foot   r_ankle     r_knee      r_hip
l_ear    l_eye       r_ear       r_eye    nose
head     neck        c_shoulder  torso    waist

Plugins before v1.3.20210818 output joints in this order

waist       torso       neck     head
l_shoulder  l_elbow     l_wrist  l_hand
r_shoulder  r_elbow     r_wrist  r_hand
l_hip       l_knee      l_ankle  l_foot
r_hip       r_knee      r_ankle  r_foot
c_shoulder  l_hand_tip  l_thumb  r_hand_tip  r_thumb

Joint location real-world XYZ coordinates

Real-world skeleton joint locations in meters @distmeter 1 or millimeters @distmeter 0.

# Max native:   @skeleton 1 @skeletonformat 1
skel playerId jointname x y z confidence

# OSC protocol: @skeleton 1
/skel/playerId/jointname x y z confidence

playerId: same integer used for a tracked skeleton
jointname: one of the joint names
x, y, z: floats of the real-world XYZ location of the joint relative to the sensor. These are affected by @distmeter, @flipx, @align, @position, @quat, @rotate, @rotatexyz, and @scale.
confidence: normalized float [0.0..1.0] for the confidence of the joint. 1.0 indicates the highest confidence. This confidence can be used to filter data by your patch or automatically with @posconfidence.

Examples

# Max native:   @skeleton 1 @distmeter 1 @skeletonformat 1
skel 2 r_shoulder -1.204 2.053 3.712 0.5

# OSC protocol: @skeleton 1 @distmeter 1
/skel/2/r_shoulder -1.204 2.053 3.712 0.5

Joint Orientation

Joint orientation output is enabled with @skeleton 2 and its format with @orientformat. Formats available are quaternion or 4x4 matrix, in hierarchical or absolute rotation.

Joint depth-space and color-space UV coordinates

Joint UV coordinate output is available in depth-space @skeldepth and color-space @skelcolor.