@skeleton attribute
🔌dp.kinect dp.kinect2 dp.kinect3 dp.oak
- signature
skeleton INT- values
0- off default
1- joints
2- joints and orientation
- examples
@skeleton 1
Skeleton, joint, and user tracking output on the fifth outlet.
Output format (native Max messages, OSC protocol, etc.) is determined by
@skeletonformat.
📝 Some sensors may not support orientation output
@skeleton 2.
User Identification
User identification messages are generated by enabling @skeleton. The data provided is an
integer playerID which is a unique identifier representing the currently tracked/lost person.
This playerID is unique within a given point of time and for a person seen by the sensor. Its
minimum value is 1 and often 6 is the most people a sensor can detect. Keep in mind that
some sensors support detecting more than six people. This same playerID is used for
matching/connecting with other sensor data like the person’s location, joints,
depthmap pixels, and sound location/association.
The three messages output are
OSC Max native
-------------------------------------------------
/new_user playerID new_user playerID
/calib_success playerID calib_success playerID
/lost_user playerID lost_user playerID
# Examples
/new_user 2 new_user 2
/calib_success 2 calib_success 2
/lost_user 2 lost_user 2
new_user Indicates a new person has been identified but full skeleton tracking is not yet available.
This is the first message in the possible progression of events for a given playerId.
calib_success Indicates the person’s skeleton joints are now being fully tracked. This can only be
received after a new_user message.
lost_user Indicates the previously identified/tracked person has been lost. This can only be received
after a new_user message. It is possible to receive this event and not have received a calib_success
message. This would indicate that a person was seen but the sensor was not able to fully track them before
they were lost. This lack of full skeletal tracking could be because of interference or because the sensor
was already skeletal tracking its maximum number of people. After receiving this message, the playerId may
be reused by the sensor for a new person.
The following legacy messages are not supported: calib_pose, calib_start, calib_fail, exit_user, and reenter_user.
User Location
User location messages, also known as center of mass, are generated when @skeleton is enabled.
Once a person is identified, you will begin receiving user location messages. It is not necessary
to have full skeletal tracking as indicated by a calib_success message. This enables you to
roughly track a person’s location when there is interference or because the sensor was already
skeletal tracking its maximum number of people.
OSC Max native
-----------------------------------------------------------------------------
/user/playerID x y z q user playerID x y z q
# Example
/user/2 42.1323 -100.8237 1984.348 0 user 2 42.1323 -100.8237 1984.348 0
- playerId
- same integer used for a tracked skeleton
- x, y, z
- floats of the real-world XYZ location of the person’s center of mass
relative to the sensor. Their values are influenced by the attributes
@distmeter,@flipx,@rotatemethod, and@translate. - q
- integer bit field that indicates the quality of the person’s location. It is possible to
have no, one, or multiple quality bit flags at the same time. The bit flags are summed together.
Quality bit flag values are:
1 = Part of the player’s body is out of frame to the camera’s right.
2 = Part of the player’s body is out of frame to the camera’s left.
4 = Part of the player’s body is out of frame above the camera’s field of view.
8 = Part of the player’s body is out of frame below the camera’s field of view.
Example: quality = 5 indicates the player’s body is out of frame to the camera’s right (1) and above (4). 1 + 4 = 5.📝 Some sensors may not support user quality output
Skeleton Joints
Skeleton joint data is output when @skeleton, @skeldepth, or @skelcolor are enabled.
A sensor may be able to locate more users than it can track joints of those users. For example,
the Kinect v1 sensor can locate up to six users yet can only track
the skeleton joints for two of them.
Joint data is affected by the other attributes @distmeter, @flipx, @align,
@position, @quat, @rotate, @rotatexyz, and @scale.
Joint names, diagrams, output order
Most plugins support the same base list of 20 joints. Some plugin ML models provide additional joints. Some of those additional joints are output by those specific plugins.
Joints have consistent names with all plugins. l_ means left. r_ means right.
| dp.kinect diagram |
dp.kinect2 diagram |
dp.kinect3 diagram |
dp.oak (MoveNet) diagram |
|
|---|---|---|---|---|
| l_thumb, r_thumb | ✔️ | ✔️ | ||
| l_hand_tip, r_hand_tip | ✔️ | ✔️ | ||
| l_hand, r_hand | ✔️ | ✔️ | ✔️ | |
| l_wrist, r_wrist | ✔️ | ✔️ | ✔️ | ✔️ |
| l_elbow, r_elbow | ✔️ | ✔️ | ✔️ | ✔️ |
| l_shoulder, r_shoulder | ✔️ | ✔️ | ✔️ | ✔️ |
| l_foot, r_foot | ✔️ | ✔️ | ✔️ | |
| l_ankle, r_ankle | ✔️ | ✔️ | ✔️ | ✔️ |
| l_knee, r_knee | ✔️ | ✔️ | ✔️ | ✔️ |
| l_hip, r_hip | ✔️ | ✔️ | ✔️ | ✔️ |
| l_ear, r_ear | ✔️ | ✔️ | ||
| l_eye, r_eye | ✔️ | ✔️ | ||
| nose | ✔️ | ✔️ | ||
| head | ✔️ | ✔️ | ✔️ | ✔️ |
| neck | ✔️ | ✔️ | ✔️ | ✔️ |
| c_shoulder | ✔️ | ✔️ | ✔️ | |
| torso | ✔️ | ✔️ | ✔️ | ✔️ |
| waist | ✔️ | ✔️ | ✔️ | ✔️ |
Joint output order
Plugins v1.3.20210818 and newer output joints in a reverse hierarchy. This order clarifies joint hierarchy, facilitates puppet animation, and supports additional joints for some ML models.
l_thumb l_hand_tip l_hand l_wrist l_elbow l_shoulder
r_thumb r_hand_tip r_hand r_wrist r_elbow r_shoulder
l_foot l_ankle l_knee l_hip
r_foot r_ankle r_knee r_hip
l_ear l_eye r_ear r_eye nose
head neck c_shoulder torso waist
Plugins before v1.3.20210818 output joints in this order
waist torso neck head
l_shoulder l_elbow l_wrist l_hand
r_shoulder r_elbow r_wrist r_hand
l_hip l_knee l_ankle l_foot
r_hip r_knee r_ankle r_foot
c_shoulder l_hand_tip l_thumb r_hand_tip r_thumb
Joint location real-world XYZ coordinates
Real-world skeleton joint locations in meters @distmeter 1 or millimeters @distmeter 0.
# Max native: @skeleton 1 @skeletonformat 1
skel playerId jointname x y z confidence
# OSC protocol: @skeleton 1
/skel/playerId/jointname x y z confidence
- playerId
- same integer used for a tracked skeleton
- jointname
- one of the joint names
- x, y, z
- floats of the real-world XYZ location of the joint relative to the sensor.
These are affected by
@distmeter,@flipx,@align,@position,@quat,@rotate,@rotatexyz, and@scale. - confidence
- normalized float
[0.0..1.0]for the confidence of the joint.1.0indicates the highest confidence. This confidence can be used to filter data by your patch or automatically with @posconfidence.
Examples
# Max native: @skeleton 1 @distmeter 1 @skeletonformat 1
skel 2 r_shoulder -1.204 2.053 3.712 0.5
# OSC protocol: @skeleton 1 @distmeter 1
/skel/2/r_shoulder -1.204 2.053 3.712 0.5
Joint Orientation
Joint orientation output is enabled with @skeleton 2 and its format with @orientformat.
Formats available are quaternion or 4x4 matrix, in hierarchical or absolute rotation.
Joint depth-space and color-space UV coordinates
Joint UV coordinate output is available in depth-space @skeldepth
and color-space @skelcolor.