dp.kinect3

dp.kinect3 is a plugin for the Cycling '74 Max development environment to use the Microsoft Kinect v3 (Azure Kinect) with a Windows computer.

Quick jump

Get started
Learn

Get started

Download, trial, license

Get a download, trial, or license from the store.

System requirements

Cycling74 Max requirements
Microsoft Azure Kinect requirements
Windows 10 April 2018 (Version 1803, OS Build 17134) release (64-bit) or newer

The Azure Kinect sensor does not have a shared system driver from Microsoft. Microsoft requires customers and applications to install independent copies of Kinect drivers and files. There are three main groups:

dp.kinect3 plugin download with all features except body tracking. ~19 MB download
Microsoft body tracking files that Microsoft creates and tests. ~430 MB download
Optional Nvidia files to run body tracking with CUDA and cnDNN apis. ~4 GB download

Setup

Install Cycling’74 Max version 6.1.9 or newer (64-bit)
Download dp.kinect3 from https://hidale.com/shop/dp-kinect3/
Decompress all the files in this download into an empty folder
Register dp.kinect3

Optional body tracking

Download and install Microsoft’s Body Tracking SDK 1.1.2. Note the install folder; often C:\Program Files\Azure Kinect Body Tracking SDK
Navigate into the SDK install folder, then the tools subfolder, e.g. C:\Program Files\Azure Kinect Body Tracking SDK\tools

Copy these files from sdk\windows-desktop\amd64\release\bin to your dp.kinect3 folder

k4abt.dll
dnn_model_2_0_lite_op11.onnx
dnn_model_2_0_op11.onnx      <- slightly more accurate

Optional CUDA and TensorRT body tracking

Body tracking with Nvidia GPUs already works well with the default DirectML provider. CUDA and TensorRT are optional providers for body tracking. They are for unusual situations. They performed no better than DirectML during testing. Please note, the download size is ~4 GB.

Setup body tracking
Download the optional-nvidia-addons.zip from https://hidale.com/shop/dp-kinect3/#download

Copy these files within the ZIP download to the dp.kinect3 folder

onnxruntime_providers_shared.dll
onnxruntime_providers_cuda.dll
zlibwapi.dll

Download Nvidia CUDA v11.4.4 for Windows x64. Typical filename cuda_11.4.4_472.50_windows

Copy these CUDA files from bin folder to the dp.kinect3 folder

cudart64_110.dll
cufft64_10.dll
cublas64_11.dll
cublasLt64_11.dll

Download Nvidia cuDnn v8.4.1.50 for CUDA 11.4-11.6 for Windows x64. Typical filename cudnn-windows-x86_64-8.4.1.50_cuda11.6

Copy these cuDNN files from bin to the dp.kinect3 folder

cudnn64_8.dll
cudnn_cnn_infer64_8.dll
cudnn_ops_infer64_8.dll

For TensorRT, follow these additional steps

Copy these additional files within the optional-nvidia-addons.zip download to the dp.kinect3 folder
```
onnxruntime_providers_tensorrt.dll
```
Copy these additional CUDA files from bin folder to the dp.kinect3 folder
```
nvrtc64_112_0.dll
nvrtc-builtins64_114.dll
```
Download Nvidia TensorRT v8.4.1.5 for CUDA 11.4-11.6 and cuDNN 8.4 for Windows x64. Typical filename TensorRT-8.4.1.5.Windows10.x86_64.cuda-11.6.cudnn8.4

Copy these Tensor Runtime files from lib to the dp.kinect3 folder

nvinfer.dll
nvinfer_plugin.dll
nvinfer_builder_resource.dll
nvonnxparser.dll

Register

Evaluate dp.kinect3 with a 14-day no-cost trial. When the plugin meets your needs, please visit https://hidale.com/shop/dp-kinect3/ to purchase a license from the store.
Open the help file or patch included within the dp.kinect3 download.
Click the top-left Register tab.
Optional: Start a 14-day no-cost trial by clicking register trial then skip to the last step.
Type your registration name in the field beside the register button. You received your registration name in an email from the online store. 🧐 Your registration name is usually not your email address.
Click the register button and use the dialog box that appears to select your dp.kinect3 registration key (.dpreg file)
You should see a successful registration. If not, please look at the Max console for any warnings or errors.

Learn

Use search

at the top or all below to find tutorials, features, attributes, methods, and more.

Tutorials

Skeleton Joint Changes Color: Learn how to get the real-world XYZ position of your left hand relative ...
View Sensor Depthmap: Learn how to view depth data (depthmap) as a monochrome image. A depthma...

Attributes

@accel: Acceleration XYZ vector usually pointing towards gravity (where 1 unit =...
@align: Align data to the perspective of the depth or color camera. This affects...
@anim: Parent jit.anim.node name adds plugin to jit.anim.node hierarchy.
@colorexposure: Color camera exposure in microseconds (µs) and ISO sensitivity.
@colorfocus: Color camera focus position; automatic or manual.
@colormap: Color image output in choice of pixel format on the second outlet.
@colormapres: Resolution of color image that was enabled with @colormap
@colortype: Color component in 8-bit unsigned integer or 32-bit normalized float. Th...
@decodercolor: Decoder for color images (e.g. MJPEG sensor data). Deprecated.
@depthconf: Filter depth pixels by the normalized confidence of their depth value. T...
@depthdelay: Depth capture delay (microseconds) relative to color capture.
@depthfps: Depth camera native capture frame rate expressed as frames per second (H...
@depthmap: Depth output on the first outlet. This combined with resolution @depthma...
@depthmapres: Resolution of depth image that was enabled with @depthmap.
@depthonlyplayers: Filter source by tracked players. Deprecated. Use @onlyplayers.
@depthvis: Depth visibility filter removes double images and edge artifacts in imag...
@distmeter: Distance output in meters 1 or millimeters 0. This affects all distance ...
@flipx: Flip or negate X-axis values of output (matrix images, joint positions, ...
@floorout: Calculate the floor plane on each bang, update the @floor attribute, and...
@hardware: Identify/select sensor hardware model. It is read-only on some plugins. ...
@idsensor: Bind specific sensor to the plugin. If @idsensor is not set when the plu...
@indicator: Enable streaming/recording indicator (e.g. LED) on the sensor. Not all s...
@irdot: Infrared (IR) laser dot projector brightness normalized to [0.0..1.0].
@irflood: Infrared (IR) flood light brightness normalized to [0.0..1.0].
@irgamma: Apply gamma correction to infrared image enabled with @irmap. Use to bri...
@irmap: Infrared (IR) image output on the third outlet. This, resolution @irmapr...
@irtype: Infrared (IR) values in 8-bit unsigned integer, long signed integer, or ...
@mlcompute: Machine learning compute engine and device. Enable multiple compute devi...
@mlcrop: Crop the machine learning input tensor. This overrides the default behav...
@mlinput: Machine learning input by source, FourCC, and shape. The default @mlinpu...
@mlmean: Subtract values from input channels before scaling: (pixel_value - mean)...
@mlmodel: Enable machine learning and run inference with the given ONNX model on t...
@mlscale: Scale input values after subtracting mean: (pixel_value - mean) * scale.
@mlswapch: Swap 3-channel ml input tensor channels 0 and 2 before inference.
@mount: Mount rotation of sensor in clockwise degrees; may optimize ai models li...
@name: Unique plugin instance name. Other Max objects can bind to this name.
@onlyplayers: Filter source by tracked players. Filtered data is zero-like. Possible v...
@opencl: OpenCL device for image processing (e.g. flipx, undistort, align, etc.) ...
@orientformat: Skeleton joint orientation as quaternion, 4x4 matrix, hierarchical, or a...
@pipeline: Sensor data transforms and output can be multi-threaded and triggered wi...
@playermap: Player indices for each pixel output on the fourth outlet. Pixels are a ...
@pointcloud: Format of pointcloud output as three or four floating point values.
@posconfidence: Filter skeleton body, joints, and face by their confidence. This confide...
@position: Offset the origin of sensor data to this XYZ value. That is, translate s...
@powerfreq: Reduce video banding and flicker related to regional power frequency.
@quat: Rotate data by a quaternion in the form: i j k w. This is always synchro...
@register: Register a license or get the status of registration.
@rotate: Rotate data by an angle (degrees) around an XYZ vector in the form: angl...
@rotatemethod: Rotate data to compensate for gravity. This compensation is combined wit...
@rotatexyz: Rotate data by Euler angles (degrees) around the positive x, y, and z ax...
@scale: Scale the sensor data by this XYZ value. That is, multiply sensor data a...
@skelcompute: Skeleton tracking compute engine and device. Enable multiple compute dev...
@skeleton: Skeleton, joint, and user tracking output on the fifth outlet. Output fo...
@skeletonformat: Message format for skeleton joints, users, faces, etc. Native Max messag...
@skelmodel: Machine learning model used for skeleton and joint tracking
@smoothing: Smoothing for skeleton joint values with either a simplified one paramet...
@syncdelay: Synchronization delay (microseconds) on a sync subordinate sensor. Sets ...
@syncmode: Select sensor’s role (main, subordinate) and medium (e.g. wired) in a sy...
@timestamp: Timestamps for skeleton and matrix data as microseconds since the Epoch.
@transcoder: Transcoder for image formats, e.g. transcoding MJPEG sensor data to colo...
@type: Depthmap value in signed integer, 32-bit float, or 64-bit float. This ty...
@undistort: Undistort the depth, color, infrared, and playermap output. The undistor...
@undistortscale: Scale factor for undistortion and cropping of unwanted/invalid pixels wh...
@unique: Unique-only matrices and messages; bang may not output when no changes. ...
@verbose: Verbose warnings and errors output to the Max Window for troubleshooting...
@whitebalance: White balance in degrees Kelvin temperature to adjust for lighting condi...

Methods

bang: Query the plugin for updated data. Then, output matrices and messages fr...
close: Close and teardown the currently open connection to a sensor.
firmware: Firmware version query and update for supported sensor devices.
getports: Get status of sensor ports; like those for synchronization.
getusbidlist: Output supported sensor identifiers on the dumpout outlet. These identif...
open: Open and initialize the connection to a sensor. When @idsensor has not b...
unregister: Unregister and remove a license.