dp.oak

dp.oak is a plugin for the Cycling '74 Max development environment to use the Luxonis OAK sensor with a Windows computer.

Quick jump

Get started
Learn

Get started

Download, trial, license

Get a download, trial, or license from the store.

System requirements

Cycling74 Max requirements
Luxonis OAK requirements vary with models
Windows 10 April 2018 (Version 1803, OS Build 17134) release (64-bit) or newer

The Luxonis OAK sensor does not have a shared system driver that all applications use. It instead has all the needed files in the same folder as the plugin. I have split the downloads into two groups:

dp.oak plugin download with all features except body tracking. ~40 MB download
Body tracking neural net files that change less often. ~18 MB download

Setup

Install Cycling’74 Max version 6.1.9 or newer (64-bit)
Download dp.oak from https://hidale.com/shop/dp-oak/
Decompress all the files in this download into an empty folder
Register dp.oak

Optional body tracking

This neural net enables body tracking for six people.

Download the neural net model for body tracking
- Open the dp.oak help file, click on the body joints tab, click download
- Or manually from https://github.com/diablodale/model-zoo/releases/latest/download/movenet_multipose_lightning_1_int32_op15_H160_W256.onnx
Place the model in the same folder as the dp.oak plugin

Register

Visit https://hidale.com/shop/dp-oak/ to get a no-cost license for pre-production releases.
Open the help file or patch included within the dp.oak download.
Click the top-left Register tab.
Ignore the trial button.
Type your registration name in the field beside the register button. You received your registration name in an email from the online store. 🧐 Your registration name is usually not your email address.
Click the register button and use the dialog box that appears to select your dp.oak registration key (.dpreg file)
You should see a successful registration. If not, please look at the Max console for any warnings or errors.

Depth technology

Depth technology of most OAK sensors is similar to stereo vision of humans. This is different than infrared-laser technology in sensors like Microsoft Kinect. Only OAK Pro sensors have infrared laser dot projectors.

Power over ethernet

Power-over-Ethernet (PoE) devices are connected with an ethernet cable instead of a short 3-meter USB cable. The ethernet connection can be a great distance, even in a different building.

Luxonis makes several OAK PoE sensor models. Some have infrared laser dot projectors to work in darkness. Others have weatherproof enclosures.

PoE devices need more time to be detected, startup, shut down, and restart. It is not unusual to need 5-8 seconds to find a PoE sensor. It is not unusual to need 20 seconds after stopping a PoE sensor for that same PoE sensor to be ready again.

Known issues

Luxonis PoE sensors may not meet some quality requirements
PoE can’t read data
PoE fails after 6 connections
PoE overloaded on CSS vpu
Each time you start Max and load dp.oak, it may have extra lag/latency on the depth and color cameras. Stopping and restarting dp.oak often removes the lag. Cause under investigation.

Learn

Use search

at the top or all below to find tutorials, features, attributes, methods, and more.

Tutorials

Skeleton Joint Changes Color: Learn how to get the real-world XYZ position of your left hand relative ...
View Sensor Depthmap: Learn how to view depth data (depthmap) as a monochrome image. A depthma...

Attributes

@accel: Acceleration XYZ vector usually pointing towards gravity (where 1 unit =...
@align: Align data to the perspective of the depth or color camera. This affects...
@anim: Parent jit.anim.node name adds plugin to jit.anim.node hierarchy.
@colorexposure: Color camera exposure in microseconds (µs) and ISO sensitivity.
@colorfocus: Color camera focus position; automatic or manual.
@colormap: Color image output in choice of pixel format on the second outlet.
@colormapres: Resolution of color image that was enabled with @colormap
@colortype: Color component in 8-bit unsigned integer or 32-bit normalized float. Th...
@decodercolor: Decoder for color images (e.g. MJPEG sensor data). Deprecated.
@depthconf: Filter depth pixels by the normalized confidence of their depth value. T...
@depthfps: Depth camera native capture frame rate expressed as frames per second (H...
@depthmap: Depth output on the first outlet. This combined with resolution @depthma...
@depthmapres: Resolution of depth image that was enabled with @depthmap.
@depthvis: Depth visibility filter removes double images and edge artifacts in imag...
@distmeter: Distance output in meters 1 or millimeters 0. This affects all distance ...
@flipx: Flip or negate X-axis values of output (matrix images, joint positions, ...
@floorout: Calculate the floor plane on each bang, update the @floor attribute, and...
@hardware: Identify/select sensor hardware model. It is read-only on some plugins. ...
@idsensor: Bind specific sensor to the plugin. If @idsensor is not set when the plu...
@indicator: Enable streaming/recording indicator (e.g. LED) on the sensor. Not all s...
@irdot: Infrared (IR) laser dot projector brightness normalized to [0.0..1.0].
@irflood: Infrared (IR) flood light brightness normalized to [0.0..1.0].
@irgamma: Apply gamma correction to infrared image enabled with @irmap. Use to bri...
@irmap: Infrared (IR) image output on the third outlet. This, resolution @irmapr...
@irtype: Infrared (IR) values in 8-bit unsigned integer, long signed integer, or ...
@mlcompute: Machine learning compute engine and device. Enable multiple compute devi...
@mlcrop: Crop the machine learning input tensor. This overrides the default behav...
@mlinput: Machine learning input by source, FourCC, and shape. The default @mlinpu...
@mlmean: Subtract values from input channels before scaling: (pixel_value - mean)...
@mlmodel: Enable machine learning and run inference with the given ONNX model on t...
@mlscale: Scale input values after subtracting mean: (pixel_value - mean) * scale.
@mlswapch: Swap 3-channel ml input tensor channels 0 and 2 before inference.
@monoexposure: Monochrome camera exposure in microseconds (µs) and ISO sensitivity.
@mount: Mount rotation of sensor in clockwise degrees; may optimize ai models li...
@name: Unique plugin instance name. Other Max objects can bind to this name.
@opencl: OpenCL device for image processing (e.g. flipx, undistort, align, etc.) ...
@orientformat: Skeleton joint orientation as quaternion, 4x4 matrix, hierarchical, or a...
@pipeline: Sensor data transforms and output can be multi-threaded and triggered wi...
@pointcloud: Format of pointcloud output as three or four floating point values.
@posconfidence: Filter skeleton body, joints, and face by their confidence. This confide...
@position: Offset the origin of sensor data to this XYZ value. That is, translate s...
@powerfreq: Reduce video banding and flicker related to regional power frequency.
@quat: Rotate data by a quaternion in the form: i j k w. This is always synchro...
@register: Register a license or get the status of registration.
@rotate: Rotate data by an angle (degrees) around an XYZ vector in the form: angl...
@rotatemethod: Rotate data to compensate for gravity. This compensation is combined wit...
@rotatexyz: Rotate data by Euler angles (degrees) around the positive x, y, and z ax...
@scale: Scale the sensor data by this XYZ value. That is, multiply sensor data a...
@skelcompute: Skeleton tracking compute engine and device. Enable multiple compute dev...
@skeleton: Skeleton, joint, and user tracking output on the fifth outlet. Output fo...
@skeletonformat: Message format for skeleton joints, users, faces, etc. Native Max messag...
@skelmodel: Machine learning model used for skeleton and joint tracking
@smoothing: Smoothing for skeleton joint values with either a simplified one paramet...
@timestamp: Timestamps for skeleton and matrix data as microseconds since the Epoch.
@transcoder: Transcoder for image formats, e.g. transcoding MJPEG sensor data to colo...
@type: Depthmap value in signed integer, 32-bit float, or 64-bit float. This ty...
@undistort: Undistort the depth, color, infrared, and playermap output. The undistor...
@undistortscale: Scale factor for undistortion and cropping of unwanted/invalid pixels wh...
@unique: Unique-only matrices and messages; bang may not output when no changes. ...
@verbose: Verbose warnings and errors output to the Max Window for troubleshooting...
@whitebalance: White balance in degrees Kelvin temperature to adjust for lighting condi...

Methods

bang: Query the plugin for updated data. Then, output matrices and messages fr...
close: Close and teardown the currently open connection to a sensor.
firmware: Firmware version query and update for supported sensor devices.
getports: Get status of sensor ports; like those for synchronization.
getusbidlist: Output supported sensor identifiers on the dumpout outlet. These identif...
open: Open and initialize the connection to a sensor. When @idsensor has not b...
unregister: Unregister and remove a license.