Depthsensor

How to use the Depthsensor robot skill

• Jump to comments
• Jump to end

Use Microsoft Kinect, Kinect One, or Asus Xtion Pro to move servos from your body joints.

This ARC plugin enables customizing angle calculations, publishing the calculations to EZ-Script variables and controlling servos. Supported Hardware: Kinect Xbox 360 (Kinect for Windows Runtime 1.8 - https://www.microsoft.com/en-us/download/details.aspx?id=40277) Asus Xtion Pro (OpenNI 2 and NITE 2) Kinect One (Kinect for Windows Runtime 2.0 - https://www.microsoft.com/en-us/download/details.aspx?id=44559)

Additional Setup Notes for Asus Xtion: Please confirm the following Windows environment variables:

OPENNI2_REDIST=C:\Program Files (x86)\OpenNI2\Redist\
NITE2_REDIST=C:\Program Files (x86)\PrimeSense\NiTE2\Redist\
PATH=...;C:\Program Files (x86)\PrimeSense\NiTE2\Redist\;C:\Program Files (x86)\OpenNI2\Redist\;

Create a NITE.INI file with the following content:

[General]
DataDir=C:\Program Files (x86)\PrimeSense\NiTE2\Redist\NiTE2

[Log]
; 0 - Verbose; 1 - Info; 2 - Warning; 3 - Error. Default - None
Verbosity=0
LogToConsole=1
LogToFile=1

save in your ARC location e.g. C:\Program Files (x86)\Synthiam Inc\EZ-Builder\

Plugin Main Area:

Left: The picture panel displays the skeleton tracking. Right: Run-time variables (checked for preview) values Top: Configuration button. Selected Device (Configuration Form) Connect Checkbox (dis)connects the device chosen Servos Attached Checkbox (dis)connects the servos (previously configured)

Configuration Form:

*** Do not forget to press Save Button ***

Expressions:

Active - Enable or disable the expression Preview - if checked, the value is displayed in the main form Publish - the variable is created and updated and can be used with EZ-Scripts Type - There are 4 types of variables: double, Vector2 (X,Y), Vector3(X,Y,Z) and Quaternions (X,Y,Z,W) Expression - C# expression to be compiled, evaluated, and assigned to a variable. Board, Port, Min, Max - When all specified a servo Position command will be executed using the value (double) from the expression.

Grid lines: Drag and drop is allowed Multiple line selection is allowed Right Button presents a context menu option to delete expressions.

Template: There is a default template [R]Angles, press Load template to populate the JD's upper body expressions. Specifying a name allows you to save a Template (Expressions plus last body values). Loading a template will override the existing expressions.

Each Joint has two variables, one for Position and another for Rotation.

Each device returns a specific number of joints (check the device tabs); if not supported, the Joint variables will be zero.

Position is a Vector3(x,y,z) representing the distance from the sensor point of view. The unit values are in meters.

Rotations is a Quaternion(x,y,z,w) is possible to obtain the Euler Angles (roll, pitch, yaw). I'm still working to unify the results, some sensors the rotations are absolute, others are relative.

For each Joint two variables are defined - For SpineBase: SpineBase (Vector3) QSpineBase (Quaternion)

If you want to use SpineBase depth distance, you can use SpineBase.Z

Available EZ-Script Functions: Vector2 ToVector2(double x, double y) Vector3 ToVector3(double x, double y, double z) Vector3 GetEuler(Quaternion q) double AngleBetween(Vector2 v1, Vector2 v2) double AngleBetween(Vector3 v1, Vector3 v2) double Normalize(double value, double start = 0, double end = 360) double Abs(double value) double Min(double a, double b) double Max(double a, double b) double Pow(double a, double b) double Sqrt(double d) double Cos(double a) double Sin(double a) double Tan(double d) double Cosh(double a) double Sinh(double a) double Tanh(double d) double Acos(double d) double Asin(double a) double Atan(double a) double Atan2(double y, double x)

*** Do not forget to press Save Button ***

Simulator

Calculations are evaluated based on the Grid input: Joint's Position values (Px,Py,Pz) Quaternion values (Qx,Qy,Qz,Qw) are only required if they are used in the expressions.

For additional feedback, questions use the following thread:

https://synthiam.com/Community/Forum/Thread?threadId=9289

Unfortunately, there is not enough time. Feel free to point out text/info errors, create a tutorial, or explore the plugin.