Control robot servos using your body and a Microsoft Kinect
How to add the Kinect Body Control robot skill
- Load the most recent release of ARC (Get ARC).
- Press the Project tab from the top menu bar in ARC.
- Press Add Robot Skill from the button ribbon bar in ARC.
- Choose the Camera category tab.
- Press the Kinect Body Control icon to add the robot skill to your project.
Don't have a robot yet?
Follow the Getting Started Guide to build a robot and use the Kinect Body Control robot skill.
How to use the Kinect Body Control robot skill
This Skill enables the control of your robot's servos by moving the joints of your body, which are detected by a Microsoft XBOX 360 Kinect only. Servos can be assigned to each joint using the Settings window. Degrees to move the servos are automatically calculated for joints about the connecting joint. For example, the wrist position in degrees is calculated based on the elbow position. The elbow position in degrees is calculated based on the shoulder position. Each joint can be assigned to control servo positions between 1-180 degrees of your robot.
Required Drivers
The SDK 1.8 can be installed from here, as you will need the libraries it includes: https://www.microsoft.com/en-ca/download/details.aspx?id=40278Main Window
2. Sitting Position Checkbox When this is checked, only the Upper Body servos will be activated, and the lower body will not be detected. As a result, the lower body servos will not be active.
3. Kinect Body Joint Visualization This section shows a visual representation of the joint values that are detected.
4. Joint Data Output This section shows a numerical representation of the joint values that are detected.
Configuration - General Tab
1. Smoothing Value Drop-down This drop-down adjusts the smoothing value for servo movement. As the smoothing value increases, it removes the servo jitter. However, more servo delay will be introduced as the smoothing value increases.
Configuration - Upper Body Tab
1. Upper Body servo Configuration This section of six servos allows you to set up the servo movement parameters for the upper body, such as servo port, board index, servo configuration, max/min degree limitations, multiple servo control, and inverted direction.
Configuration - Lower Body Tab
1. Lower Body servo Configuration This section of 6 servos allows you to set up the servo movement parameters for the Lower body, such as servo port, board index, servo configuration, max/min degree limitations, multiple servo control, and inverted direction.
*Note: The Lower Body will not be detected when the Sitting Position checkbox is enabled (in the Main Window).
How to Use Kinect Body Control
Download and install the Kinect SDK (link in the "Required Drivers" section above).
Plug your Kinect sensor into wall power and a USB port on your PC.
Add the Kinect Body Control skill to your ARC project (Project -> Add Skill -> Misc -> Kinect Body Control).
Use the Kinect Body Control Configuration tabs to set up any servos you want to move.
Connect ARC to the servos/robot you want to control.
Ensure your body is at least 3 feet from the Kinect sensor to be detected (the ideal distance is 6 feet) and start moving around.
If you are experiencing servo jitter, try adjusting the Smoothing value in the Configuration General tab.
Video
Requirements
Kinect Compatibility We're unsure what Kinects work with the SDK in Windows due to limited information published by Microsoft (or at least hard-to-find information). There are multiple Kinect versions; the one we use is the Xbox 360 Kinect.Control Commands
There are two control commands for pausing and unpausing the Skill.- "PauseOn" will check the pause checkbox
- "PauseOff" will uncheck the pause checkbox
Work In Progress
- The servos for legs have not been implemented yet
- The shoulder rotation servos have not been implemented yet
@DJ Nice plugin!... I actually have an asus xtion so in the morning I will see if it will work with your kinect plugin....
The plugin is for Microsoft xBox Kinect
@DJ Totally understand that... but the xtion is a very close clone to the kinect... It might actually work...just sayin'. If it doesn't it's ok... The xtion has been discontinued anyway, I believe...
It won't because the drivers aren't compatible. The only similarities between the two devices is their intended use-case. I don't want to give you any false hope
It'd be different if the Windows OS had a built-in API that standardized these device types. Similar to how a camera (of any manufacturer) is compatible because they're built with drivers to fullfill API requirements. This allows a standard for any video hardware to connect and become a camera in the Windows OS. However, with the kinect, the driver is specific and actually a PITA because large SDK files have to be downloaded and installed, etc...
@DJ... I appreciate that and no worries... Funny thing is the timing couldn't have been better... I just finished building the second heavy duty robot arm.... The plugin would work perfectly with them....
If it doesn't work I will eventually pick up a Kinect anyway...
Stay tuned as I should have a video of my BaristaBot within the next week or so...
looking forward to seeing that!
as for the kinect - don't get an xbox kinect. Get the Kinect For Windows. At least, that's what i've been reading. I'm under the impression that the xbox kinect won't work outside of a development environment (i.e. visual studio).
Maybe @ptp has some input on that subject, if he sees this post
Noted... Thanks
Ops, cancel that - looks like the Kinect For Windows is discontinued or being discontinued: https://www.engadget.com/2015/04/02/rip-kinect-for-windows/
But i have an xBox 360 Kinect, and it doesn't work with latest drivers. So word on the street is to get the xBox One Kinect
@DJ
@DJ, @Richard
It's not forgotten but the scope got bigger and i was waiting to get a alpha version.
I'll look to my code base, and i'll post something during the weekend.
Any news about Kinect V2 support?
https://synthiam.com/Products/Controls/
Depth Sensor (Kinect V1, Kinect V2, Asus): https://synthiam.com/redirect/legacy?table=plugin&id=173
Nice this works
Body control would still be nice
it just needs to be opdated to take kinect20.dll too with kinect10.dll...
From your perspective (the user) it seems a simple change swap kinect10.dll with kinect20.dll and voila!
The reality is different: Kinect 2.0 has a different API, you will need:
All the above is only possible if you have the real stuff (Kinect V2).
I have installed the plugin and installed the right kinect software also. (kinect 1, Windows 10)
The kinect works when I start the Microsoft Kinect software: I can see camera vision and movements.
When I try to use the EZ-robot with kinect plugin, I can connect the EZ-Robot software to the kinect (it's is found by the ez-robot software), but then it ends. No red wire movements or whatever.
I got some info on the plugin-window once, but didn't got that also anymore.
Any help would be nice !
With kind regards,
;-) Arthur
Hi Arthur - did you install the driver specified in the description above? I believe it is 1.8, and the link is included above. What do you mean by "then it ends"? Are you meaning the computer shuts down or ARC crashes? I'll need as much information as you can tell me to help
Also, @ptp made a much better plugin that you can find here: https://synthiam.com/Software/Manual/DepthSensor-15873
Oh that's right I forgot there was a Kinect plugin ,I have one Kinect camera from Xbox 360 but it has the funny USB connector that won't fit standard computer USB ports. I guess I need to buy the special adapter to fit on computers,I saw them cheap on Ebay and Amazon. Not sure if it will work with EZ software but I do see articles online how to get it working on any windows 10 Laptops so will try both ways as the camera only cost me 13 dollars at one of those hock shops ,it looks brand new and hopefully will power up when I get the special USB adapter to Laptop kit.
Do I have to install the Developer tool kit, because I do not see any lines in the plugin box when i move around.
Did you install the stuff that is mentioned in the plugin description?
Yes I installed the 1.8sdk and the plugin for ARC
Great - then long as you’re using the correct 360 Kinect, it works as instructed in the video.
@Autiboy, just double checking, do you have the power supply for the Kinect plugged in? It cannot work with just USB.
Yes I have the usb ac power supply for the xbox 360. The problem is that I cannot see the xbox 360's camera in ARC when I move.
In the demo video above, what part is not working for you? You never see the 360’s camera output. It’s expected to see your skeleton when standing in front of the Kinect 360.
Next Friday I will demo how to use the Kinect with ARC.
Thanks DJ That would be nice. Also the 2nd Fridays video does not work did you not stream that day?
Nice app. Works great. When will you inplement the shoulder rotation servos then it would be ideal for my inmoov.
My Kinect module is showing this below:
I am fully updated, windows 10 also
what can I do?
sorry, I do not have the Microsoft xBox 360 -
You can follow the instructions above to install the appropriate drivers. And ensure your connect is a compatible version
Thanks,
I am following the instructions. I did install the libraries - https://www.microsoft.com/en-ca/download/details.aspx?id=40278
no errors now
All I need is the Microsoft xBox 360 Kinect only
I am looking around now for a Microsoft xBox 360 Kinect only - not hard to find
Have a great New Year! 2020
Received my Xbox 360 Kinect only - works well, thanks
Right on! That’s super cool
Thanks Dj and Jeremie
Almost 2020 soon - sounds futuristic, lol
how did this thread get here it is from Dec, 2019
It’s not a thread. It’s a robot skill and it was updated. You responded to an update for a robot skill.
got it, thanks Dj
I did the Kinect Body Control earlier
wee remote - xboz 360
more tomorrow :-)
EzAng
I am trying to teach my humanoid to use this but struggling. The shoulder rotary doesn't seem to work, the hands show up but can't link to servo, The lower body is labeled shoulder elbow same as upper body.
Doesn't seem like those are developed...
Updated to include a ControlCommand() to pause and unpause the robot skill
works great, thanks