Thumbnail

Kinect Xbox 360 Depth Mapping

by Microsoft

Use the microsoft xbox 360 connect to send navigation/obstacle information into the ARC NMS (navigation messaging system)

Requires ARC v3 (Updated 2/15/2022)

How to add the Kinect Xbox 360 Depth Mapping robot skill

  1. Load the most recent release of ARC (Get ARC).
  2. Press the Project tab from the top menu bar in ARC.
  3. Press Add Robot Skill from the button ribbon bar in ARC.
  4. Choose the Navigation category tab.
  5. Press the Kinect Xbox 360 Depth Mapping icon to add the robot skill to your project.

Don't have a robot yet?

Follow the Getting Started Guide to build a robot and use the Kinect Xbox 360 Depth Mapping robot skill.

How to use the Kinect Xbox 360 Depth Mapping robot skill

Use the microsoft xbox 360 connect to send navigation/obstacle information into the ARC NMS (navigation messaging system). This skill publishes obstacle scan distance data to ARC's NMS (navigation messaging system) to be used in SLAM or The Navigator skills.

Because the entire image depth is unnecessary to be processed for mapping, only a portion is. This prevents your robot from picking up distances of things irrelevant to its size and distance. Using the options for the detection area, tune the pink box to the area of detection.

Required Drivers

The SDK 1.8 can be installed from here, as you will need the libraries it includes: https://www.microsoft.com/en-ca/download/details.aspx?id=40278

*Note: under development (see To Do)

User-inserted image

To Do - additional testing to verify distance from point cloud is calculated correctly

  • compensate for distance with camera angle
  • add variables (min, max depth & sound direction)
  • move detect rectangle into config menu

ARC Pro

Upgrade to ARC Pro

Don't limit your robot's potential – subscribe to ARC Pro and transform it into a dynamic, intelligent machine.

Portugal
#1  

Nice! Must get me a Kinect.

PRO
Synthiam
#2  

So far my test results aren’t that great. I think the intel depth camera is the way to go ... or a 360 degree lidar

Portugal
#3  

Yes, the lidar seems to be the logic choice with the T265. It way less recourse hungry.

Portugal
#4   — Edited

Still waiting for my T265............

PRO
Synthiam
#5  

That t265 is super great. Wish they were cheaper and more of them lol

PRO
Canada
#6  

I'm very interested in this skill!

I really like the idea of using Kinect sensors on robots because you can find them for pretty cheap these days. They are usually around $10-15 CAD at second-hand stores and sites.

In the future will this skill be able to move the Kinect pan/tilt motors?

Has anyone tried powering the Kinect's 12V line with a lower voltage? like 11.1V?

PRO
Synthiam
#7   — Edited

There’s no advantage to moving the motors, if they exist in the model. It would decrease the accuracy of detected obstacles degrees/distance by the Kinect

Portugal
#8  

I don't have a kinect, but would love to see this skill in action. A video about it would be great.

PRO
Canada
#9   — Edited

@DJ I have to disagree with you, having the pan/tilt would be highly advantageous!

You could map at different vertical levels or do mapping while in a stationary position. Say you have a robot that doesn’t move geographically (like an arm) but the environment changes around it, it would be very helpful to do a quick scan within the range of the Kinect sensor

The onboard motors can also help with the camera view itself (excluding the IR depth sensing). I would much rather use the motors inside the Kinect to look around instead of having to install my own servos.

Lastly, you could use the motors are they were originally intended as well! Use them for human body scanning with the Kinect Body Control skill.

PRO
Synthiam
#10  

Again, there's no way to identify the degrees for the NMS. Please read how the NMS works and there's some additional reading about the accuracy of angles for correct pose navigation.

PRO
Canada
#11  

I thought I was in a 70's disco. Think my kinect is really badly calibrated. Maybe needs a calabration checkerboard routine.

PRO
Synthiam
#12  

The Kinect has no calibration - as it’s a camera. And the camera has a lens. And the camera has a resolution. And that pixel resolution with the lens angle gives you angles of distances in degrees.

it’s not great for mapping, as you probably just discovered. What you’re experiencing are why these technologies are often abandoned for robotics. New technologies are being made to replace this old stuff.

the intel depth camera I think will be good. They have a lidar too. Might have to check those out one day

PRO
Canada
#13   — Edited

Nicolos Burrus had written some code to software calibrate the kinect with a checkerboard map to align the  speckled laser depth image and camera, when you get it right it creates a fairly good 3D Slam. http://nicolas.burrus.name/index.php/Research/KinectCalibration looks like latest versions do auto calibration been years since I played with a kinect    Hmm you have to cut and paste url or change https to http, looks like it changes to https when you click on it

PRO
Synthiam
#14  

Hmmm - I think that's for the first-generation Kinect, not the box 360 version

PRO
Canada
#15  

I know primesense had sold their chip in a number of devices (Asus xtion, eyeplay etc) but from a "Kinect" standpoint the XBOX 360 had the first Kinect version.