Kinect Xbox 360 Depth Mapping

How to use the Kinect Xbox 360 Depth Mapping robot skill

Turns a Microsoft Xbox 360 Kinect into an obstacle scanner for ARC's Navigation Messaging System (NMS). Each frame of the Kinect's depth stream is converted into a per-degree distance scan and published to the NMS, where it can be consumed by The Navigator, SLAM mappers, or any other NMS-aware navigation skill for path planning and obstacle avoidance.

Requirements

• A Microsoft Xbox 360 Kinect sensor connected via the Kinect power/USB adapter.
• The Microsoft Kinect for Windows SDK v1.8 installed on the PC running ARC.

If the SDK is not detected when the skill loads, the log in the skill window will display a message prompting you to install it. No sensor data will be processed until the SDK is present. Download from Microsoft:

• Kinect for Windows SDK v1.8: https://www.microsoft.com/en-us/download/details.aspx?id=40278
• Kinect for Windows Developer Toolkit v1.8: https://www.microsoft.com/en-us/download/details.aspx?id=40276

*Note: Restart ARC after installing the SDK.

How it works

1. The Kinect's depth camera streams a 320×240 depth image at 30 Hz.
2. Inside the configurable Detection Area (the pink rectangle overlaid on the live preview), every valid depth pixel is examined.
3. For each image column inside the detection area, the nearest valid depth reading is kept.
4. Each column is mapped to a degree bucket across the Kinect's ~57 horizontal field of view using pinhole-corrected geometry (column-to-angle mapping is precomputed at startup for speed and accuracy).
5. Depth values are converted from millimeters to centimeters and published to the NMS as a 1D obstacle scan, on every frame.

Any column with no valid reading (pixels that are too near, too far, or flagged as unknown by the Kinect) reports float.MaxValue for that degree, which NMS consumers treat as "no obstacle."

Detection Area

Because the entire depth image is not needed for mapping, only a portion of it is processed. This prevents the robot from reacting to objects outside the size and height range that matters - for example, the floor directly under the robot, the ceiling above it, or parts of the robot's own chassis that sit inside the camera's view.

Configure the detection area with the four number selectors in the skill window:

Tune the pink box so it covers only the horizontal slice of the depth image that corresponds to the height range you want the robot to react to. A narrow horizontal strip at roughly the robot's chest height is a good starting point for most ground-based robots.

The rectangle is automatically clamped to the frame if any value is out of bounds - the skill will never crash from an oversized region.

Published data

The skill publishes to ARC.MessagingService.Navigation2DV1.Messenger.UpdateScan() using scan points with:

• Distance - nearest obstacle depth in centimeters for that angular bucket (or float.MaxValue if no reading).
• Intensity - fixed at 255 (the Kinect depth sensor does not report confidence per pixel in a form suitable for intensity).
• Heading - world-relative angle in degrees, with 0 pointing forward, measured clockwise. The scan covers roughly 331.5 through 28.5 (i.e. 28.5 around forward).

Any NMS-aware consumer in ARC (The Navigator, SLAM skills, etc.) can subscribe to this stream.

Audio

The skill also subscribes to the Kinect's beam-forming microphone array and logs the active sound-source direction and beam angle to the skill's log window. This is informational only - no audio data is published to NMS.

Tips

• Mount the Kinect as level as possible and at a height where the detection area will hit typical obstacles (chairs, walls, people) without picking up the floor or ceiling.
• If SLAM or The Navigator seems to "see" phantom obstacles above or below the robot, narrow the detection area's Height.
• If The Navigator can't see obstacles near the edges of the Kinect's view, widen the detection area (up to the full 320×240 frame).
• The Kinect's usable depth range is roughly 0.8 m to 4 m. Readings outside that range are reported by the sensor as unknown and are skipped.

Required Drivers