Intel Realsense T265

How to use the Intel Realsense T265 robot skill

With its small form factor and low power consumption, the Intel RealSense Tracking Camera T265 has been designed to give you the tracking performance for your robot. This ARC user-friendly robot skill provides an easy way to use the T265 for way-point navigation.

The T265 combined with this robot skill provides your robot a SLAM, or Simultaneous Localization and Mapping solution. It allows your robot to construct a map of an unknown environment while simultaneously keeping track of its own location within that environment. Before the days of GPS, sailors would navigate by the stars, using their movements and positions to successfully find their way across oceans. VSLAM uses a combination of cameras and Inertial Measurement Units (IMU) to navigate in a similar way, using visual features in the environment to track its way around unknown spaces with accuracy. All of these complicated features are taken care of for you in this ARC robot skill.


Update Firmware
The device sensor may require a firmware update.

1) Visit the Realsense GitHub page, scroll to the bottom of the page, and install the Intel.Realsense.Viewer.exe from here: https://github.com/IntelRealSense/librealsense/releases/latest



2) Click the hamburger settings icon and select Install Recommended Firmware





Robot Skill Window
The skill has a very minimal interface because it pushes data in the NMS and is generally used by other robot skills (such as The Navigator).



1) Drop-down to select Realsense device by the serial number. This is useful if there are multiple devices on one PC.

2) START/STOP the Intel T265 connection. 

3) The confidence of the tracking status between 0 (low) and 3 (highest). In a brightly lit room with many points of interest (not just white walls), the tracking status will be high. Tracking will be low if the room does not have enough light and/or detail for the sensor to track.

4) Log text display for errors and statuses.

Config Menu



1) Mounted Radius Offset (mm) is the distance in mm of the T265 from the center of the robot. A negative number is toward the front of the robot, and a positive number is toward the rear. The sensor must be facing 0 degrees toward the front of the robot. The sensor must not be offset to the left or right of the robot.

2) Enable Video Stream will send the fisheye b&w video from the T265 to the selected camera device. The selected camera device robot skill must have Custom specified as the input device. Also, the camera device will need to be started to view the video.

3) Distortion Correction will use a real-time algorithm to correct the fisheye lens, which isn't always needed and is very CPU intensive.

Video Demonstration
Here's a video of the Intel RealSense T265 feeding The Navigator skill for way-point navigation


ARC Navigation Messaging System
This skill is part of the ARC navigation messaging system. It is encouraged to read more about the messaging system to understand available skills HERE. This skill is in level #3 group #2 in the diagram below. This skill contributes telemetry positioning to the cartesian positioning channel of the NMS. Combining this skill with Level #3 Group #1 skills for obstacle avoidance. And for Level #1, The Navigator works well.






Environments
This T265 will work both indoors and outdoors. However, bright direct light (sunlight) and darkness will affect performance. Much like how our eyes see, the camera will is also susceptible to glare and lack of resolution in the dark. Because the camera visual data is combined with the IMU, the camera must have reliable visible light. Without the camera being able to detect the environment, the algorithm will be biased to use the IMU and will experience drift, which greatly affects the performance of the sensor's accuracy.

Screenshot
Here is a screenshot of this skill combined with The Navigator in ARC while navigating through a room between two way points.




Starting Location
The T265 does not include a GPS/Compass or any ability to recognize where it is when initialized. This means your robot will have to initialize from a known location and direction to reuse saved maps. Make sure you mark the spot on the ground with masking tape where the robot starts from.

How To Use This
1) Connect your Intel RealSense T265 camera to the computers USB port

2) Load ARC (version must be >= 2020.12.05.00)

3) Add this skill to your project

4) Now we'll need a Navigation skill. Add The Navigator to your project

5) Press START on the Intel RealSense skill and data will begin mapping your robot's position

How Does It Work?
Well, magic! Actually, the camera is quite interesting and it breaks the world down into a point cloud of features. It remembers the visual features so it can re-align itself on the internal map. It uses a VPU, is what Intel calls it. Here's a video of what the camera sees.

import time

cleft = 0
cright = 0
fleft = 0
fright = 0
bleft = 0
bright = 0

while (True):

  if getVar("$RoombaLightBumperCenterLeft")==1:
     Navigation.SetNavigationStatusToPause();
     print "Bumper Center Left";
     cleft += 1;
     print cleft
     #time.sleep(2);
  if cleft == 2:
     Movement.Right(30,3500);
     Movement.Forward(30,1500);
     Movement.Stop();
     Navigation.SetNavigationStatusToNavigating();
     cleft = 0
     cright = 0
     fleft = 0
     fright = 0
     bleft = 0
     bright = 0
     print cleft
     
  if getVar("$RoombaLightBumperCenterRight"):
     Navigation.SetNavigationStatusToPause();
     print "Bumper Center Right";
     cright += 1;
     print cright
     #time.sleep(2);
  if cright == 2:
     Movement.Left(30,3500);
     Movement.Forward(30,1500);
     Movement.Stop();
     Navigation.SetNavigationStatusToNavigating();
     cleft = 0
     cright = 0
     fleft = 0
     fright = 0
     bleft = 0
     bright = 0
     print cright
     
  if getVar("$RoombaLightBumperFrontLeft")==1:
     Navigation.SetNavigationStatusToPause();
     print "Bumper Front Left";
     fleft += 1;
     print fleft
     #time.sleep(2);
  if fleft == 2:
     Movement.Right(30,3000);
     Movement.Forward(30,1500);
     Movement.Stop();
     Navigation.SetNavigationStatusToNavigating();
     cleft = 0
     cright = 0
     fleft = 0
     fright = 0
     bleft = 0
     bright = 0
     print fleft
     
  if getVar("$RoombaLightBumperFrontRight")==1:
     Navigation.SetNavigationStatusToPause();
     print "Bumper Front Right";
     fright += 1;
     print fright
     #time.sleep(2);
  if fright == 2:
     Movement.Left(30,3000);
     Movement.Forward(30,1500);
     Movement.Stop();
     Navigation.SetNavigationStatusToNavigating();
     cleft = 0
     cright = 0
     fleft = 0
     fright = 0
     bleft = 0
     bright = 0
     print fright 
     
  if getVar("$RoombaLightBumperRight")==1:
     Navigation.SetNavigationStatusToPause();
     print "Bumper Right";
     bright += 1;
     print bright
     #time.sleep(2);
  if bright == 2:
     Movement.Left(30,2000);
     Movement.Forward(30,1500);
     Movement.Stop();
     Navigation.SetNavigationStatusToNavigating();
     cleft = 0
     cright = 0
     fleft = 0
     fright = 0
     bleft = 0
     bright = 0
     print bright
     
  if getVar("$RoombaLightBumperLeft")==1:
     Navigation.SetNavigationStatusToPause();
     print "Bumper Left";
     bleft += 1;
     print bleft
     #time.sleep(2);
  if bleft == 2:
     Movement.Right(30,2000);
     Movement.Forward(30,1500);
     Movement.Stop();
     Navigation.SetNavigationStatusToNavigating();
     cleft = 0
     cright = 0
     fleft = 0
     fright = 0
     bleft = 0
     bright = 0
     print bleft
else:
  Navigation.SetNavigationStatusToNavigating();
  time.sleep(1);
  cleft = 0
  cright = 0
  fleft = 0
  fright = 0
  bleft = 0
  bright = 0