Thumbnail

Intel Realsense T265

Use the Intel Realsense T265 tracking camera for way-point robot navigation

+ How To Add This Control To Your Project (Click to Expand)
  1. Make sure you have the latest version of ARC installed.
  2. Select the Get button in this page to download the archive file.
  3. Double click the downloaded archive file to execute installer.
  4. The installer will add this control to ARC.
  5. Load ARC and press the Project -> Add Control button from the menu.
  6. Choose the Navigation category tab.
  7. Press the Intel Realsense T265 icon to add the control to your project.

Manual

With its small form factor and low power consumption, the Intel RealSense Tracking Camera T265 has been designed to give you the tracking performance for your robot. This ARC user-friendly robot skill provides an easy way to use the T265 for way-point navigation.

The T265 combined with this robot skill provides your robot a SLAM, or Simultaneous Localization and Mapping solution. It allows your robot to construct a map of an unknown environment while simultaneously keeping track of its own location within that environment. Before the days of GPS, sailors would navigate by the stars, using their movements and positions to successfully find their way across oceans. VSLAM uses a combination of cameras and Inertial Measurement Units (IMU) to navigate in a similar way, using visual features in the environment to track its way around unknown spaces with accuracy. All of these complicated features are taken care of for you in this ARC robot skill.


Update Firmware
The device sensor may require a firmware update.

1) Visit the Realsense GitHub page, scroll to the bottom of the page, and install the Intel.Realsense.Viewer.exe from here: https://github.com/IntelRealSense/librealsense/releases/latest
User-inserted image


2) Click the hamburger settings icon and select Install Recommended Firmware
User-inserted image




Robot Skill Window
The skill has a very minimal interface because it pushes data in the NMS and is generally used by other robot skills (such as The Navigator).
User-inserted image


1) Drop-down to select Realsense device by the serial number. This is useful if there are multiple devices on one PC.

2) START/STOP the Intel T265 connection. 

3) The confidence of the tracking status between 0 (low) and 3 (highest). In a brightly lit room with many points of interest (not just white walls), the tracking status will be high. Tracking will be low if the room does not have enough light and/or detail for the sensor to track.

4) Log text display for errors and statuses.

Config Menu
User-inserted image


1) Mounted Radius Offset (mm) is the distance in mm of the T265 from the center of the robot. A negative number is toward the front of the robot, and a positive number is toward the rear. The sensor must be facing 0 degrees toward the front of the robot. The sensor must not be offset to the left or right of the robot.

2) Enable Video Stream will send the fisheye b&w video from the T265 to the selected camera device. The selected camera device robot skill must have Custom specified as the input device. Also, the camera device will need to be started to view the video.

3) Distortion Correction will use a real-time algorithm to correct the fisheye lens, which isn't always needed and is very CPU intensive.

Video Demonstration
Here's a video of the Intel RealSense T265 feeding The Navigator skill for way-point navigation



ARC Navigation Messaging System
This skill is part of the ARC navigation messaging system. It is encouraged to read more about the messaging system to understand available skills HERE. This skill is in level #3 group #2 in the diagram below. This skill contributes telemetry positioning to the cartesian positioning channel of the NMS. Combining this skill with Level #3 Group #1 skills for obstacle avoidance. And for Level #1, The Navigator works well.
User-inserted image





Environments
This T265 will work both indoors and outdoors. However, bright direct light (sunlight) and darkness will affect performance. Much like how our eyes see, the camera will is also susceptible to glare and lack of resolution in the dark. Because the camera visual data is combined with the IMU, the camera must have reliable visible light. Without the camera being able to detect the environment, the algorithm will be biased to use the IMU and will experience drift, which greatly affects the performance of the sensor's accuracy.

Screenshot
Here is a screenshot of this skill combined with The Navigator in ARC while navigating through a room between two way points.
User-inserted image



Starting Location
The T265 does not include a GPS/Compass or any ability to recognize where it is when initialized. This means your robot will have to initialize from a known location and direction to reuse saved maps. Make sure you mark the spot on the ground with masking tape where the robot starts from.

How To Use This
1) Connect your Intel RealSense T265 camera to the computers USB port

2) Load ARC (version must be >= 2020.12.05.00)

3) Add this skill to your project

4) Now we'll need a Navigation skill. Add The Navigator to your project

5) Press START on the Intel RealSense skill and data will begin mapping your robot's position

How Does It Work?
Well, magic! Actually, the camera is quite interesting and it breaks the world down into a point cloud of features. It remembers the visual features so it can re-align itself on the internal map. It uses a VPU, is what Intel calls it. Here's a video of what the camera sees.

PRO
USA
#97  
@DJ,

Do you thing if I place the T265 in the chest of an Inmoov
 will work correctly or does it need to be ground leave?


Thank you, Cheers
PRO
Synthiam
#98  
I’m guessing it will work fine. It can’t have anything blocking it. So no hands can be in front. Otherwise it won’t know where it is when the robot drives around
PRO
USA
#99  
Thank you DJ
, I will try it soon.
PRO
Synthiam
#101  

Quote:


when it came to the roughly 10 customers who bought RealSense products in relatively low volumes. There were only a few that were buying multiple units, like 10 to 40 at a time


well I feel deceived. I was told directly in a zoom call by the realsense cto that customers were purchasing thousands at a time for volume discounts.
PRO
Canada
#103  
There are 5 realsense in every Boston dynamics spot.   I hope they stocked up.
PRO
Synthiam
#104  
Boston dynamics bought at least 40 lol O_o

It's amazing the poor ethics of businesses to lie about their sales volume - boom, you've been busted
PRO
Canada
#105  
I believe BD have only sold about 400 Spot world wide.  Several other dogs use them as well but not in volume.  They are also used in various restaurant food service bots, delivery bots, warehouse bots etc but again not in volume. 

I am surprised they did not sell IP to a robot company or spin off to a startup for a royalty fee on every sale. There is probably value for a small startup to continue to sell and develop with an existing customer base.  

This makes we wonder if there is some other issue (China Manufacturing, Chip Shortage, Shipping Issues etc)
PRO
Synthiam
#106  
Boston dynamics sold 400? Or 4?
#107  
Thought Id pass this along..was mentioned via anothere group, A company called StereoLabs makes a simular product called the ZED, Might be worth a look?
PRO
Synthiam
#108  
I do not believe there is a similar product on the market for navigation & mapping. The other devices (ie stereo lab zed) ate depth sensing cameras and therefore require external applications for navigation/slam mapping.

If the comparison is depth camera functionality, then there are other options.  But feature to feature there aren’t
PRO
Portugal
#109  
I wonder why and why now. Just last week Xiaomi released its robot cyberdog that will storm the markets at a very low price. They use a D455 realsense camera. I wonder, will they sell the realsense technology to Xiaomi?
But as always the tech market is what it is and always evolving. I just hope the prices drop and I get some cameras.
PRO
Synthiam
#110   — Edited
I think the robot dog market peaked. Even at a lowered price, value can’t be added to something with no purpose. I doubt a robot dog company will generate enough profit on revenue to justify the high cost of owning a that technology specially for a single product function. 

It’s not financial feasible for a product company to own the technologies. The only company that I think came close to buying the guts of realsense would be Boston dynamics - but that ship has sailed since they can’t figure out a product/revenue mode. They’re like the Honda Asimo team and being used for marketing and will fade away soon. Their videos are receiving less attention. People want to know what’s it for? What’s it do? now. 

Realsense IP will most likely remain in intel’s hands and end up in their dusty museum like everything else. Intel isn’t a company known for selling or sharing IP.

I don’t think it’s all gloom and doom. The removal of realsense from the market will most likely stimulate a demand for a similar product line.
#111  
I think the robot cyberdog would make an awesome security guard, patrol and alarm system. That could be a market for them to explore.  However you would have to have two of them. One to watch the other so it wouldn't get stolen. LOL.
PRO
Synthiam
#112   — Edited
Security patrol robots have been attempted for the last 40 years. They aren’t effective because

1) criminals don’t get scared by a robot with a camera. 

2) cameras already exist and can be mounted on buildings (cheaper)

3) hiring humans means they can chase or discourage or offer help etc etc etc. humans do way more than a robot. Such as clean and check doors to see if they’re locked. Oh man there’s a million reasons why a human security guard is better than a robot 

4) humans don’t need to recharge their batteries every 25 minutes:)

5) humans don’t get stuck on a pebble in the parking lot

6) humans can get wet in the rain 

7) thieves most likely Won’t shoot a human. But they’ll hit a robot with a bat
#113  
LOL DJ. Great list. Mostly love #4. You should have met some of the humans I've worked with over the years.xD
PRO
Synthiam
#114  
Ha ha - that’s so true. Robots have their place, it’ll just take a bit more time:)
PRO
Canada
#115   — Edited
Interesting news today, looks like all the RealSense Stereo Depth (D series) cameras that we know and love will continue to live on!

LinkenIn post by Steve Crowe

Unfortunately the T series (tracking cameras) are EOL
PRO
Canada
#116   — Edited
I'm guessing some of you don't have LinkedIn so here's the link to the Robot Report page:

https://www.therobotreport.com/intel-issues-end-of-life-notice-realsense-lidar/

And here's the pertinent info that relates to this skill page:

Intel RealSense T265 will be best replaced by Intel RealSense D455 or D435i with 3rd Party SLAM SW

Here's the link to the ARC skill for the D435i: https://synthiam.com/Support/Skills/Navigation/Intel-Realsense-D435i?id=20404