Thumbnail

Virtual Reality Robot

Plugin for all SteamVR supported virtual reality headsets (Google Cardboard, Oculus Rift, Windows Mixed Reality, HTC Vive, etc)

+ How To Add This Control To Your Project (Click to Expand)
  1. Make sure you have the latest version of ARC installed.
  2. Select the Get button in this page to download the archive file.
  3. Double click the downloaded archive file to execute installer.
  4. The installer will add this control to ARC.
  5. Load ARC and press the Project -> Add Control button from the menu.
  6. Choose the Virtual Reality category tab.
  7. Press the Virtual Reality Robot icon to add the control to your project.

Manual

Use Virtual Reality with your robot to see what the robot sees, and control servos by moving your head. This plugin supports all virtual reality headsets through Steam VR (OpenVR), Google Play and iOS, including Google Cardboard, Oculus, Windows Mixed Reality and HTC Vive. The plugin does two things, it allows you to see what the robot sees and control the servos with the Pitch and Yaw of the headset. This means when you move your head, the robot head can move as well, mimicking your movement. 

Requirement: This plugin requires ARC 2019.12.11.00 or newer

In this video, it was a test of the plugin on a Synthiam JD Humanoid. Sadly, he had no arms during the test so it's a bit funny to see. Professor E from The Robot Program was controlling the robot's head and viewing the robot's camera through the VR headset. In this example, we were using OpenVR in STEAM with a Samsung Odyssey headset.





In this video below, we were controlling an Synthiam Six Hexapod with a joystick and the VR headset. When we moved the head left to right, the legs of the hexapod will rotate the body so we can look around. This also applied to looking up and down, the legs would tilt the body so we can look up and down as well. In this example we were using an Android phone with Google Cardboard Glasses.




Here is another interesting example where we combined this VR Headset plugin with 2 WiiMotes to create custom animations. Each time a button was pressed on the WiiMote, a new FRAME was created with the current position of all servos and appended to an ACTION. DJ keeps pressing the button and the FRAMES keep getting appended to the ACTION until he stops. One he presses the trigger on the WiiMote, the code instructs the ACTION to be played back. And thus, all FRAMES are played back because they are part of the action.




Here is a screenshot of the plugin running on an ARC workspace with a camera.

User-inserted image


What Hardware Does It Support?
There are two methods that connect to the plugin which open a wide variety of hardware. 

1. SteamVR (OpenVR)
On PC, the supported headsets use Steam VR (OpenVR). This means any headset supporting Steam VR will work with this plugin, given the headset is directly connected to the PC. The most popular SteamVR headsets are Oculus, HTC Vive and Windows Mixed Reality.

2. Google Cardboard (iOS and Android)
The Google Cardboard is both a product and standard that turns your mobile device into a virtual reality headset. This means you can either purchase the cardbooard version from Google, or you can shop online and purchase a more robust plastic/foam padded version from amazon.

Download the Android App here: https://play.google.com/store/apps/details?id=com.ezrobot.virtualrealityrobot

User-inserted image


Allow Firewall Access
User-inserted image

When the plugin is loaded, you may receive a message to allow firewall access. Be sure to check Private and Public networks. If you do not check both, you may end up in a situation in the future where you're connected to a public network and the plugin isn't working. In this case, it's because ARC doesn't have firewall permission. Also, if you do not see this message when the plugin loads, it could be because you've already answered this in the past. In which case, you should visit the firewall rules to see if ARC has correct access.

1) Press the Windows button and type FIREWALL to search for Windows Defender Firewall
2) Select "Allow an app or feature through windows defender firewall"
User-inserted image


3) Press CHANGE SETTINGS
User-inserted image


4) Ensure both PRIVATE and PUBLIC are checked for ARC and press OK
User-inserted image




Attach/Detach Camera Button
User-inserted image


This plugin requires a camera, whether on your robot or a USB device. Add the camera device to your project and connect to the camera. Once you have a successful camera operational, press the Attach to Camera button on the plugin. This will connect the plugin to the camera device so the video stream is sent to the VR headset.

Start/Close VR Client Button (SteamVR only)
User-inserted image


If you are have a Steam VR compatible headset connected to your PC, this button will allow you to use that headset. This button will start the VR client, which connects to the VR Headset. The client can be launched either full screen or in a window. By default, the setting is to launch in the plugin window. This setting can be altered for your project in the configuration dialog.

Google Cardboard Users
For those using a google cardboard (iOS or Android) compatible device and headset, your device needs to be on the same network as the ARC PC. This is because your mobile device will connect to the ARC plugin by typing in the IP address of the PC. The IP Address for your ARC PC is displayed in the plugin. Simply load the Virtual Reality Robot mobile app on your device and enter the ip address of the ARC PC.

User-inserted image



Configuration Button
User-inserted image


This allows you to select the servos that will be assigned to the Horizontal (yaw) and Vertical (pitch) of the headset which correspond to the robot. So when you move your head, the robot moves as well.


Virtual Reality Caution
Read the warnings on your device before using a virtual reality (VR) viewer with this plugin to reduce the risk of personal injury, discomfort or property damage. Ensure you use this plugin with adequate space to avoid hurting yourself or others by walking into something. Nausea and motion sickness may be experienced and worse for some users. Do not use this plugin for extended periods of time to avoid experiencing nausea.:D

#42  
Well two outta three problems solved. As things do..they all of a sudden start working with out doing anything to them.

Final item is still the dynamixels. Please watch this video. Also sorry for the lethargy ...days of prepping for CES...exhausted.

Synthiam
#43   — Edited
Then set the servo ranges to match the head position. Move the positions until the vertical head angle is looking forward. You will have to experiment with the values, the software can't guess what they should be. Figure it out with experimentation...
#44  
No worries. I'm gonna pass on this. Too many other things to do. Thanks for the help.
Synthiam
#45  
I watched a documentary years ago about how our brains solve problems. They had these people play an arcade game for skiing. The people would play all day and have to keep beating their high score. Then, they’d get a full nights rest. The next day, on their first attempt, they all beat their previous high score. 

It’s because our brain processes information of the previous day while we sleep - it’s a process similar to defragging a hard drive. But, it’s also solving problems and sorting the day’s information into knowledge. 

1) organize your TO DO list and prioritize items between NEEDS and WANTS

2) when you’re feeling defeated, move to another task and revisit it the next day after a full night of rest
#46   — Edited
We have 48 hours until we step on a plane to Vegas. I've managed to wear all the hats (as usual) and produced all the flyers, business cards, banners in photoshop, dismantle the robots, design, build and install a bluetooth speaker systems, cast new faces in silicone and build shipping containers, created a pitch deck, made a press kit, etc. And alot to do in the remaining time. (No investor can deny my sweat equity.)

The reality, I only had 12 hours to get this plugin to work with my robots. That's all the time available in my schedule. It was a last minute decision to try and demonstrate telepresence, something not planned for the event. If I had a week, to mess with it then maybe I could figure out the dynamixel issue. The reality is I just don't. 
 
As of this morning we have 84 companies who have scheduled to see us. Hopefully none of them request a telepresence demonstration:) Maybe we fake it til we make it:D
Synthiam
#47  
Will - have you spent any more time with this control? It's so much fun!
#48  
Ha not yet, I have to pry the headset from my kids hands. They are on it everyday!

I hope to revisit soon!
Synthiam
#50  
Thanks for the reminder - i'll take a look and see where it's at
#51  
It there a app I can use my iPhone to use the  EZ-Robot Virtual Reality Viewer?
#52  
is there an app for am iPhone to use the  EZ-Robot Virtual Reality Viewer? I don't have a Android
#54  
This is what I am interested in doing this summer outside when my robot goes on patrol outside. See what the robot see in 360 or 180 degree VR 3d and take control of robot drive and arm movements. I just got the Oculus Go and it is a fantastic VR device to try this, will the plugin support all VR head sets?
#55  
Oculus Go is great, but using my iPhone seems a little more in my range  :-)
Synthiam
#56   — Edited
@EZang60, the iOS iPhone app is not available at this time

@RobotRad, the supported VR headset requirement is in the plugin description. Steam compatible headsets are supported - or google cardboard
#57  
Thanks for your response,
#58  
Hey thanks DJ I do have the steam client working with my GO ,will try out the ARC skill plugin as well. Glad to know you are working on this cutting edge technology way before anyone else, I bet you are already working on adding EZ chip to bionic arms and legs for disabled people ten years in the future!
#59   — Edited
Is their a way that I can move my robots head when using my htc vive as it is not working right now.  Never mind got it had to activate servos