Exosphere combines telepresence, artificial intelligence, machine learning and human assistance to give autonomous behaviors to cloud robotics.
How to add the Exosphere robot skill
- Load the most recent release of ARC (Get ARC).
- Press the Project tab from the top menu bar in ARC.
- Press Add Robot Skill from the button ribbon bar in ARC.
- Choose the Remote Control category tab.
- Press the Exosphere icon to add the robot skill to your project.
Don't have a robot yet?
Follow the Getting Started Guide to build a robot and use the Exosphere robot skill.
How to use the Exosphere robot skill
Exosphere is a game-changing robot product that allows telepresence and remote control of any robot. You can use a phone or tablet to remote control your robot. This product gives you the power to help make the world safer and improve business efficiency with telepresence.
Exosphere allows any robot to execute autonomous tasks or have telepresence capabilities in the workplace or at home with human intelligence. Exosphere bridges the gap between robotics, machine learning, and artificial intelligence.
Who Is Exosphere For? A human can remotely control any robot to achieve tasks or become a telepresence host.
The way we think, including moral hesitation, judgment, and responsibility, is being taught to AI algorithms. This is why AI greatly benefits from human interaction. With nearly 3 million robot connections on Synthiam’s platform, robots worldwide are added daily with new tasks when the exosphere is publicly released. These tasks are created for real robots with real problems to solve, getting AI out of the lab and into the real world.
Exosphere can control as much or as little as necessary to help robots achieve their goals, even if that means using human intelligence to fill in the 10% edge cases that robots can’t do.
How Does It Work?
- A robot is instructed to execute a task.
- The task is submitted to Exosphere as a contract request.
- AI/ML and humans provide a confidence bid to win the contract.
- If AI/ML can’t control the robot to perform a task successfully, a human operator is issued another contract to take over or assist the robot. In turn, training the global AI knowledge base for future task requests.
Main Window - Control Tab
1. Control and Log Tabs This selects between the Control and Log sections of the Exosphere Skill main window.
2. Task Description Field Enter the description of the task you are generating and detailed instructions for the user who will accept it.
3. Submit/Cancel Task Button Once the task is submitted, a remote operator will take control of the robot. It is highly recommended that the robot be monitored remotely to avoid hazards to the robot or damage to the environment. Website users will be notified once a task is submitted to the Synthiam servers. You can also use this button for an emergency stop to cancel the task when it's running.
4. Task Details Field Displays the time the task was requested, who picked up the task, and when.
Main Window - Log Tab
1. Log Selection Drop-Down This selects the information level you want to see in the Log Display window: Verbose, Debug, Error, or None.
2. Clear Log Button Clears the Log Display Window.
3. Log Display Window Displays the Log information when errors occur. The Log Selection Drop-down tailors the amount of information.
Configuration - General Tab
1. Robot Model and Description Write in your Robot's Make, Model, and Title. Then, add a description of the robot's qualities.
2. Session Password and Minimum Rating Optionally, you can set a password and a minimum rating a user must have to control your robot. Note: These features have not been implemented yet
3. Maximum Joystick Drive Speed Use the drop-down to set the maximum speed of your robot being controlled by the Joystick. If your robot is connected to ARC, you can use the on-screen Joystick to test the speed.
4. Keyboard Drive Speed Use the drop-down to set the maximum speed of your robot being controlled by the Keyboard. If your robot is connected to ARC, you can use the "Test Forward" and "Stop" buttons to test with.
Configuration - Scripts Tab
1. Task Created, Accepted, and Completed Scripts This section allows you to add scripts that will execute when the following conditions are met: your task is created, accepted, or completed.
There is also a script for remote speaking, which is when the operating types in a phrase are to be spoken. This script will execute to say the words the operator enters if enabled. This feature is enabled on the Advanced tab. You can use this script to use your preferred text-to-speech robot skill engine.
2. User Defined Scripts Add additional scripts to your generated Exosphere task. They will show up as playable scripts on your generated Task in the order you have them added.
Configuration - Advanced Tab
1. Audio If enabled, this section allows the user who accepts your task to hear through your robot's microphone with the settings you configure. This will allow you to communicate with the user in real-time.
2. Text to Speech Check-box Once enabled, the user can write text responses that will be converted to audio. The audio will come out of your robot's speaker.
3. Video Settings You can select an installed video device for the user to use for real-time visual feedback. You can also use the check box to send video tracking information to the user.
4. Telepresence Mode Enabling this allows two-way communication between the remote operator and the robot. The remote operator's camera video and audio will be displayed on the robot on fullscreen. This allows the robot to approach someone and converse with the remote operator. At the end of the session, the "rating" window is not displayed to the remote operator. Click here for a tutorial on configuring a robot for exosphere telepresence.
5. Experimental Features These are internal development features. These features may not work and are not documented.
Configuration - User Access Tab
1. Username Field This field allows you to write in a username.
2. Access Type Once the add button is pressed, the username inputted will be added to your Access list and assigned to the selected access type (in the drop-down).
3. Access List This window displays the users you have added and their access type.
How to Use the Exosphere Skill
Add the Camera device skill to your ARC project (Project -> Add Skill -> Camera -> Camera Device).
Add a Movement Panel skill to your ARC project that matches your robot (Project -> Add Skill -> Movement Panels).
Add the Exosphere skill to your ARC project (Project -> Add Skill -> Machine Learning -> Exosphere).
Add a brief description of your task to the task description field in the Exosphere skill main window.
Click the Submit Task button. The task is now on the Synthiam servers and is awaiting a user's pickup.
Once the task is completed. Rate the user in three areas on how well the task was completed. 5 stars = the best job. This will help advance the machine learning database with reinforcement learning.
Compatibility
With pandemics like COVID-19, Exosphere needed compatibility with existing robots to leverage their ability with human operators while training the knowledge base for future autonomous behaviors. Exosphere was designed as a web service & skill to allow connections from existing robots or control software, such as ROS or ARC. However, it only takes minutes to add any robot to Exosphere with Synthiam’s ARC software. As per the Synthiam way, we’ll add many tutorials demonstrating how to add various robots to Exosphere using ARC.
Control Commands for the Exosphere robot skill
There are Control Commands available for this robot skill which allows the skill to be controlled programmatically from scripts or other robot skills. These commands enable you to automate actions, respond to sensor inputs, and integrate the robot skill with other systems or custom interfaces. If you're new to the concept of Control Commands, we have a comprehensive manual available here that explains how to use them, provides examples to get you started and make the most of this powerful feature.
Control Command Manual// Set the camera output to the original video stream.
- controlCommand("Exosphere", "SetOutputOriginal")
// Set the camera output to the processsed video stream.
- controlCommand("Exosphere", "SetOutputProcessed")
// Toggle the video stream output between original and processed.
- controlCommand("Exosphere", "ToggleOutput")
// Request a new task to be posted.
- controlCommand("Exosphere", "RequestTask", "Task Description as a string")
// Close the current task.
- controlCommand("Exosphere", "CloseTask")
// Return the state of the task. NoTask = 0, Starting = 1, WaitingForPuppeteer = 2, InProgressByPuppeteer = 3, Closing = 4. (Returns Integer [-2,147,483,648 to 2,147,483,647])
- controlCommand("Exosphere", "GetTaskState")
nounASTRONOMY the outermost region of a planet's atmosphere.
Looks like Synthiam is going to space. Can we all come ?
.....Machine learning....now you have my attention..........
so what do we do with this?
Maybe they are trying to tell us something. like get lost ie. lost in space.
??Cloud based machine learning?? My guess. Eventually every object known to man will be cataloged and easy for machines to know what they are "looking at" plus all the relevant description/use of that object i.e proper use/s.
The Matrix...but instead of Neo, its a robot.."I want to learn jiu jitsu"
I wonder if we have to worry about agent smith. aka agent Sures.
"Shall we play a game?" Wargames
what was that video? and who is agent smith?
Dr Smith? Lost in Space.
Just saying maybe we should be worried more about the machines learning then humans. Lol.
No Dave it is the "agent smith" you know black suit and dark glasses in the Matrix. the movie Dave is showing EZang60 is about a kid who connects to what he thinks is a game machine but he connects to a defense computer and just about starts a thermonuclear war for real.
Nallycat, Oh ya! How could I forget that character! Thanks!
Wont be long now....Skynet will have its day...
...could be an echo from the telepresence robot project synthiam was working on....maybe streaming audio and video for client and server connection? Controls for a long distance robot?
Keep the guesses coming
version 3 of Exosphere, what do we do with it?
This must be a 2020 great project, DJ has us guessing.
We are waiting, we are staying tuned, lol
what happened to the Exosphere?
sounds exciting.
When will hear about this?
As stated on the exosphere plugin details, development is in progress.
development is in progress.
It is 4/3/2020 now
Hope all are ok
downloaded your new version 40
Any instructions yet?
Updated with new server mapping
no incomming from synthiam server,new error?
also the arrows keys are not desame as mine.the bot goes only one direction on the web.
set it up I will try it Nomad
EZAng
ok let me know.
Only the camera works, no movement of robot
Try making your robot walk
the robot turns only to the right.it got wheels.adventurebot.
over here, looks like only the camera moves in a circle
I see the robot, no moving
do you have a JD robot?
that is correct.the robot turns only to the right.its an adventurebot.
oh, I see the camera going in a circle,
use a walking robot with a camera
oka moment.i set a jd.
jd is set up.how do i ad a task?
It worked that time, go into Blockly and do a auto-position wave
give me a good rating :-)
Use the latest exosphere plugin update. It has a feature in the config menu to test the joystick speed. Ensure your joystick and speed is correct before publishing. You can make sure it doesn't spin in a circle lol
dj
in the Movement Panel are 3 speed settings.forwart/backwarts/stop. witsh one do i need in the config?
You need to update the plugin to get the latest update
ah i just had an update some 2 houres ago.and here's allreddy another. i get the adventurebot going straigh forwart only using 180 forwart speed.?
i see where the error starts.in the config page for setting speed.you have to click that with you mous button. that is diff when using the arrow key's on your board.i cant get the setting good.
when i set the settings for speed so the adventurebot go straigh forwart in my arc, it changes when using exosphere .
Good morn, I noticed you said you would explain ROS
I looked it up,
For Linux - I am windows 10 :-( very happy ARC uses windows...
Robot Operating System (ROS)
The ROS Development Studio At The Construct, we have believed in this approach since 2015 and developed a platform that integrates all those tools. The ROS Development Studio (ROSDS): Runs on the cloud Provides a robotics development environment with simulations that already work Allows the transfer to real robots in a standardized procedure And, of course, it is based on ROS! When you create a ROS program in the ROSDS, you can use any of the already provided simulation, or you can create your own. What is interesting is that, when you make your code work with the selected simulation, that work can be shared and reproduced by anyone under the same exact conditions. If it worked for you, it will work for anybody else that gets your ROSDS project. and a lot more...
EZang
Dj said he would explain
i set the speed to 180,it drives straigh forwart except on the web page.
Your camera gets me dizi - goes round and round
thats the problem.i cant get it drive straigh on the web page. when i click on the settings page it drives straigh.
ok try to fix it
Hi Nomad, where did you go?
I am ready to test your robot now
EzAng
Nomad - can you do me 2 favors?
Can you send me your robot project that is having trouble going Forward with exosphere?
Can you tell me what button you use for forward from this picture?
@DJ.... On a separate note. Will Exosphere be expanded down the road to accommodate other types of robots? I am interested in using it to control a robotic arm to serve drinks, food or to perform other tasks possibly in my workshop....
It works with absolutely any robot
just add the exosphere control to your robot and start using it.
It’s awesome. I’m gonna start using it at the office to get my drinks haha. I’ll rig something up tomorrow.
gonna use your inmoov one day to! You can add custom buttons to move the arms and such
@DJ.... Ok then... Inmoov will be cooking me dinner soon... Thank GOD I printed him out of ABS... Wouldn't want him melting over the stove... cool Still going to have to use exosphrere to teach him how to use a fire extinguisher, though....
hi all
dj i use the ( click to use arrow keys ) below. i send my project
i had to get some sleep.
Nomad, what arrow key makes your robot go forward? it should be the one pointing up.
richard, drinks... dinner... sounds like a robot date! That’ll solve half of my isolation challenges lol.
dj yes arrow key forwart is up.i also notest the adventure turn left and right is way to fast. cant get it to line up.
thank you
The instructions explain how to adjust the maximum speed. For that, you’ll have to read and follow the instructions. There’s even a pop up that asks if you’ve tested the speed with when editing.
you’ll figure it out. Have faith in you
@Richard, what did you print out of ABS? God or inmoov? I knew you were a powerful dude but this? LOL
Hahaha - If God 3d printed the universe, it definitely took more than 7 days
. With the corvid-19, we might be in a failed print
I'm sure he has a better 3D printer then we could ever dream of creating. What's the use of being God If everyone has the same stuff to make what you do?
Skillzzzzzzz.... it's all about the skillz
Nomad are you awake yet?
EzAng
hi
i am awake.watshing mike tyson.xD
How did it go today?
i am taking a few days off.no robotics or 3d printing. and waiting for parts.
It is good to take a break, refresh yourself
it is 10:30 pm here Wednesday.
I should be back tomorrow
Good morn 9 hour later, Nomad, 7:32 am here Thursday
ready to test your robot if you want
EzAng
Anyone want me to test your robot?
EzAng
I did the advance a few time, what is next?
I have my robot waiting for someone to activate it
EzAng
Someone completed it - great!
EzAng
Bragging but not bragging - ARC is amazing. 3 robots running with 5 cameras for the Exosphere playground and check out the CPU/Memory usage... 50mb per robot ain't too shabby
good job nallycat but you crashed into them
try again please EzAng
@EZAng Thanks for telling me I thought crashing into them was acceptable, I was not sure how the program would know I selected the card. lots of fun. how do you indicate you select the card? Tell DJ he is Bragging.
great stuff DJ - CPU/Memory usage... 50mb per robot
EzAng
my fault, sorry, nallycat my speed was to high, try again please
get as close as possible where you see the color most
EzAng
@EzAng what do you do just stop in front of the card. and then move on?
that would be good
charging the robot
Hi Nomad, nothing was moving
EzAng
hi EzANG
indeed.i am using roli.moment.
you can test now ezang.
deleted post.was ezangs robot.
You did it Nomad, can you hear me talk?
EzAng
yes i can hear you talk.just a little distortion in audio.
Nomad no movement, camera is pointed to a small robot
EzAng
sometime is wrong, when I click to star it goes to a different page
i set the speed at highest.try it.
still no movement
was that the robot to reach?
EzAng
its comming good.
So the was the robot with a sign that says "robot"
Great
EzAng
here's where i think the confusing starts.in dj video,he set the left speed settings at 50 . adventurebot turns in circles and roli almost dont move.i set bolt at 200 now.
my adventureBot I set max speed 100
you set that in exosphere
Only Nomad, and myself are doing this, one time nallytcat - of coarse the admins
Where is everyone else?
EzAng
your setting from adventurebot Movement Panel is set to 100 max?
In exosphere I put 100
up at the top, scripts tab - auto-position wave or whatever....
EzAng
thank you for the pics.
Pictures are better then words, :-)
EzAng
I am back if you want to test again Nomad
EzAng
moment i set it up.
that was perfect,
you did set the speed in exosphere, great - you also can test the speed there
EzAng
Well we can do it, the 3 DJ made and set our own robot up, what about all the other people here?
EzAng
i a few minutes i have another test for you. look for the mini robot.xD
I did it !!
100% again
We need better obstacles
EzAng
can you hear the robot?
yes and if you talk I hear you also
All sounds
seems like the app goes right into our home, visual and audio :-) EzAng
robot says hi ezang
I did not hear him, want to try again?
EzAng
yes i set roli up
Yes I hear him, who is the little robot?
EzAng
thta is robot MINI from ubtecth
ok I will kook up on net
EzAng
its on sale in holland .only for dissabled people.cost 1400 euro + 500 each year to use all apps. way too much.i set him up again so you can see the eyes change hoppefully.lol
I can't say hi or anything, but I guess that was good 100%
EzAng
Nomad I put my seed down to 75 in exosphere
You did great, do did Dj
need to make a harder one :-)
EzAng
okay i change tomorrow the setting to 75.
Ok good night, I know it is late there
DJ try again please or who ever is out there EzAng
Thanks DJ , you did it again
EzAng
We need one the picks something up and drops it off in a certain color
or one that pushes a small ball to a destination
EzAng,
Anyone test any robots today?
EzAng
Happy Sunday Nomad, are you there to test robots on exosphere?
EzAng
Anyone out there, test my robot please
EzAng
hello ezang
robot doesn move.
I will put a new charge on the battery, give me a moment, thanks
EzAng
Nomad, it says move to the white sheet, then pink sheet, end up at the yellow sheet, don't crash into anything
i cant see the diff between white and yellow.
oh, I will fix, sorry
EzAng
I wrote the names of the colors on the sheets
EzAng
You did it!
EzAng
it works great.you can set the words with a fat black marker.
You read my mind, I just did that with a black marker :-)
Did you try the avitar program yet? Works fine
EzAng
https://www.en.cloudminds.com/
sounds like DJ
amazing
EzAng
Funny - except I've been working on this theory before 2015. See this video in my basement dated Oct 28, 2015. Except back then we called it ExoSphere not Exosphere it seems.... I remember the day Alan and I came up with the name - it was in Cuba sitting in a pool having a conversation about it. We wanted something that was bigger than the cloud. Something that fed the cloud. Something that controlled the cloud. Exosphere was born
Oct 28, 2015, amazing how time passed so fast, now 4/20/2020
I am very happy you followed up with your theory - You said, something that was bigger than the cloud. Something that fed the cloud. Something that controlled the cloud.
Exosphere was born and we all can participate in this.
Thanks again for such amazing technology
EzAng
@EzAng Cloudmind looks cool...but Exosphere supports more hardware, and also soon ROS! Which will make even more users participating! What a time to be alive!!
I believe it, we are in the right pot with DJ at the wheel.
All the best
EzAng
I am still trying to understand the value of Exosphere. I see EzAng making Nomad's robot wave and I understand it generates a database for Synthiam, but to what end? I'm anxious to see how others implement it to help me better understand. I am sure it will get there so I am along for the rind now. I did the tasks and drove the EZ robots on the site. It was fun as a telepresence exercise.
Perry - give this a read: https://synthiam.com/Products/Exosphere
I keep doing the advance, any other test?
Did the advance again 4/24/2020
EzAng
new test does not work, please check
EzAng
still does not work
@Jeremie/@DJ: I wanted to runaway but at that speed I have doubts If I could reach the door
Hahahaha - yes. I made a delivery bot so we can move stuff back and forth between offices. Eventually it'll learn the layout of the office and be fully automated!. We're putting a tray on the robot for that usage. I'm modifying it now so you can move the camera up and down. stay tuned
sounds good
EzAng
where is Jeremy's office?
EzAng
I dunno where it is - you'd have to find it. Or, find someone to ask. Follow the instructions
How doe you ask:
FIY error:
I am back, try again try now ptp
ptp: That's the temporary error if a task has expired.
ezang: You can figure it out
ptp, it is your turn, lol
EzAng
It's not hard. Find a person, ask where the office is and go to it.
I hit jer's foot he did answer?
He said you drove into his foot. But you didn't find his office - or ask where it was. You just kept pressing the "Excuse me" button over and over and over and over and over and over and over and over and over and over and over and over and over and over and over and over and over and over and over and over and over and over and over and over and over and over and over and over and over and over and over and over and over and over and over and over and over and over and over and over and over and over and over and over and over and over and over and over and over and over and over and over and over and over and over and over and over and over and over and over and over and over and over and over and over and over and over and over and over and over and over and over and over and over and over and over and over and over and over and over and over and over and over and over and over and over and over and over and over and over and over and over and over and over and over and over and over and over and over and over and over LOL
I pressed it 2 x
The way the rooms looks to go and the bottom left camera on my screen are not in synch it seems
EzAng
nallycat try again
@DJ Sures I only pressed it 3 times maybe 4 and EzAng is correct the camera is not in synch.
Hey, I was wondering....where is Jeremie's office?
I saw the way but
my robot froze can't move
@Mickey666Maus You have to find it. I thought Jer's office was the game consoles. just kidding Jer.
OMG...I edited my post three times to get Jerimies name right!! I am drunk!!
Ready for Exoshpere...you are up Ezang?
@Mickey666Maus This is in Canada not Germany it is illegal to drive Robots Drunk.
Where did everyone go? lol
EzAng
come back and try it
very good admin
how do you get it to ask a question? like directions, excuse me
EzAng
Add a script in the exosphere control
ok I will try
EzAng
I figures it out
EzAng
find Jeremy's office today?
Ezang
Added optional two way telepresence option in config menu in latest update
I will try it soon as I back from the farm tomorrow.
Thanks,
EzAng
Ok, I feel like I'm late to the party here. We can start controlling each other's robots? Oh my!
Updated to not prompt for ML reinforcement questions when used in telepresence mode
Updated for a few bug fixes
How about the custom control configuration DJ? Any plans?
Yah - with api access there’s a few enterprise customers that have custom UI. Api access isn’t available with pro. It’s documented on the ARC product page
in the meantime for pro users, create scripts and the buttons will show up.
Version 67 has been updated with bug fixes. Specifically, recovery from network error will launch the Completed script rather than sit idle after presenting the error.
Updated v68 to remove the Use Custom User Interface option in preparation for a significant ARC update that deprecates this functionality and replaces it with a new feature.
v70 has been updated with a major fix for the telepresence mode, where the full-screen video window would not close after a client disconnected.
There are also several performance and stability improvements.
v71 has a new script that executes for speaking text entered by the remote operator. This option can be enabled on the Advanced tab. Before, it would use the default built-in speech synthesis. With this script, you can use your preferred text-to-speech engine.
v72 has been updated for new features of ARC v2025.03.05.00. This also includes some optimizations for performance.
v73 has been updated with several new error handling capture if network is disconnected
v74 updated to let errors bubble to the top of the call stack when there's a disconnection so the disconnect is handled correctly