Enable real-time object vision training using scripts
How to add the Train Vision Object By Script robot skill
- Load the most recent release of ARC (Get ARC).
- Press the Project tab from the top menu bar in ARC.
- Press Add Robot Skill from the button ribbon bar in ARC.
- Choose the Camera category tab.
- Press the Train Vision Object By Script icon to add the robot skill to your project.
Don't have a robot yet?
Follow the Getting Started Guide to build a robot and use the Train Vision Object By Script robot skill.
How to use the Train Vision Object By Script robot skill
In order to train objects to the camera device, the Train Object menu is used in the camera device skill. This menu normally requires human intervention to enter the object name and use the mouse. This skill allows your robot to learn objects in real-time triggered by ControlCommand() scripts programmatically. Take a look at the Cheat Sheet within the Script skill settings to see what ControlCommand() this skill accepts.
Main Window
1. Beginning Learning Button This button will attach/detach the camera configured in the camera device skill to the program with pictures skill window.
2. Attach/Detach Camera Button This button will attach/detach the camera configured in the camera device skill to the Train Vision Object by Script skill window.
3. Train Status Bar This will show the object training process time remaining. During this time move the object around to train it at different angles.
4. Status Display This field will display the status and name of the learned object and any errors that occur.
How To Use the Train Vision Object by Script Skill
Add a camera device skill (Project -> Add Skill -> Camera -> Camera Device).
Connect to a camera.
Add this skill (Project -> Add skill -> Camera -> Train Vision Object by Script).
Press the ATTACH button. This will attach this skill to the camera device skill. You will now see a mirror of the camera's video stream in this skill.
Center the onscreen pink box on the object that will be trained. Have the object fill up the pink box completely. Press the BEGIN LEARNING button, then physically move the object around during the training process to get multiple angles.
Wait until the Train Status bar has completed and make note of the object name in the Status display so that you can use it in your scripts.
Script Sample
Here's an script sample that learns an object titled DJ FACE via ez-script:
# Check if the camera is active. If not, exit the script and notify user
if (!$IsCameraActive)
say("The camera is not active")
halt()
endif
# This is the name of the object that will be stored
$LearnedObject = "DJ FACE"
# Attach to the camera for the control
ControlCommand("Train Vision Object By Script", Attach)
# Introduce verbally out of the PC speaker what it is about to happen
SayWait("Prepare to learn " + $LearnedObject + ". 3, 2, 1, learning")
# Instruct the skill to begin learning
ControlCommand("Train Vision Object By Script", Learn, $LearnedObject)
# Wait until the skill has completed learning the object
WaitFor($CameraIsLearning == 0)
# Speak what the object has been learned
Say("I have learned " + $LearnedObject)
# Enable object tracking in the camera control
ControlCommand("Camera", CameraObjectTracking)
Hi EzAng
Thanks for this. I tried it out, but it gave an error about using the "." for the $LearnedObject = line.
Also, just wanted to clarify, the first part of the program, the Audio.say( ) gets entered in the Bing Speech script or in the camera ?
follow the one below
did it work for you?
HI EzAng, Thanks for the quick reply
I must be missing something, Still cannot get it to work here. Its late so I will try again tomorrow hopefully.
ttyl
Hi, good morning, try this:
DJ's code Add:
Camera code in camera
Audio.sayWait("I see the ");
Audio.say (getVar("$CameraObjectName"));
Conversational Menu Bing Speech Recognition Train Object By Script Speech Recognition
In Speech Recognition add:
learn a new object
code: Audio.sayWait("what is the name of the new object you want me to learn");
setVar("$BingSpeech", "" );
ControlCommand("Bing Speech Recognition", "StartListening");
while (getVar("$BingSpeech") == "" );
sleep(250);
if (getVar("$BingSpeech") == "cancel"){
Audio.sayWait("cancelling");
return; }
Audio.sayWait("Let me know when you are ready to teach me what a " + getVar("$BingSpeech") + "is." );
var resp = Audio.waitForSpeech(60, "ok", "cancel");
if (resp != "ok"){ Audio.sayWait("ok, I will not learn the "+ getVar("$BingSpeech"));
return; }
ControlCommand("Camera", "CameraObjectTrackingDisable");
ControlCommand("Train Vision Object By Script", "Learn", getVar("$BingSpeech"));
sleep(1000);
while (getVar("$CameraIsLearning"));
sleep(250);
Audio.SayWait("Great, I know what a " + getVar("$BingSpeech") + "is.");
ControlCommand("Camera", "CameraObjectTracking");
It works here, let me know
HI EzAng,
thanks again. I will hopefully try it later on tonight. One question, that Im still stuck on. The First part, Where you say code in camera , is to add the Audio.say command script into the Camera? Do I enter this under the Scripts&variables tab inside the camera config? (add to camera start under Activity tag?) Thats what Im a bit lost with :-) Yeah I'm a lil bit out of practice with this stuff. And its a Java thing right? Rest of this I think I got.
Will def let you know if I get it working here. Look forward to it.
Ps. Hopefully soon to start another bot build. I have a wheeled platform kit coming from SuperDroid on the way to start things off.
Great,
Yes, add Camera, then in the Scripts&variables tab inside the camera config
add: Audio.sayWait("I see the "); Audio.say (getVar("$CameraObjectName"));
correct the rest is java script is the "Speech Recognition" control
include the: Conversational Menu Bing Speech Recognition Train Object By Script
and you are set to go
EzAng, What does the Conversational Menu do for this script? I assume that when I press START in the Bing Speech I then ask it to "learn a new Object", then I click to STOP after I state name ofobject before it continues on?