8. Vision
Let's choose your robot's vision system! If your robot does not have a camera, skip to the next step.
Robots with cameras provide navigation, tracking, and interactive benefits. Synthiam ARC's software is committed to making robot programming easy, including computer vision tracking. The ARC software includes a robot camera skill to connect WiFi, USB, or video capture devices. ARC's camera device skills include tracking types for objects, colors, motions, glyphs, faces, and more. You can add additional tracking types and computer vision modules from the skill store.
Choose the Camera Type
Connects directly to a computer with a USB cable. You can only use this type of camera in an embedded computer configuration. This is because the USB cable will tether the camera to the PC. You can use any USB camera in ARC. Some advantages of using USB cameras are high resolution and increased framerate.
Connects wirelessly to a PC/SBC over a WiFi connection. Generally, this approach is only used in remote computer configurations. Few I/O controllers support a WiFi wireless camera transmission due to latency causing low resolution and potentially unreliable radio interference. For a wireless camera application, the most popular I/O controllers are the EZ-Robot EZ-B v4 and IoTiny.
Add Camera Device Robot Skill
Whichever camera type you choose, the robot skill needed to connect to the camera is the Camera Device Robot Skill. Add this robot skill to connect to the camera and begin viewing the video feed. Reading the manual for this robot skill, you will find many options for tracking objects.
Add Camera Device Robot SkillAdditional Computer Vision Robot Skills
Now that you have the camera working on your robot, you may wish to add additional computer vision robot skills. Computer vision is a general term for extracting and processing information from images. The computer vision robot skills will use artificial intelligence and machine learning to track objects, detect colors, and recognize faces. There are even robot skills to detect facial expressions to determine your mood.
This skill will overlay an image on any detected object, face, color or glyph. Any type of detectable tracking type in the ARC Camera skill can be used. Simply select your image and voila! Its best to use a transparent PNG<img src=/images/emoticons/smiles.svg alt=:) class=emoji /> <h2>Main Window</h2> <div class="text-center"><a href="/uploads/user/F894A22A79D8EE11B554FCE4A5AA349F/fgias00l.PNG" target=_blank><img class="img-fluid pt-1...
Add this skill to your ARC project and bring the camera to life. The video will be interactive, where you can click on objects to center. There are hot spots along the edges which moves the camera as well. <h2>Main Window</h2> <div class="text-center"><a href="/uploads/user/F894A22A79D8EE11B554FCE4A5AA349F/qstjeyvi.PNG" target=_blank><img class="img-fluid pt-1 pb-1"...
This skill uses any camera installed on your PC or robot to combine computer vision tracking with movement and data acquisition. Computer vision is an experimental technology that requires a clean and bright environment to detect objects and colors accurately. If a camera is mounted on your robot, you can use this skill to track color objects, motion, human faces, or view the camera image. The attributes of the detected image can be adjusted manually in this skill to remove any false positives....
This skill only works with the Object tracking inside the <a href="//synthiam.com/Docs/Skills/Camera/Camera-Device?id=16120" target=_blank rel="nofollow">Camera Device skill</a>. This skill displays the detected object name on the Camera Device skill video stream. <h2>Main Window</h2> <div class="text-center"><a href="/uploads/user/F894A22A79D8EE11B554FCE4A5AA349F/yyrexyg5.PNG" target=_blank><img class="img-fluid...
<p> This skill will save snapshots from an active camera in the camera device skill. It will save a picture to your drive or device storage. The image will be saved to a folder called "My Robot Pictures" on your drive in the "Pictures" directory. This skill can also take a snapshot at a time interval, modified in the settings menu. </p> <p> You may also instruct the <a...
Overlay image packs onto the camera and control them using a specified control variable. <strong>GENERAL USE</strong> Choose an image pack from the drop down menu next to the <em>Overlay</em> button. Set the x and y positions, the width and height, and the variable that will control this image pack. In the <em>Auto Assign</em> tab set the min and max values of the control variable. Press Auto Assign. Start the camera. Press the <em>Overlay</em>...
<script type="text/javascript" src="/SyntaxHighlighter/scripts/shCore.js"></script> <script type="text/javascript" src="/SyntaxHighlighter/scripts/shBrushCSharp.js"></script> <link type="text/css" rel="stylesheet" href="/SyntaxHighlighter/styles/shCoreDefault.css" /> <script type="text/javascript">SyntaxHighlighter.all();</script> <p> Use the Microsoft Cognitive Emotion...
<p> Use the Cognitive Face cloud service to detect faces, describe emotions, guess age and get the persons name from a worldwide database. The images come from the Camera Device robot skill added to the project. This plugin requires an internet connection. Please consult the appropriate lessons in the learn section to configure your EZ-B to WiFi client mode or add a second USB WiFi adapter. </p> <br /> <br /> <h3>Currently Disabled</h3> <p> This robot...
<script type="text/javascript" src="/SyntaxHighlighter/scripts/shCore.js"></script> <script type="text/javascript" src="/SyntaxHighlighter/scripts/shBrushCSharp.js"></script> <link type="text/css" rel="stylesheet" href="/SyntaxHighlighter/styles/shCoreDefault.css" /> <script type="text/javascript">SyntaxHighlighter.all();</script> <p> Use the Microsoft Cognitive...
You only look once (YOLO) is a state-of-the-art, real-time object detection system. using Tiny YOLOv3 a very small model as well for constrained environments (CPU Only, NO GPU) Darket YOLO website: https://pjreddie.com/darknet/yolo/ Requirements: You only need a camera control, the detection is done offline (no cloud services). <a href="/uploads/user/2B1D28F2F5CE67940FB6592880013B63/b4qbezka.PNG" target=_blank><img class="img-fluid pt-1 pb-1"...
This Skill enables the control of your robots servos by moving the joints of your body, which are detected by a <strong>Microsoft XBOX 360 Kinect only</strong>. Servos can be assigned to each joint using the Settings window. Degrees to move the servos are automatically calculated for joints about the connecting joint. For example, the wrist position in degrees is calculated based on the elbow position. The elbow position in degrees is calculated based on the shoulder position. Each...
This control allows you to broadcast live audio and video from the camera control to the Web. Live stream implements HLS protocol from Apple and works cross-browser. You will have to configure your router to access the live broadcast link from external networks. - If you would like to serve a webpage with an embedded video stream without audio, check out the <a href="//synthiam.com/Products/Controls/General/HTTP-Server-Custom-16126" target=_blank rel="nofollow">Custom...
This control listens to incoming live stream connection from web and playbacks the video and audio stream inside ARC. With this control you can open a web page (currently supports Chrome and Firefox on Desktop) from anywhere and start live streaming directly to ARC camera control. Network configuration might be required to access the server. If you are looking for boadcasting camera feed to web check out <a href="//synthiam.com/Products/Controls/Camera/Live-Stream-Broadcast-18664"...
Omron HVC-P plugin for ARC (onboard computer). This is used in Rafiki. These are some case STL files so that you can protect this sensor while messing with it. <a href="/uploads/user/9B549C4DF0F8E1BE4AE204F29F78C354/OmronCase2-635997953480734964.stl" rel="nofollow">OmronCase2.stl</a> <a href="/uploads/user/9B549C4DF0F8E1BE4AE204F29F78C354/OmronCase1-635997953654860973.stl" rel="nofollow">OmronCase1.stl</a> <a...
<strong>Required Download And Installation:</strong> Download <a href="//synthiam.com/uploads/user/9B549C4DF0F8E1BE4AE204F29F78C354/Python_Installs-635998022563766585.zip" target=_blank rel="nofollow">Python_Install_Zip</a> Unzip it and install Python and PySerial. This is an updated version of the original plugin David Cochran created for the Omron HVC-P which used the Omron EvaluationSoftware_v1.2.0 software for trained faces. My version of the...
Omron HVC-P2 plugin for ARC Windows, onboard PC required. This will also work with the original HVC-P. <strong>Required Download And Installation:</strong> Download <a href="//synthiam.com/uploads/user/9B549C4DF0F8E1BE4AE204F29F78C354/Python_Installs-635998022563766585.zip" target=_blank rel="nofollow">Python_Install_Zip</a> Unzip it and install Python and PySerial. This was created to be used with a <strong>second Omron HVCP(2)...
Integrate state-of-the-art image generation capabilities directly into your robot apps and products. DALLE 2 is a new AI system that can create realistic images and art from natural language descriptions. Have your robot programmatically generate images from speech recognition robot skills descriptions. Or, have the image from the camera sent to Dall-e for its AI interpretation. Experience the whacky world of AI image generation with your Synthiam-powered robot or webcam. <div...
This skill super imposes a camera video stream on top of another camera video stream. <h2>Main Window</h2> <div class="text-center"><a href="/uploads/user/F894A22A79D8EE11B554FCE4A5AA349F/4ltrp3c2.PNG" target=_blank><img class="img-fluid pt-1 pb-1" src="/uploads/user/F894A22A79D8EE11B554FCE4A5AA349F/4ltrp3c2.PNG" alt="User-inserted image"></a></div> <strong>1. Active Checkbox</strong> This...
Using the camera, this skill will allow you to add programming commands by holding up pieces of paper with images printed on them. The concept is to allow programming instructions using visual representations. Each card has an image that represents a specific command, such as move forward, turn right, turn left, or reverse. Using cue cards, the robot can be programmed for specific movements to navigate a maze, for example. The order in which the cards are shown to the robot are stored in memory...
<p> The QR Code Generator will create a QR Code with the text you enter. By default, the QR Code text is "Synthiam," and the QR Code Graphic is the same. Using your phones QR Code Scanner App to scan the graphic will say "Synthiam." This control works in conjunction with the Camera Control. </p> <p> QR codes, short for Quick Response codes, are two-dimensional barcodes that have become ubiquitous in various industries, including robotics. These codes consist...
Capture the output display of a robot skill and send it to a camera device. Specify the robot skill to capture, and it will send the video to the selected camera device. The Ultrasonic Radar Scan area is captured in this screenshot below and sent to the Camera Device as a video stream. <a href="/uploads/user/DB763BE15E695777689418BE7364E0A3/ci5fjyqe.png" target=_blank><img class="img-fluid pt-1 pb-1"...
Rubiks Cube Solving Robot skill. This skill is to be combined with a specific robot project to build. Find the robot project here: https://www.thingiverse.com/thing:2471044 *** Version 5 *** Fix for ARC 2020.02.19.00 release *** First Calibrate Arms & Grippers: <a href="/uploads/user/2B1D28F2F5CE67940FB6592880013B63/ArmsCalibration-636538449726891956.png" target=_blank><img class="img-fluid pt-1 pb-1"...
Capture any area of the screen and send it to a camera device. Specify the screen area to capture, and it will send the video to the selected camera device. You will need a camera device added to the project. Select the camera device from the configuration menu. In the camera device, select the Custom option in the device list. Then press the start button. *Note: the display resolution scaling must be 100% for accurate capture area.
The Sighthound Cloud Detection API returns the location of any people and faces found in robot camera video. Faces can be analyzed for gender, age, pose, or emotion; and a landmark detector can find the various facial features in the detected faces, including eyes, nose and mouth, by fitting 68 landmark points to those features. <a href="/uploads/user/DB763BE15E695777689418BE7364E0A3/wsg3ymgx.PNG" target=_blank><img class="img-fluid pt-1 pb-1"...
Stream all video sources from any video URI protocol or codec (RTMP, m3u, m3u8, Mkv, MOV, mpg, etc.). The video stream is sent to the selected camera device. This supports webcams or any type of video device/service that provides a video feed over a network. <a href="/uploads/user/DB763BE15E695777689418BE7364E0A3/b53tbbsy.png" target=_blank><img class="img-fluid pt-1 pb-1" src="/uploads/user/DB763BE15E695777689418BE7364E0A3/b53tbbsy.png"...
Select one of the included templates, or select your own. The images are translucent PNG files which are overlaid on each frame of the camera stream. <div class="text-center"><a href="/uploads/user/F894A22A79D8EE11B554FCE4A5AA349F/n3rpu2ko.PNG" target=_blank><img class="img-fluid pt-1 pb-1" src="/uploads/user/F894A22A79D8EE11B554FCE4A5AA349F/n3rpu2ko.PNG" alt="User-inserted image"></a></div> <h2>Main...
Display a variable on the processed camera device image. Specify the X/Y coordinates of the variable location, and the variable name. There are ControlCommand() for attaching the skill to a specific camera device, or use any available device.
Object detection is fundamental to computer vision: Recognize the objects inside the robot camera and where they are in the image. This robot skill attaches to the Camera Device robot skill to obtain the video feed for detection. <h2>Demo</h2> <div class="embed-responsive embed-responsive-16by9"><iframe class="embed-responsive-item" src=https://www.youtube.com/embed/DKqWFrKd_Ck?rel=0&autoplay=0 frameborder=0 height=300 width=480...
In order to train objects to the camera device, the Train Object menu is used in the camera device skill. This menu normally requires human intervention to enter the object name and use the mouse. This skill allows your robot to learn objects in real-time triggered by controlcommand() scripts programmatically. Take a look at the cheat sheet within the Script skill settings to see what controlcommand() this skill accepts. <div class="text-center"><a...
<strong>*Note: </strong> Vuzixs decision to no longer support the Synthiam platform with their newer products. Therefore, this control is limited to the deprecated 920VR headset, which may be used on eBay. There will be no further development on this control. For headset support, we recommend the <a href="/Products/Controls/Virtual-Reality/Virtual-Reality-Robot-15982">Virtual Reality Robot</a>.<br /> <Br /> The Vuzix augmented reality control enables...