8. Vision

Let's choose your robot's vision system! If your robot does not have a camera, skip to the next step.

Robots with cameras provide navigation, tracking, and interactive benefits. Synthiam ARC's software is committed to making robot programming easy, including computer vision tracking. The ARC software includes a robot camera skill to connect WiFi, USB, or video capture devices. ARC's camera device skills include tracking types for objects, colors, motions, glyphs, faces, and more. You can add additional tracking types and computer vision modules from the skill store.

Choose the Camera Type

Wired USB Camera
Connects directly to a computer with a USB cable. You can only use this type of camera in an embedded computer configuration. This is because the USB cable will tether the camera to the PC. You can use any USB camera in ARC. Some advantages of using USB cameras are high resolution and increased framerate.

Wireless Camera
Connects wirelessly to a PC/SBC over a WiFi connection. Generally, this approach is only used in remote computer configurations. Few I/O controllers support a WiFi wireless camera transmission due to latency causing low resolution and potentially unreliable radio interference. For a wireless camera application, the most popular I/O controllers are the EZ-Robot EZ-B v4 and IoTiny.

Add Camera Device Robot Skill

Whichever camera type you choose, the robot skill needed to connect to the camera is the Camera Device Robot Skill. Add this robot skill to connect to the camera and begin viewing the video feed. Reading the manual for this robot skill, you will find many options for tracking objects.

Add Camera Device Robot Skill

Additional Computer Vision Robot Skills

Now that you have the camera working on your robot, you may wish to add additional computer vision robot skills. Computer vision is a general term for extracting and processing information from images. The computer vision robot skills will use artificial intelligence and machine learning to track objects, detect colors, and recognize faces. There are even robot skills to detect facial expressions to determine your mood.

Augmented Reality by Synthiam

Overlay PNG/JPG images in real-time onto detected objects, faces, colors or glyphs using ARC Camera tracking; attach, preview, detach.

Requires ARC v9

Camera Click Servo by Synthiam

Interactive camera control for ARC: click-to-center and edge hotspots to pan/tilt servos, adjustable fine-tune and multi-camera support.

Requires ARC v12

Camera Device by Synthiam

Use PC or robot cameras for vision tracking: color, QR, glyph, face, object, motion; record video/data, control servos and movement, run scripts

Camera Overlay by Synthiam

Displays trained object names as overlays on the Camera Device video stream using Camera Device's object-tracking (shows $CameraObjectName).

Requires ARC v9

Camera Snapshot by Synthiam

Save camera snapshots to 'My Robot Pictures' (Pictures folder), manual or timed (0-100s), and trigger programmatically via controlCommand.

Camera Source (Ezb) by Synthiam Inc.

Enable an EZB video camera as ARC's camera source for video recording, vision recognition and other robot vision features.

Requires ARCx v1

Camera Source (Usb) by Synthiam Inc.

USB camera video source for recognition, recording and other vision tasks in ARC.

Requires ARCx v3

Camera Variable Overlay by James Graham Hu

Overlay image packs onto camera feed and map images to control-variable ranges with auto/manual assignment, position and sizing options.

Requires ARC v5

Cognitive Emotion by Microsoft

Uses Microsoft Cognitive Emotion cloud to analyze camera images, returning emotion descriptions and confidence for speech/output (requires internet).

Cognitive Vision by Microsoft

Microsoft Cognitive Vision integration for ARC: real-time object, face, emotion detection, OCR, confidence data and script-triggered robot actions.

Darknet YOLO (Obj Detection) by ptp

Tiny YOLOv3 CPU-only real-time object detection using a camera; offline detection, script-triggered on-changes or on-demand results with class/scores.

Requires ARC v5

Face Tracking by Synthiam Inc.

Detects and tracks faces from any ARC video source, providing real-time face localization for robot applications.

Requires ARCx v2

Kinect Body Control by Microsoft

Control robot servos via Xbox 360 Kinect body joints with per-joint servo mapping, automatic degree calculation, smoothing and upper/lower control

Requires ARC v11

Live Stream Broadcast by Amin

Broadcast live audio and video from ARC camera to the web via HLS; cross-browser streaming. Requires router configuration for external access.

Requires ARC v9

Live Stream Receiver by Amin

Play live web video and audio streams inside ARC from Chrome/Firefox, streaming directly to ARC camera control; network configuration may be needed.

Requires ARC v7

Omron HVC-P by CochranRobotics

Omron HVC-P ARC plugin: real-time body, hand, face, gaze, gender, age, expression and eye estimation; facial recognition via Omron software

Requires ARC v7

Omron HVC-P2 by JustinRatliff

Omron HVC-P/HVC-P2 ARC plugin: Python-based camera integration for body, hand and face detection, gaze, age/gender, expressions and face recognition.

Requires ARC v6

Omron HVC-P2 (Second Camera) by JustinRatliff

Windows ARC plugin for Omron HVC-P/HVC-P2: Python-based face/body/hand detection, gaze, age/gender, expressions and recognition; dual-camera support.

Requires ARC v7

Openai Dall-E Image by OpenAI

Generate and modify images with DALL·E 2 inside Synthiam ARC robots-create images from text or camera input via API and control commands.

Requires ARC v3

Picture In Picture by Synthiam

Picture-in-picture camera overlay: superimpose one camera onto another; configure source/destination, position, size, border, processed frames, and swap

Requires ARC v8

Programming With Pictures by Synthiam

Use camera-detected printed cue cards (direction, pause, start) to record and run stored movement sequences for ARC robots.

Requires ARC v9

QR Code Generator by Synthiam

Create customizable QR codes for ARC, display/scan via Camera Control, trigger scripts on recognition and save decoded text to variables.

Robot Skill To Camera by Synthiam Inc.

Capture a robot skill's display and stream selected area as video to a configured camera device with FPS and live preview crop.

Requires ARC v5

Rubik's Cube Solving Robot by ptp

Control a custom ARC robot to manipulate and solve a Rubik's Cube using calibrated arms and grippers; integrates with Thingiverse build.

Requires ARC v5

Screen To Camera by Synthiam Inc.

Capture any screen area and stream video to a configured camera device; requires a Custom device and 100% display scaling.

Requires ARC v7

Sighthound Cloud API by SightHound

Cloud-based detection of people and faces in robot camera video; returns locations, gender, age, pose, emotion, plus 68-point facial landmarks.

Requires ARC v12

Stream All The Video Sources by Synthiam

Stream video from any URI/protocol (RTMP, RTSP, HTTP, UDP, etc.) to a selected ARC camera device for real-time network feed playback.

Requires ARC v3

Target Overlay by Synthiam

Overlay translucent PNG targets onto a camera stream with loadable templates, attach/detach control, preview and status for visual alignment.

Requires ARC v8

Text Overlay by Synthiam

Overlay a variable on a processed camera image at X/Y coordinates; name the variable and attach to a specific or any available camera

Requires ARC v2

Tiny Yolo2 by Synthiam

Real-time TinyYolo object detection for ARC Camera Device: tracks 20 classes, populates camera variables, triggers tracking scripts, 30+ FPS in HD.

Requires ARC v13

Train Vision Object By Script by Synthiam

Train camera vision objects via controlcommand(), attach camera, start learning, monitor progress, and return object name for scripts.

Requires ARC v6

Video Recorder by Synthiam Inc.

Record any video source to a local file for playback, archiving, and processing.

Requires ARCx v1

Vuzix Augmented Reality by Vuzix

Connect Vuzix 920VR AR headset to ARC to map head movement to robot servos or drive, control camera pan/tilt; deprecated 920VR support only.

Next Step