Object detection is fundemental to computer vision: Recognize what the objects are inside the robot camera and also where they are in the image.
How to add the Tiny Yolo2 robot skill
- Load the most recent release of ARC (Get ARC).
- Press the Project tab from the top menu bar in ARC.
- Press Add Robot Skill from the button ribbon bar in ARC.
- Choose the Camera category tab.
- Press the Tiny Yolo2 icon to add the robot skill to your project.
Don't have a robot yet?
Follow the Getting Started Guide to build a robot and use the Tiny Yolo2 robot skill.
How to use the Tiny Yolo2 robot skill
Object detection is fundamental to computer vision: Recognize the objects inside the robot camera and where they are in the image. This robot skill attaches to the Camera Device robot skill to obtain the video feed for detection.
Demo
Directions
Add a Camera Device robot skill to the project
Add this robot skill to the project. Check the robot skill's log view to ensure the robot skill has loaded the model correctly.
START the camera device robot skill, so it displays a video stream
By default, the TinyYolo skill will not detect objects actively. Check the "Active" checkbox to begin processing the camera video data stream.
Detected objects use the Camera Device robot skill features. The tracking On Tracking Start script will execute when objects are detected, and $CameraObject_____ variables will be populated. Check the Camera Device robot skill page for a list of camera variables.
Camera Device Integration
This robot skill integrates with the camera device by using the tracking features. If the servo tracking is enabled, this robot skill will move the servos. This is an extension of the camera robot skill. The On Tracking Start script will execute, and camera device variables will be populated when tracking objects.Performance
In HD webcam resolution, Tiny Yolo is processing 30+ FPS with 38% CPU, sometimes more, depending on the processor of your PC.
Variables
The detected objects are stored in global variables in the array provided by the camera robot skill. The number of detected objects determines the size of the array. The detected object's location, confidence, and name are all stored in variables. Detected objects use the Camera Device robot skill features. The tracking On Tracking Start script will execute when objects are detected, and $CameraObject_____ variables will be populated. Check the Camera Device robot skill page for a list of camera variables.
Trained Objects
Tiny Yolo robot skill includes an ONNX model with 20 trained objects. They are... "aeroplane", "bicycle", "bird", "boat", "bottle", "bus", "car", "cat", "chair", "cow", "diningtable", "dog", "horse", "motorbike", "person", "pottedplant", "sheep", "sofa", "train", "tvmonitor"The ImageNetData is trained with the image resolution of 416x416 because it uses the TINY model. Regular-sized models are trained at 608x608.
ToDo
- control commands for starting, stopping
- ability to specify custom training model files
This looks interesting will give it a try!
I tried it again, only can see "a person" , thinks my iPhone in a tv-monitor, :-)
Also the box that pops up is to the left of the "person" it sees - the person is not in the box
Will keep trying
V8 not working for me. I get this message and no scrolling of detected items.
You'll have to press the ACTIVE button - and an active camera will need to be running. You can see the instructions in video above
I do have an active camera running and the ACTIVE button is depressed as in the pic above. Followed the vid to a T. That is all I ever see in the plugin. I do notice in the variable watcher that it is detecting and the variable changes when it detects me or something else, but the plugin never changes.
DJ
I tried it again, only can see "a person" , thinks my iPhone in a tv-monitor, :-)
Also the box that pops up is to the left of the "person" it sees - the person is not in the box
Fixed the boxes around the detected objects
Perry, the plugin doesn't change or display anything. The objects detected are in the camera view and the variable manager. They're not in the display of the plugin. Would it help if they were also listed in the plugin display for you to see in there as well?
Perry, I added a DEBUG checkbox that will display debug information in the plugin of detected objects as i think that's what you'd like to see? The detected object will also be highlighted in the camera device