Object detection is fundemental to computer vision: Recognize what the objects are inside the robot camera and also where they are in the image.
How to add the Tiny Yolo2 robot skill
- Load the most recent release of ARC (Get ARC).
- Press the Project tab from the top menu bar in ARC.
- Press Add Robot Skill from the button ribbon bar in ARC.
- Choose the Camera category tab.
- Press the Tiny Yolo2 icon to add the robot skill to your project.
Don't have a robot yet?
Follow the Getting Started Guide to build a robot and use the Tiny Yolo2 robot skill.
How to use the Tiny Yolo2 robot skill
Object detection is fundamental to computer vision: Recognize the objects inside the robot camera and where they are in the image. This robot skill attaches to the Camera Device robot skill to obtain the video feed for detection.
Demo
Directions
Add a Camera Device robot skill to the project
Add this robot skill to the project. Check the robot skill's log view to ensure the robot skill has loaded the model correctly.
START the camera device robot skill, so it displays a video stream
By default, the TinyYolo skill will not detect objects actively. Check the "Active" checkbox to begin processing the camera video data stream.
Detected objects use the Camera Device robot skill features. The tracking On Tracking Start script will execute when objects are detected, and $CameraObject_____ variables will be populated. Check the Camera Device robot skill page for a list of camera variables.
Camera Device Integration
This robot skill integrates with the camera device by using the tracking features. If the servo tracking is enabled, this robot skill will move the servos. This is an extension of the camera robot skill. The On Tracking Start script will execute, and camera device variables will be populated when tracking objects.Performance
In HD webcam resolution, Tiny Yolo is processing 30+ FPS with 38% CPU, sometimes more, depending on the processor of your PC.
Variables
The detected objects are stored in global variables in the array provided by the camera robot skill. The number of detected objects determines the size of the array. The detected object's location, confidence, and name are all stored in variables. Detected objects use the Camera Device robot skill features. The tracking On Tracking Start script will execute when objects are detected, and $CameraObject_____ variables will be populated. Check the Camera Device robot skill page for a list of camera variables.
Trained Objects
Tiny Yolo robot skill includes an ONNX model with 20 trained objects. They are... "aeroplane", "bicycle", "bird", "boat", "bottle", "bus", "car", "cat", "chair", "cow", "diningtable", "dog", "horse", "motorbike", "person", "pottedplant", "sheep", "sofa", "train", "tvmonitor"The ImageNetData is trained with the image resolution of 416x416 because it uses the TINY model. Regular-sized models are trained at 608x608.
ToDo
- control commands for starting, stopping
- ability to specify custom training model files
Hi DJ. I noticed the boxes that appear around detected objects onscreen. The text was tiny and unreadable in the highest resolution I was previously stuck with. I had expected to see the plugin updating with the detected objects because that is the way it works in the video.
Does the new debug work for you? I can make it so the debug value saves with the project configuration as well - if it's something you'd like to see on every frame
Are the pictures captured in a file, stored on your laptop like PTP's DarkNet Yolo stores the files is a captured files ?
The reason I a asking is, I like deleting the captured files to save on storage on my laptop - I do delete the DArkNet Yolo files / pictures in the C:...Documents\YOLO.Plugin\Captures folder
thanks
EzAng
There’s no files saved. Do not delete any files
got it
thanks
EzAng
Any chance that I can load my own Yolo or ONNX trained neural network in this skill?
Yeah, you totally can. There's a folder that contains the data file. It'll be in the plugin folder, which should be something like...
I can quickly change the robot skill to allow you to select your own data file if you'd like. I don't think that exists in this.
Nice!. I can make a Kaggle notebook (when I have some time) with instructions how people can train their own neural network with Yolo and export it to ONNX. I will look like this: https://www.kaggle.com/code/charel/yolov5-1st-place-world-championships-robocup-2021 Maybe I can send you a trained NN first to test.