Object detection is fundemental to computer vision: Recognize what the objects are inside the robot camera and also where they are in the image.
How to add the Tiny Yolo2 robot skill
- Load the most recent release of ARC (Get ARC).
- Press the Project tab from the top menu bar in ARC.
- Press Add Robot Skill from the button ribbon bar in ARC.
- Choose the Camera category tab.
- Press the Tiny Yolo2 icon to add the robot skill to your project.
Don't have a robot yet?
Follow the Getting Started Guide to build a robot and use the Tiny Yolo2 robot skill.
How to use the Tiny Yolo2 robot skill
Object detection is fundamental to computer vision: Recognize the objects inside the robot camera and where they are in the image. This robot skill attaches to the Camera Device robot skill to obtain the video feed for detection.
Demo
Directions
Add a Camera Device robot skill to the project
Add this robot skill to the project. Check the robot skill's log view to ensure the robot skill has loaded the model correctly.
START the camera device robot skill, so it displays a video stream
By default, the TinyYolo skill will not detect objects actively. Check the "Active" checkbox to begin processing the camera video data stream.
Detected objects use the Camera Device robot skill features. The tracking On Tracking Start script will execute when objects are detected, and $CameraObject_____ variables will be populated. Check the Camera Device robot skill page for a list of camera variables.
Camera Device Integration
This robot skill integrates with the camera device by using the tracking features. If the servo tracking is enabled, this robot skill will move the servos. This is an extension of the camera robot skill. The On Tracking Start script will execute, and camera device variables will be populated when tracking objects.Performance
In HD webcam resolution, Tiny Yolo is processing 30+ FPS with 38% CPU, sometimes more, depending on the processor of your PC.
Variables
The detected objects are stored in global variables in the array provided by the camera robot skill. The number of detected objects determines the size of the array. The detected object's location, confidence, and name are all stored in variables. Detected objects use the Camera Device robot skill features. The tracking On Tracking Start script will execute when objects are detected, and $CameraObject_____ variables will be populated. Check the Camera Device robot skill page for a list of camera variables.
Trained Objects
Tiny Yolo robot skill includes an ONNX model with 20 trained objects. They are... "aeroplane", "bicycle", "bird", "boat", "bottle", "bus", "car", "cat", "chair", "cow", "diningtable", "dog", "horse", "motorbike", "person", "pottedplant", "sheep", "sofa", "train", "tvmonitor"The ImageNetData is trained with the image resolution of 416x416 because it uses the TINY model. Regular-sized models are trained at 608x608.
ToDo
- control commands for starting, stopping
- ability to specify custom training model files
It appears that you are encountering a memory allocation issue with the Tiny Yolo2 model. This could be due to insufficient system resources or other limitations on your machine. Here are a few steps you can try to resolve the issue:
Restart your computer: This can free up memory and system resources that may be tied up by other processes.
Close other applications: Make sure to close any unnecessary applications that may be consuming memory and processing power.
Update ARC: Ensure that you are running the latest version of ARC, as updates often include bug fixes and performance improvements.
Check system requirements: Verify that your computer meets the minimum system requirements for running ARC and the Tiny Yolo2 model. You may need a machine with more memory or a better GPU.
Reduce camera resolution: Lowering the resolution of the camera feed can reduce the amount of memory required for processing the video stream.
Check for updates to the Tiny Yolo2 skill: Sometimes, updates to robot skills can address issues like the one you're experiencing.
If you continue to have trouble after trying these steps, you may want to reach out for further assistance through the Synthiam support section at https://synthiam.com/Support. There, you can find additional resources and potentially contact support for more personalized help.
I am a customer support GPT-powered AI and still learning. If you have doubts about my response, please use the support documentation to verify my answer.
I think @athena is correct about the memory/resources for your PC. One of the most significant programs could be Chrome or a web browser, which usually takes many gigabytes of RAM. I tested the robot skill, which takes 700 MB max in ARC.
It's great to hear that you've tested the robot skill and have an idea of its memory usage. If you're experiencing issues with the Tiny Yolo2 model or any other robot skill, it's always a good idea to monitor your system's resource usage to ensure that there is enough memory and processing power available for ARC to operate smoothly. As mentioned earlier, closing unnecessary applications and browser tabs can help free up resources. If you have any further questions or need assistance with ARC or robot skills, feel free to reach out for support.
I am a customer support GPT-powered AI and still learning. If you have doubts about my response, please use the support documentation to verify my answer.
I am using a desktop PC with 32 GB and 12 core and only ARC running. I had added yolo to the project for the InMoov and it produced an error I closed ARC and opened up a blank project and just used yolo and camera and also error. Not sure if some type of memory / resource leak. Anyway rebooted after Athena answered again just loaded blank cam and yolo. Tester and it worked, so once again loaded full ez-robot inmoov project added yolo and produced same error.
I tried in a VM on my MAC (intel / parallels 8GB 4 core assigned) same issue. I haven’t fired up another windows PC yet but I will try and test yolo with just the cam and then with some other plugins added tomorrow.
I just tried again on 3rd computer #athena this time I used a Rock Pi X on your supported computer list. Only Yolo2 and Camera loaded no other apps. This time it errors on start of Yolo2 Loading models: C:\ProgramData\ARC\Plugins\19a75b67-c593-406c-9789-464aa3ba998b\models\TinyYolo2_model.onnx... Set Configuration: The type initializer for 'Microsoft.ML.OnnxRuntime.NativeMethods' threw an exception.
@athena read above post
It seems like you're encountering a different error on the Rock Pi X, which is related to the initialization of the Microsoft ML OnnxRuntime. This could be due to a compatibility issue with the OnnxRuntime and the hardware or operating system on the Rock Pi X. Here are a few steps you can try:
Check for ARC updates: Make sure you have the latest version of ARC installed, as updates may include compatibility fixes.
Install Visual C++ Redistributable: The OnnxRuntime may require the Visual C++ Redistributable for Visual Studio 2015, 2017, and 2019. You can download it from the official Microsoft website.
Check the OnnxRuntime compatibility: Ensure that the OnnxRuntime is compatible with the operating system on your Rock Pi X. You may need to look for any specific versions or requirements for ARM-based systems if that's what you're using.
Reinstall the Tiny Yolo2 skill: Try removing and then reinstalling the Tiny Yolo2 skill to ensure that all components are correctly installed.
Check for missing dependencies: There may be additional dependencies required by the OnnxRuntime that are not present on your system. Check the documentation for the OnnxRuntime to see if there are any other required components.
If you continue to experience issues after trying these steps, you may want to reach out for further assistance through the Synthiam support section at https://synthiam.com/Support. There, you can find additional resources and potentially contact support for more personalized help.
I am a customer support GPT-powered AI and still learning. If you have doubts about my response, please use the support documentation to verify my answer.
Rock pi is supported for ARC / not verified for 700 3rd party robot skills
the rock pi probably won’t run that robot skill because it’s missing enough gpu. You can look up the error on google to understand more.
I’m imagining the project combined with this robot skill is using too many resources. There’s a limit per application - so your 47473737gb of ram is irrelevant to that
. I guess this is why most tensor DLL stuff runs in separate applications domains. Each robot skill does run in a separate thread but it’s impossible to be a separate application domain.
Id look into an alternative solution or maybe consider looking at making the robot skill a stand-alone application that talks to ARC via some protocol.
or maybe a smaller image model? That would be a good solution.