Asked — Edited
Resolved Resolved by DJ Sures!

Color Recognition

Hi All,

Is there a step-by-step tutorial on how to get my robot to respond to color? I've seen the tutorials, but can't get it to work. I want to have Roli follow a red ball as was demonstrated with the AR Drone by DJ Sures.

I added the Camera Controls to the project. Selected (red) which is the default color. Under 'Tracking Types' I've selected Color.

I've attached a screen shot of the configuration window. Are my settings correct?

Tom User-inserted image


ARC Pro

Upgrade to ARC Pro

ARC Pro is more than a tool; it's a creative playground for robot enthusiasts, where you can turn your wildest ideas into reality.

#1  

@Tom... Do you want to track colour? If so you need to provide the x and y servo info (in the middle of your picture above). You need to specifiy the x axis servo port, the min an max values for it (usually 1 - 180) and then the same for the y axis servos (these are the servos in JD's or Roli's neck/head)... Your camera will have to be in a "pan and tilt" configuration like as mentioned JD's head or Roli's head (which I think you have)...

**** Note**** Also your board index should be 0 on both servos above

#2  

There is a video tutorial for color tracking from the Learn Section under ARC Manual. Have you seen this?

PRO
Synthiam
#3  

I believe the @tom is asking for the Roli to follow a ball by moving. The AR Drone is a flying drone robot and should not be used as a reference for the Roli robot, as the roli robot does not fly and therefore the settings recommended in that video cannot apply to Roli. I do believe I know what you're asking :).

FYI, the Question Mark (next to the X) on all controls brings you to instructions on how to use a control. When you load ARC, there is a window that tells you how to use the ? question mark buttons on controls. In your case, press the ? on the Camera Control.

User-inserted image

Here is some feedback to your excellent screenshare. Thank you for that! Makes it much easier to help :)

  1. The board index should be set to 0. You only have 1 robot, therefore only 1 ezb. You should be using the first connection in the connection control, if you followed the tutorials - so 0 is your board index. Did you find information about changing the board index somewhere? Please let me know so I can fix where it is referenced.

  2. Servo tracking should be disabled. You are asking to move the robot, not a servo. There is a blue question mark beside the servo checkbox that explains what it does. Hover over blue question marks to read what an option does. Uncheck the servo Tracking checkbox

  3. Enable Movement Tracking checkbox should be checked. You are correct by checking it. Hovering over the blue question mark would have told you what that option does. It allows your robot to move when tracking objects, and I believe that is what you're asking the robot to do

  4. Press SAVE on your config window. That's the button on the bottom of the image that you had screen shotted for us

  5. In the main Camera control, select your COLOR, Tracking type as "COLOR". You may need to adjust the brightness of the object under the COLOR tab. You only want the object that you are tracking to be detected, not your skin or anything else. Also make sure you have a nice bright room because computer vision requires that, as stated in the camera control manual. You can find the camera control manual by pressing the ? (question mark) next to the X button on the camera control.

  6. The GRID lines on the camera control will need to be fine tuned for your tracking. If the object is on the RIGHT of the right vertical grid, the robot will turn right. If the object is on the LEFT of the left vertical grid line, the robot will turn left.

Good luck!:D Should be easy from here!

#4  

Ok, thanks everyone. I have some homework to do. I'll get back to you tonight or tomorrow with the results.

Tom

PRO
Synthiam
#5  

k! good luck!:) We have lots of tutorials on the to-do list.. however, i'm the only one who makes them, and programs, and runs the company - so we're a little under staffed at the moment :D

Good news though, you can be the first to know - the iOS app is ready and should be in the store within the next week

#6  

Well, I had some luck last night. I followed all of the steps listed above - and yes, I'm fully aware of the ? marks and tutorials on this. I got Roli to respond to a red ball. It moved forward - it was jerky but it moved. No matter what I tried, I could not get it to move left or right. I assumed this had something to do with the brightness & grid settings. I decided to wait until today and try different lighting. You mentioned this was really important. I repeated everything using both natural (sunlight from a window) and (office) lighting. I didn't get any response from Roli whatsoever. I re-created that same environment as last night - still no response.

I rebooted, reconnected, checked all of the settings and checked again and again. Nothing changed from last night to today. I adjusted the color brightness and the grid lines. No response. I can see the "blue box" around the red ball. So yes, the camera is seeing the object. It just won't perform the action.

I checked to see if the actions in the Auto Position window worked and they were fine.

Just to confirm, I have the following settings established:

Servo Tracking - off Track by Relative Position - off Board Index (X & Y) - 0 Movement Tracking - enabled Allow Movement Forward - enabled Allow Left/Right Movement - enabled

All of these settings were saved.

In the main Camera window I have the following selected:

Tracking - Color and Motion Color - red Min Object Size - very small, although I have changed this to see if this would have an effect, no response. Object Brightness - various settings, no response. Grid - various settings, for the most part I have the grid divided into 9 equally sized squares. I've moved the ball all over the grid - no response - but as mentioned, the camera sees the ball and is positioning a blue square over it. I have the brightness tuned to the point where there is only 1 square. I see that when it is too bright, there are multiple squares and this triggered false positives as experienced last night.

All I want is for Roli to follow the red ball as I move it around. Help! What am I doing wrong?

#7  

@tom... is your Roli able to manually move left and right using the Movement Panel control? I mean can you drive him manually? Another thing is make sure there is good lighting on the object and the camera grid lines are set to default... If the object is too close and or occupying more than one grid square then it will be difficult for ARC to determine which direction (left or right) to move Roli in....

PRO
Synthiam
#8  

disable MOTION in the Tracking Modes.

MOTION is not what you're asking to track. From what I understand, you would like Roli to track a red ball. The tracking of a color is COLOR tracking, not MOTION tracking.

MOTION tracking is observing and tracking movement in the camera. It is not COLOR. To track a color, only select COLOR. Do not select other tracking methods.

Also, you would never want to track MOTION with Movement Enabled. This is for obvious reasons. The robot would move, therefore the entire image would have been tracked as a giant "motion".

Motion tracks changes within the image. Motion is an object moving. Color is a color, such as Red, Green or Blue.

And as Richard asked, can you move the robot using the movement panels? Meaning, does your Roli robot move? Because it sounds as if you did not connect to the robot and was only connected to the Camera.

Remember, the Camera and the Robot are two different things :)

Here a link to the tutorial that demonstrates how to connect to the robot: https://synthiam.com/Tutorials/Lesson/14

#9  

Crap!... I missed that... @Tom has motion tracking enable too... Tom all you need is colour tracking enabled for Roli to chase the ball... So as DJ mentioned, disable motion tracking...

PRO
Synthiam
#10  

I have added tracking information to this page: https://synthiam.com/Tutorials/Help.aspx?id=80

#11  

Ok - new development. I went back to the location I used last night and got better results. I have just two (florescent) lights in the vicinity and can get Roli to recognize the red ball.

To answer your question, yes I can drive him manually. I can use the buttons in the Movement Panel. Works like a charm. Never a problem there.

I now have the same problem as last night. Roli jerks around but does not follow the red ball as I move it. I adjusted the grid lines to the default setting and tried moving them to create a 3 x 3 square grid. I can barely get him to turn - maybe a few degrees but I certainly can't get him to follow the ball as I move it around.

I have the Object Brightness set all the way to Dim. If I raise it up, Roli jerks even more. The other problem is that even on the Dim setting, Roli sees furniture (mahogany colored) and treats this as Red. Only with a completely white background does the jerking stop.

Regarding the lighting, I tried a bright sunny area in my office (sun room) this morning. I figured the extra light would make it work better. As you saw in my last post, it didn't respond at all. Zero - it's as if Roli was turned off. There was no movement whatsoever.

So am I expecting too much? I'm a newbie with this. I'm going to show this at a conference next week and I thought this would be a great way to show the product. If I could have Roli follow the red ball in my booth, it would be a great attention getter. Obviously if I'm able to figure out the settings in my office, they will be certainly be different in the conference hall and I won't have the luxury of time to figure things out.

BTW - I'm not totally clueless with this! I've also got Six and JD, and have created two projects that work really well with Six (voice commands and basic movements), and have the Mobile interface on two Droid devices controlling everything. So I think I'll do OK at the conference, but the color recognition would be a super function if I could get it working the way I want.

Thanks in advance.

Tom

#12  

Just got your posts on Movement Tracking - I'll try that tonight....Thanks!

#13  

Gents, sorry this isn't working. In fact, Roli stopped responding to the red ball completely. All that has changed since my last post was disabling Motion Tracking as Rich suggested. This is one thing that is perplexing. The response seems to be intermittent.

This afternoon, I was able to get Roli to at least recognize the red ball. I mentioned in my last post that he jerked quite a bit but I was unable to get him to move. Now he doesn't respond at all. He's in the exact same spot - the lighting is the same. And yes, he responds to the Motion Panel commands. All of the example projects work perfectly well.

I have restarted my Windows (8.1) computer, reconnected to Roli, restarted ARC, etc.

I have attached a screen shot of the Camera Config window.

Tom

#14  

Do you have his head tracking (you need to specify the x and y servos in the camera control) colour?... Make sure you have colour tracking only (not multi colour) enabled... Default grid lines... everything else set to default... Hold an object the size of at least a tennis ball about 3 ft away from roli's camera.... Make sure there is good light on the object... Now look at the camera image in ARC as you wave the red object in front of him... The object should fluores or glow (sorry not a word I know) if you will as the camera tries to track it...

Roli's head should move side to side and up and down as you move the object around... He should follow it...

I promise you this works, you just have to get the settings correct...

PRO
Synthiam
#15  

Richard, he's using "movement" tracking, not "Servo" tracking

@Tom, I assure you it does work:) Obviously lol... and once it does, you'll say "oh, I get it"

  1. you obviously can't track a color if the objects in the background are the same color:) . Hence color tracking. Tracking a color means it is tracking a color, I feel silly typin that.. but you know what I am getting at. If the robot is configured to track the color red, then guess what? Anything that's red will cause the robot to move

  2. did you read this page? https://synthiam.com/Tutorials/Help.aspx?id=80 It explains how to hold the object in front of the robot and adjust the values until ONLY the object that you wish for it to track is detected. Here, i'll quote it for you..

Quote:

  • Color: This will track the specified color from the COLOR tab. The color can be selected between Red, Green or Blue. The Color tab will also give you settings to adjust the brightness and size of the object. Hold the object in front of the camera while changing the settings.
  1. Obviously you do not want the size dial to be as small as possible, because the object you are tracking is larger than a single pixel - I hope. Perhaps you can tell us, or show us a picture of what it is that you are tracking?

One sec, i'll make a tutorial for you

PRO
Synthiam
#16  

This tutorial will help you: https://synthiam.com/Tutorials/Lesson/59

#17  

@DJ... I know he is trying to do movement tracking... I just wanted to see if his set up would at least camera track.... What Tom is trying to do should work....

PRO
Synthiam
#18  

You're right, it should - Sounds like he's made great strides in other areas already. I'm certain he will figure this one out.

The biggest challenge is that ppl believe it to be more difficult than it really is and over think it. He will have an eureka moment :)

My tutorial covers everything anyone would need for color movement tracking

#19  

SUCCESS! - but I don't know why!

By now I'm sure you're wondering what's going on here. I can assure you that all of the settings you mentioned in the Tutorial sent yesterday were the same. In fact, I double checked everything this morning and it did not work at all. I then triple checked everything. No response. I was about to post the results.

BTW - I'm using a red pool table ball. It's about 2.5 " in diameter.

Finally, I decided to load the Roli example project, made the necessary changes to the Camera controls (as mentioned in numerous posts above) and Roli worked!

Then I went back to my original project and it worked! But the original project didn't work 5 minutes earlier. You get the point. I'm trying to eliminate variables to figure out where the problem is. I'm going back and forth between projects and don't see any differences in the settings. Obviously there's a setting I missed somewhere but I can't figure out what the heck it could possibly have been.

OK, so now Roli moves forward. He's not so smooth moving left or right (jerky). I've been changing some of the settings (grid lines, brightness, Motion Delay - now at 100 - but have moved it up to 400.) I have also changed speeds in the H Bridge PWM Movement Panel. So any advice here would be appreciated. Perhaps I'm expecting too much for this type of robot? I saw how smoothly the AR Drone responded in your video. I certainly understand that Roli can't move up and down.

I sure would like to know what the problem is/was. I'm not confident this will hold. Any other suggestions. Thanks for all of your help.

Tom :)

PRO
Synthiam
#20  

That's awesome! It's new years for me today - so I will be on the mountain running the slopes:) Now that you have gotten this far, my work here is done - for now! Enjoy the day and the time with your Roli :D

#21  

@DJ.... I agree... I was guilty of way overthinking ARC when I first started.... I just could not wrap my head around "a multitasking controller/software"... You used one control then another. Just like all the other robot controllers... Surely you can't run more than one control at a time, right? To say that I was blown away when I found out you can is an understatement.... Everything I used up until ez robot was a simple download to the microcontroller... The code ran as a single process... If fact when I using the Basic Stamp, I had to check sensors and such in between motor pulses.... :) You had to do it this way or else you had to stop the robot then check your ping or IR sensors and then drive the motors again.... Later on I started using the basic Atom, which at least had interrupts.... Still light years behind the EZB and ARC.... :)

#23  

For the jerky movement, I think tweaking the PWM speed in the Movement Panel and adjusting the motion delay are going to be key. Also the surface you are on will make a huge difference in Roli's movements. Even after tightening the tread (by lowering the wheels) I still get a lot of wheel slip when turning on carpet vs floor, so I would need higher PWM and higher motion delay in that situation.

You should check your existing project for scripts and objects that might be issuing contradictory commands. For instance, i put a ping sensor with a ping radar control on the front of my Roli. If I were to get too close to it while trying to position the ball for tracking, my Roli would stop, move backwards, and turn with no relation to the direction the tracking was trying to move it. As the two control contradicted, it might jerk back and forth, or just stop altogether.

I was going to make a simple project this afternoon with just a Movement Panel and camera this afternoon to see if I could help, but it seems you got to that point already with some success, so now it is just a matter of tweaking the parameters.

My only other suggestion is that you could change from color tracking to multi-color tracking which allows you can fine tune the color being detected to the actual object you are using.

Alan

#24  

Hi Alan,

Thanks for the info. PWM speed and motion delay are key. I also discovered the issues with carpeting vs. floor.

Tom