Welcome to Synthiam!

Program robots using technologies created from industry experts. ARC is our free-to-use robot programming software that makes features like vision recognition, navigation and artificial intelligence easy.

Get Started
Canada
Asked
Resolved Resolved by DJ Sures!

How Do You Pick Something Up?

If we have a stationary robot and a stationary item sure we can pick it up, but if we have a robot that can move and an object that can be anywhere in any orientation how do we pick it up? 

We have Lidar, we know where the robot is with Realsense, we have object recognition and yolo so we can find an object, we have servos like dynamixel and hacked HDD that know their position so what else do we need in order to pick something up?


Related Hardware EZ-Robot EZ-B v4
AI Support Bot
Related Content
Synthiam
PRO
Synthiam
#1  
You don’t need dynamixel servos. Because ARC knows the position of everything it controls. It would be different if you were to manually move the servo with your hand, then you’d need a bidirectional communication like dynamixel

You don’t need anything more than you described another than a very minor amount of programming. 

Well, I guess you need the actual robot haha. 

putting a camera on the gripper is a good idea because it can see where the object is in x/y coordinates to align the servo. Using the built in bracketing system of the camera device would be easiest way to do it. Constantly have a servo moving the arm down and the camera device bracketing will adjust the servos automatically to keep the object in the center. 

maybe to pick up the object you’d need to know the distance from it. If it’s facing straight down? Because the camera would get dark. Maybe a distance sensor in the middle of the gripper. 

I guess it still boils down to a robot design and use case. Have any ideas?
PRO
Canada
#2  
Here is a pick. I stuck a joystick on so I can pick it up manually but I want the robot to pick it up no matter where it is or the orientation.   User-inserted image
PRO
Synthiam
#3  
No matter what the orientation will require a rotation servo somewhere like here.. to rotate the gripper 

User-inserted image
PRO
Canada
#4   — Edited
my orientation servo was the first one in the stack attached to Roli so I don't put extra weight on lift servo but roli needs to orient himself versus just orientation servo next to claw.  Ill redo with some heavy duty servos and put one next to claw.

This isn't really my problem though what I would like to do is have a setup so Roli can pick the pen up no matter where it is.  I can manually drive around and all works fine and script the actual pick when I am in the right  position. Long term goal fetch bot / clean up bot (Get me a beer, pick up shoes and put them away etc).  ie Robot Arm on the Roomba

ill keep playing.
PRO
Synthiam
#5  
Well, first you'll have to let the robot know where it is. Chances are you'll need to tell the robot. It doesn't have a good view at it's low angle to see the room. Unless you want to use some model of merely randomly driving around like a roomba bouncing off walls until it sees a yellow object.

Break your goal down into smaller goals. Split it down into a process - we go over that a bit in the Getting Started guide of this website (First link in the SUPPORT tab). Break down your goals and write them down. Organize them into smaller tasks. Then tackle them in order, not skipping. 

I added rotation support for detected colored objects for you in the next build. See this video...
PRO
Canada
#6  
Thanks DJ this should really help a lot. Assembling a new robot ARM to take advantage of this.  Appreciate the help and update to platform with orientation support. This is great.
PRO
Canada
#7  
Love the ez bits as they make working out a design really ez. 

Looks like you need camera centered between 2 claws for this to work.  

camera rotation tracking doesn’t work for me as well is it is for you.  Reads zero most of the time so ignore 0’s. Think I will need to use a better camera and a faster computer.

User-inserted image

I think my next step is to build a 5 or 6 DOF robot arm with a built in camera in the claw.
PRO
Synthiam
#8  
Hey @Nink, here's a few tips

1) Use multicolor rather than just color. The multi color might detect a better rectangle out of the detected shape. If it doesn't work, i do have another idea.

2) Do you have any soft foam? If so, the gripper versions after your purchase include foam in side. This helps them pick up things and hold them in the gripper rather than the edge of the gripper. It also saves on servos burning out because of how they used to hold it.

Here's an image from the ez-robot gripper product page:
User-inserted image
PRO
Synthiam
#9  
Okay - i have a build for tonight that will better analyze the detected object to calculate rotation. You should see better numbers now with the new algorithm. Stay tuned for 2020.12.13.00
PRO
Canada
#10   — Edited
I actually ripped the foam off the grippers as the pen just rolled into the gripper better without it but i put it aside and will glue back in later. 

looking forward to update if you grab a highlighter and a standard ez-robot camera you will see what my robot sees .I did play with the various colour tracking options and settings as well as different colour highlight pens.  

Thanks again for all your great help.
PRO
Synthiam
#11   — Edited
Hey there. Not sure if you saw but there’s a new update yesterday with improved angle calculation. https://synthiam.com/Products/ARC/Releases/ARC-2020-12-13-00-Early-Access-20125

it was tough and that’s about the best I can do at the moment. The algorithm is a bit crazy because the detected colors don’t always make a perfect shape with right corners and straight lines.

oh and how’s the auto update button working out
PRO
Canada
#12   — Edited
Woohoo it works.  You are amazing thank you. Now my grippers line up to 90 degrees.  THANKS.

I will try and see if I can teach rolli to play fetch.
PRO
Synthiam
#13  
That's great to hear:D . I spent a lot of time trying to figure that algorithm out ha
PRO
Canada
#14  
Question How do you pass a variable between action scripts in AutoPosition or not change a servo position in an autopostion 

Reason
I found tracking angle works better if the Camera is Directly above the object  (90 degrees) because when it is at an angle you can't triangulate (only 1 camera).  As a result I have to Get the camera rotation using 1 AutoPosition action and then move the robot and claws to pick it up with another autoposition. 

Problem I have is I can not see pen when I go to pick it up as claw blocks camera (Had to move camera far away for GetCameraRotation to work).  So I set var PickPostion to GetCameraRotation and then try to read the PickPosition variable in the pickup object script but does not pass variable between scripts. 

I hope this makes sense?
PRO
Synthiam
#15  
An Auto Position is just that, auto position. It moves into pre-defined positions. If you want to move a servo manually within a frame, you have to set that servo to a -1 position. 

To move a servo, use the Servo.setPosition() in javascript. Here's the manual: https://synthiam.com/Support/javascript-api/Servo/setPosition

You can obtain information about the location of the object using variables. The camera has a bunch of variables that describe the detected object. See the variable manager manual to see what variables are detected by the camera: https://synthiam.com/Support/Skills/Scripting/Variable-Watch?id=16056

From the Auto Position manual page: https://synthiam.com/Support/Skills/Movement-Panels/Auto-Positioner-Gait?id=16057

User-inserted image
PRO
Canada
#16  
Ok thanks so I just need to set to -1 perfect that should work.
PRO
Synthiam
#17  
Also note, that you'll probably want to limit the number of detected objects to 1 in the camera device. Because you're only focusing on one object at a time.

See this screenshot, change that to a 1

User-inserted image
PRO
Canada
#18  
Thanks that is helpful.  

this problem is working out harder than I thought.
Look around room, Find pen, goto pen, move into position in front of pen, align claws to angle of pen, align robot in pickup pen position, pickup pen.

  You see all these robots with claws and hands but they are all pretty useless without being able to actually pick something up.  Anyway getting closer.  Here is what the latest monstrosity looks like but now I need to add another servo as I need another DOF for certain pen angles.  

User-inserted image
PRO
Synthiam
#19  
Haha that is so awesome!!! Can’t wait to see it operational.  You’ll hit a Moment of eureka and it’ll all click