Asked
Resolved by DJ Sures!
If we have a stationary robot and a stationary item sure we can pick it up, but if we have a robot that can move and an object that can be anywhere in any orientation how do we pick it up?
We have Lidar, we know where the robot is with Realsense, we have object recognition and yolo so we can find an object, we have servos like dynamixel and hacked HDD that know their position so what else do we need in order to pick something up?
Related Hardware EZ-B v4
You don’t need anything more than you described another than a very minor amount of programming.
Well, I guess you need the actual robot haha.
putting a camera on the gripper is a good idea because it can see where the object is in x/y coordinates to align the servo. Using the built in bracketing system of the camera device would be easiest way to do it. Constantly have a servo moving the arm down and the camera device bracketing will adjust the servos automatically to keep the object in the center.
maybe to pick up the object you’d need to know the distance from it. If it’s facing straight down? Because the camera would get dark. Maybe a distance sensor in the middle of the gripper.
I guess it still boils down to a robot design and use case. Have any ideas?
This isn't really my problem though what I would like to do is have a setup so Roli can pick the pen up no matter where it is. I can manually drive around and all works fine and script the actual pick when I am in the right position. Long term goal fetch bot / clean up bot (Get me a beer, pick up shoes and put them away etc). ie Robot Arm on the Roomba
ill keep playing.
Break your goal down into smaller goals. Split it down into a process - we go over that a bit in the Getting Started guide of this website (First link in the SUPPORT tab). Break down your goals and write them down. Organize them into smaller tasks. Then tackle them in order, not skipping.
I added rotation support for detected colored objects for you in the next build. See this video...
Looks like you need camera centered between 2 claws for this to work.
camera rotation tracking doesn’t work for me as well is it is for you. Reads zero most of the time so ignore 0’s. Think I will need to use a better camera and a faster computer.
I think my next step is to build a 5 or 6 DOF robot arm with a built in camera in the claw.
1) Use multicolor rather than just color. The multi color might detect a better rectangle out of the detected shape. If it doesn't work, i do have another idea.
2) Do you have any soft foam? If so, the gripper versions after your purchase include foam in side. This helps them pick up things and hold them in the gripper rather than the edge of the gripper. It also saves on servos burning out because of how they used to hold it.
Here's an image from the ez-robot gripper product page:
looking forward to update if you grab a highlighter and a standard ez-robot camera you will see what my robot sees .I did play with the various colour tracking options and settings as well as different colour highlight pens.
Thanks again for all your great help.
it was tough and that’s about the best I can do at the moment. The algorithm is a bit crazy because the detected colors don’t always make a perfect shape with right corners and straight lines.
oh and how’s the auto update button working out
I will try and see if I can teach rolli to play fetch.
Reason
I found tracking angle works better if the Camera is Directly above the object (90 degrees) because when it is at an angle you can't triangulate (only 1 camera). As a result I have to Get the camera rotation using 1 AutoPosition action and then move the robot and claws to pick it up with another autoposition.
Problem I have is I can not see pen when I go to pick it up as claw blocks camera (Had to move camera far away for GetCameraRotation to work). So I set var PickPostion to GetCameraRotation and then try to read the PickPosition variable in the pickup object script but does not pass variable between scripts.
I hope this makes sense?
To move a servo, use the Servo.setPosition() in javascript. Here's the manual: https://synthiam.com/Support/javascript-api/Servo/setPosition
You can obtain information about the location of the object using variables. The camera has a bunch of variables that describe the detected object. See the variable manager manual to see what variables are detected by the camera: https://synthiam.com/Support/Skills/Scripting/Variable-Watch?id=16056
From the Auto Position manual page: https://synthiam.com/Support/Skills/Movement-Panels/Auto-Positioner-Gait?id=16057
See this screenshot, change that to a 1
this problem is working out harder than I thought.
Look around room, Find pen, goto pen, move into position in front of pen, align claws to angle of pen, align robot in pickup pen position, pickup pen.
You see all these robots with claws and hands but they are all pretty useless without being able to actually pick something up. Anyway getting closer. Here is what the latest monstrosity looks like but now I need to add another servo as I need another DOF for certain pen angles.