My robot would be sitting at a table, and it would look down at the objects on that table. It would be able identify the objects on the table: salt, pepper, a can of beer, a knife, a plate a spoon, and carrots on the plate. It would also know the coordinates of each object in 3D space of each object. Knowing that it would reach out and pick up the beer. I think Tensorflow can do some of this already. Microsoft Cognitive vision gives you the objects it sees, but not the locations.
Other robots from Synthiam community

Tonzatonka's Inmoov 3D Printed Robot
Progress on the lower and mid stomach! Just getting the neo pixel ring running, hopefully will be installing soon! Not...

Steve's Mini 6 Fabricated Robot
Mini 6 fabricated Robot This is my next EZ Robot, after helping my grandson Hunter build his first robot. I have always...

DJ's Jd Humanoid Controlled By Microsoft Kinect
Microsoft had sent me a Kinect a few years ago, and I embarrassingly finally got around to doing something with it. I...