United Kingdom
Requested

Posenet Integration Score 1291

Hi Synthiam folks

I've been starting to work with PoseNet which allows my computer to recognize various poses but also to detect a human shape in the room. Is it possible to have a plugin or camera integration with Posenet to allow my InMoov (using Ez-B and ARC) to be able to detect human poses? I'm not an advanced enough coder myself to try to tie it in as a plugin....but thought you might like this idea (if you aren't already aware of it).

PoseNet GitHub PoseNet Local Installation

Just a thought and forgive me if you've already got something like this that I've missed.

Thanks guys

Steve

Want to see this feature happen? Like it to increase the score.

ARC Pro

Upgrade to ARC Pro

Discover the limitless potential of robot programming with Synthiam ARC Pro – where innovation and creativity meet seamlessly.

#1  

Hey, did you check Teachable Machine..its very easy to use! Or you can use ml5js, which is also super easy!! :)

https://ml5js.org/

United Kingdom
#2  

Looks good...but can they integrate with Arc?

ml5js uses the same TensorFlow/PoseNet, I think.

PRO
USA
#4  

stevehaze:

Posenet returns 2D coordinates: User-inserted image to translate to real life joints you need 3D coordinates. You can get them with a 3D camera e.g. Kinect, or you need a good 3d model with a lot of computation behind scenes to infer the 3D coordinates.

PoseNet and 3d coordinates :  https://github.com/tensorflow/tfjs/issues/1811  User-inserted image User-inserted image

It's a lot of work, and as expected they are not free. PoseNet is a "light model" = inaccurate model, some machine learning models obey to license constrains, for example OpenPose is much more accurate but requires a commercial license (25k per year).

There are some Posenet implementations using game engines running on the browser to replicate the pose: User-inserted image and with some work, maybe can be replicated outside of the virtual world, but expect some serious limitations: accuracy is one, the pose will be another one.

Teachable Machine is a tool to simply the above concepts and to allow creative minds to integrate or prototype ideas using the existent free models (which require a lot of resources to train) and using reinforcement learning and the browser GPU you can get explore the concepts without spending money and resources training models .

You want to use Posenet to control your robot, although is not impossible but requires additional research & work the other way is to use PoseNet model to detect specific poses.

Mickey trained some poses e.g. Arms Up, Down, etc and using the Teachable Machine running in the browser communicates to ARC (Http control) the recognized position, and from there you can execute the desired AutoPosition movement.

Feel free to ask/debate your feature request.

United Kingdom
#5  

All good points...but me personally....I dont want to use Posenet to control my robot...but for the robot to recognize that there is a human in it's viewfield and if certain gestures are happening.

Without going into too much detail....Imagine a paintball robot that stops firing if you raise your hands.

PRO
USA
#6  

Quote:

I dont want to use Posenet to control my robot
You are right, I wrote the post on notepad, and i missed a chunk, where i explained 2 different goals for the Posenet live tracking and pose recognition.  It's gone :)

So coming back to your goal, I believe you can replicate Mickey steps.

#7   — Edited

Quote:

Is it possible to have a plugin or camera integration with Posenet to allow my InMoov (using Ez-B and ARC) to be able to detect human poses?
That was also my entry point to the subject...some kind of easy hobby approach to Pose estimation! Hopefully, as tech evolves at such an increasing speed, we might also soon be able to use live tracking to  drive our robots movement!

But that might take a bit I guess!! :)

#8  

@Mickey666Maus  Thanks for all this information.  I will be trying out some of your ideas soon.   I had to do quite a few modifications on Cog's wheel base. as I was having two issues one the ground clearance was to small and going over carpets would cause cog to hang up and second if Cog moved at max speed and did a abrupt stop would cause Cog to fall over forward.  so I have increased the bottom clearance and moved the front one swivel caster to two outrigger types further forward.  not as great of a look but more functional.  also I have designed a battery charging station and pickups using the same kind of design as the Synthiam mobile robot using two probes magnetically attracted to the charging station.  Hope everyone is doing well, and keeping safe.

#9  

Looking forward to see the improvements on your robot!!

Hopefully I can also show some progress on my new build soon...

Leep us posted!! :)