stevehaze
United Kingdom
Requested
Posenet Integration Score 1452
Hi Synthiam folks
I've been starting to work with PoseNet which allows my computer to recognize various poses but also to detect a human shape in the room. Is it possible to have a plugin or camera integration with Posenet to allow my InMoov (using Ez-B and ARC) to be able to detect human poses? I'm not an advanced enough coder myself to try to tie it in as a plugin....but thought you might like this idea (if you aren't already aware of it).
PoseNet GitHub PoseNet Local Installation
Just a thought and forgive me if you've already got something like this that I've missed.
Thanks guys
Steve
Want to see this feature happen? Like it to increase the score.
Hey, did you check Teachable Machine..its very easy to use! Or you can use ml5js, which is also super easy!!
https://ml5js.org/
Looks good...but can they integrate with Arc?
ml5js uses the same TensorFlow/PoseNet, I think.
Yes you can here is an example...
stevehaze:
Posenet returns 2D coordinates:
to translate to real life joints you need 3D coordinates. You can get them with a 3D camera e.g. Kinect, or you need a good 3d model with a lot of computation behind scenes to infer the 3D coordinates.PoseNet and 3d coordinates : https://github.com/tensorflow/tfjs/issues/1811
It's a lot of work, and as expected they are not free. PoseNet is a "light model" = inaccurate model, some machine learning models obey to license constrains, for example OpenPose is much more accurate but requires a commercial license (25k per year).
There are some Posenet implementations using game engines running on the browser to replicate the pose:
and with some work, maybe can be replicated outside of the virtual world, but expect some serious limitations: accuracy is one, the pose will be another one.Teachable Machine is a tool to simply the above concepts and to allow creative minds to integrate or prototype ideas using the existent free models (which require a lot of resources to train) and using reinforcement learning and the browser GPU you can get explore the concepts without spending money and resources training models .
You want to use Posenet to control your robot, although is not impossible but requires additional research & work the other way is to use PoseNet model to detect specific poses.
Mickey trained some poses e.g. Arms Up, Down, etc and using the Teachable Machine running in the browser communicates to ARC (Http control) the recognized position, and from there you can execute the desired AutoPosition movement.
Feel free to ask/debate your feature request.
All good points...but me personally....I dont want to use Posenet to control my robot...but for the robot to recognize that there is a human in it's viewfield and if certain gestures are happening.
Without going into too much detail....Imagine a paintball robot that stops firing if you raise your hands.
So coming back to your goal, I believe you can replicate Mickey steps.
But that might take a bit I guess!!
@Mickey666Maus Thanks for all this information. I will be trying out some of your ideas soon. I had to do quite a few modifications on Cog's wheel base. as I was having two issues one the ground clearance was to small and going over carpets would cause cog to hang up and second if Cog moved at max speed and did a abrupt stop would cause Cog to fall over forward. so I have increased the bottom clearance and moved the front one swivel caster to two outrigger types further forward. not as great of a look but more functional. also I have designed a battery charging station and pickups using the same kind of design as the Synthiam mobile robot using two probes magnetically attracted to the charging station. Hope everyone is doing well, and keeping safe.
Looking forward to see the improvements on your robot!!
Hopefully I can also show some progress on my new build soon...
Leep us posted!!