Thinking about how to tie the strings together, and also in terms of Unity integration... Why not use a super basic REST server, so we can send all sorts of data over to ARC?
I created a basic example, that is receiving an array of servo positions, that I am sending out from Unity. These values can be used in ARC to loop thru and create live servo position updates!
Any thoughts on how to improve this are very welcome, all types of data can be send to ARC this way, you could access a database and store different types of variables...so I guess its being a very versatile way to tie things together?
This is my first attempt to do things this way, so forgive me if the setup is super basic!!
Check if python3 is installed on your system python3 -V
if not download python https://www.python.org/downloads/
Make sure you are having Flask installed using the packet manager of choice... pip install Flask or py -m pip install Flask
Now create a file called EZ_ARC_REST.py
and copy the code into the file
from flask import Flask, json ,request
positions = [90, 100, 120, 110, 90, 90, 70, 90, 70, 90]
app = Flask(__name__)
@app.route('/set_positions/', methods=['GET','POST'])
def set_positions(position):
global positions
positions = position.strip('][').split(', ')
return json.dumps(positions)
@app.route('/get_positions', methods=['GET'])
def get_positions():
return json.dumps(positions)
@app.route('/')
def index():
return 'ARC REST Index Page'
@app.route('/hello')
def hello():
return 'Hello, greetings from different endpoint'
#adding variables
@app.route('/user/')
def show_user(username):
#returns the username
return 'Username: %s' % username
@app.route('/post/')
def show_post(post_id):
#returns the post, the post_id should be an int
return str(post_id)
if __name__ == '__main__':
app.run()
Now you can save and run the file...
There are some extra endpoints so you can get the idea, what can be done with a setup like this!
Test your server by putting some endpoints to your browsers address bar... http://127.0.0.1:5000/ http://127.0.0.1:5000/hello http://127.0.0.1:5000/set_positions/[70, 110, 120, 100, 90, 80, 65, 90, 70, 60]
The last line sends a servo array to import into ARC, you can retrieve it like this
from System.Net import HttpWebRequest
uri = "http://127.0.0.1:5000/get_positions"
webRequest = HttpWebRequest.Create(uri)
response = webRequest.GetResponse()
from System.IO import StreamReader
import clr
clr.AddReference('System.Web.Extensions')
from System.Web.Script.Serialization import JavaScriptSerializer
streamReader = StreamReader(response.GetResponseStream())
jsonData = streamReader.ReadToEnd()
js = JavaScriptSerializer()
dataDict = js.DeserializeObject(jsonData)
print(dataDict)
servo =(dataDict)
for position in servo:
print(position)
Once the array is available, you can use it to drive servo positions, or you can just use this method to get all kinds to data over to ARC...
It's pretty neat and very ingenious! I don't want to be a bummer, but that's going to be expensive on resources - but if you have the processing power then it could work. What don't you like about the current servo camera server? It's very fast and elegant. Super easy to use. I can make it easier if you have trouble with it. https://synthiam.com/Support/Skills/Servo/Servo-and-Camera-Server?id=17663
Let me know. I'd rather see you building and programming your awesome robots than fiddling with this kind of stuff hehe
@DJSures thanks a lot for the kind words on my ctrl+v code!! I am really not familiar with anything about that topic, so I was just throwing some ideas out to the open, mainly to see if anyone can contribute and make this a working solution! The Servo-and-Camera-Server is great, the only thing is that its functionality is limited to Camera and Single servo Position streaming, if I remember it correctly? I had some trouble getting multiple servos working for live movement, but that could also have been a pebkam...
The main reason why I proposed this idea was, that it is very versatile on what data type is being transferred. The server basically takes everything, and is able to spit it out for further processing!
But maybe the Servo-and-Camera-Server could be modified? I would really like to see this as a starting point, this is not at all meant as a working solution more a POC type of thing!
All of the servos are streaming with that. Maybe I should make another video on how to use it?
It allows all 24 of the EZ-Robot EZ-B v4 servos to be controlled in unity.. it also allows Unity to receive the camera video from ARC. I'll add it to my live hack list for you. Is there anything else you'd like this skill to do?
*Edit: i added the servo Camera Server to the live hack list
Cool, thanks a lot...that will be helpfull!!
My robot ZOE is running on LX16A servos, this is another reason why I was choosing this route for my current setup, maybe that was my issue back then? The current setup works great on BLYNK local server, but since it is 3rd party software, I thought Flask is being a good alternative!
All works great within ARC, but any info on making ARC integration more flawless and help bringing more ARC Users into building some fun stuff in VR would be awesome!!
If you are doing a live hack, maybe you could integrate some idea of how to store those received servo positions to ARCs Auto Positioner... The way I am doing it at the moment, is to store those arrays into a big chunk of JSON and read them out from ARC, using the same method as above! I was planning to do a video about it so Will can see how it works, he was also interested in live movement and playback!
But I will wait for your live hack, because I think using the native functionality of the Auto Positioner in ARC might work better than my bloated system!!
There are always many ways to do something, but I can definitely advocate for what Mickey has posted here. It is simple and works and is applicable to a wide range of use cases. I am posting a few examples to reinforce what Mickey is saying and get some ideas flowing about additional use cases. There are many more but I had to be selective.
A Hello World Example with Arguments (2+2 = 4): to illustrate using input arguments.
Example to Leverage a 3rd Party NLP Library There are many libraries in the python world. This example shows how to use one of them to get natural language features about some text using the popular Spacy library. This will provide parts of speech, dependences, lemmas, and many other features about every word in the input.
It is up to you to then figure out what you want to do with all this information
Machine Learning Example #1: Object Recognition This example takes in a filename, loads the image from disk using OpenCV2, and sends it off to a machine learning model (yolo3) in this case. The SendOutput function simply spits out any object as JSON with a time duration attached as well.
There is more code for some of the functions, this is just meant to demonstrate one way to solve the use case at a high level.
Machine Learning Example #2: Natural Language Question Answering This one answers a natural language question based on a given context. The context could be a corpus of text like an article or wiki page, a conversation history, whatever.
There are many other use cases in areas like SLAM, Mapping, Navigation, Image Processing, Machine Learning, and many more in NLP. Most of everything that has ever been done in robotics exists in some form in python, and to a much lesser extent in other languages. Because I use a lot of windows and C#, this means I sometimes need to call python to leverage this world, passing inputs and processing outputs. Flask is one of many ways to do that.
I will leave one final code example, showing the SendOutput function in the examples above.
Disclaimer: If anyone wants to quarrel about the quality or lack thereof in my python examples, I am sorry, I am new to the space, and have my own style and considerations. I am just trying to encourage thought and innovation and hope this helps others to pursue their own goals.
To me this was a very good insight on how your system works!! I was actually using a third party REST server called BLYNK, but now I guess I will be switching...because request.args.get() is awesome, I would have never stumbled upon it by chance, I guess I will have to do a bit of research on FLASK and REST!!
I also had to do some Google searches myself, so anyone wanting to try the query for the first example in the browser...
http://127.0.0.1:5000/sum?num1=1&num2=10
Posting JSON is a bit finicky, I am using Insomnia for testing querys, since the Postman plugin for Chrome is no longer supported...
You need to set Insomnia to POST and the body to JSON, with the following content!
JSON is always messy to deal with, and slow...but I am also by no ways a coder, so it comes in handy at times!
Hey guys, after using ARC to drive some servos based on a trained dataset, I was tempted to drive my ZOE robot using ARC by sending some live servo updates to the Super EZ ARC REST Server... I would really recommend to use EZ-Robot HDD Servos for your project, since they are very easy to use, but in this project we will be using Serial Bus Servos! I will make a detailed example soon, but for now you can just send a string to the Super EZ ARC REST Server like this...
http://127.0.0.1:5000/set_positions/[500, 500, 500, 500, 500, 500, 500, 500, 500, 500, 500, 500, 500, 500]
And see your robot move...
All you need in ARC is this script, and change your COM port to the port your BusLinker board is connected to!!
What serial bus servos are you using? Because you can add the robot skill for those servos and simply set Servo.setPosition() in your code rather than writing to the servos via the COM directly with their protocol. That way your servos work across all ARC robot skills.
Hey DJ, this is just some example that I did, because ZOE is the only robot I am having assembled for testing! But of course it is also working with the ARC Servo.setPosition(), as I showed in the facemask tutorial for ARC!!
Its just one line of code to make it work with EZ-Robot HDD servos!
I understand the change for hdd / pwm hobby servos. But my suggestion is to use the Dx or Vx ports so it can be compatible with all existing skills that use servo controls.
If your robot uses LewanSoul servos. Then all you have to do is add the LewanSoul robot skill and configure the servo ID's to respective Vx ports. Then, your PHP code merely needs to Servo.setPosition(V0, 50); or whatever...
The advantage of using the existing Robot Skills as servo drivers allows your code to seamlessly interact with all other robot skills. When a servo position is modified in ARC, other robot skills can interact with that position change. By talking to the servo directly through your PHP code, you lose that ability for people to merge it with the ARC framework. It loses the ability to be able to work with other robot skills.
Thats a good point...I will change an test the code tonight! Will be out of town during the day!!
I also realized that it is quiet difficult to install flask system wide on a fresh Windows 10, you need all the Python, Pythonpath, pip, sudo, blablabla stuff... So I was also thinking of another REST Server solution...
You pretty much created a robot skill - so why not just do that?
You're already using the .net System.web.extensions and serializer. So all your program is quite basic. Again, if you're calling this from unity, not sure why you're just not using the camera servo server
I'll have to make a demo for you
Another good point there...I will have a look tonight!!
Ask me questions if You have any about the process