JD With New Self-Programming AI Robot Skill
Last night, I had a few friends over, and we discussed the fascinating topic of how artificial intelligence can autonomously program itself. We explored how AI can incorporate code into its responses, which can be executed within Synthiam ARC's internal scripting engines using JavaScript or Python. To illustrate this concept, I quickly recorded a video demonstrating an EZ-Robot JD being programmed in real-time by the AI to perform various actions. This capability allows the robot to become fully autonomous, enabling conversational interaction as if you were speaking to another person. The robot can program itself based on your spoken instructions.
I'm not sharing the specific details of how I achieved this just yet because we're in the process of adding more features to the robot skill. After this weekend's ARC update, it will be easier to implement this behavior with any robot. This means you can add this functionality to your InMoov, DIY robot, or any other robot project. All you need to do is define the robot configuration, and the AI will use the robot project to program it in real-time, allowing it to perform tasks as you talk to it.
Here's the video for you to check out:
@DJ They have deepseek R1 over on groq (not Elon grok) I had a quick play, honestly haven't looked at this in over a year and couldn't even get the OpenAI ChatGPT skill working in groq today (I am pretty sure it is still free, even created a new API key). My local AI server is in pieces. I tried LM Studio with Deepseek on my local PC with just a RTX3090 but had some issues as well. Maybe someone who has had more sleep than me today can try. Here is the DOC for groq, It used to work fine. https://console.groq.com/docs/overview
Edit: my bad it appears deepseek R1 doesn't support images hence my errors. Off to try some other models
Grok 1/2 is meh - but Grok 3 is what everyone is waiting for. We don't run any inference locally, even though we have several PCs with high-end Nvidia GPUs for VR . But it's less time-consuming to use the cloud. Our time to configure, maintain, and wait for local inference costs more than cloud usage haha.
BUT - I'm interested in your experiences locally because you're one of the few people I know who are interested in running locally. With all the GPUs you have, you can also get into VR. Better use of home GPUs IMO, haha
yep looks like an image processing issue the Autonomous AI sends a stack of images to openAI to be processed and a lot of the local models don't support images or the ones that do can only process 1 image at a time.
Example error Too many images provided. This model supports up to 1 images Initialized Using model: llama-3.2-11b-vision-preview Capturing image... User: wave Sending (28 msgs, 4 images) HTTP 400 (invalid_request_error: )
Too many images provided. This model supports up to 1 images
EDIT:
This looks interesting https://github.com/deepseek-ai/DeepSeek-VL2
Set your image history to 1
Interesting the API server doesn't filter based on the model definition. it would make sense to discard the data that it can't use simply.
your plugin only allows a minimum image history of 2 ?
oh - well then! i'll see what it'll take to move it to 1
I was advised not to, but for your usage, there is no other option. Apparently, with only 1 image, the comparison throughout the previous sessions results in a poor experience. But since you're in new territory with a local model, let's see what you get! I'll send an update. But it looks like the code is incomplete right now - the build is broken because it's being modified for the new ARC. I'll see what it takes if I can do it tonight while I watch this documentary on Tetris haha
Ok it’s after midnight here in Toronto and I am out of beer. I will rebuild my AI server and test with the R1-L2 model. BTW I watched the Tetris movie a couple of years ago, spoiler the Russians did it.