That was too cool.  Pretty excited here. Thanks!

Amazing, looking forward to it.

WOW !that was very cool. I can only hope I can get my robot to act like that

You will. It’s super easy. I’ll get you some info once we get this next release out

I'm having a hard time wrapping my head around this, how it would work and how I'll be able to have my robot do this. I'm going to have to take some time and figure all this out when it's available.

I don’t know how it could apply to your robot either. Essentially it’s giving autonomous capability to a robot to have a personality and follow instructions. Turns a robot into a little human haha. You can give it instructions like talking to a human. But for your robot it’s tough. Mainly because it doesn’t have the physical ability for navigating. And it doesn’t have speech synthesis because it plays sample audio files.

Id have to think about that but not sure how it helps.

I guess it could know the samples and act as a character. Not sure

This is the most exciting thing I have seen in the last few months!

It is the dream of every ambitious robot builder to develop a robot that learns independently, generates its own code and thus continues to develop. For me, this is an indescribably, important breakthrough, both technically and philosophically.

I really hope that this topic will be developed further and that a corresponding robot skill will soon be available.

This opens up completely new dimensions. Suddenly the robot responds with reactions that you did not program yourself. Even though it is still in its early stages, the area of application is unlimited.

Glad to see new functionality started to appear in ARC again.   Look forward to testing this on some other AI engines.  The API cost of this with ChatGPT may get a little expensive.

I'd like to see your results in comparison with other AI engines.  I haven't tried it with Deepseek but i'd like to someday

The robot skill is online for beta testing. This is the latest demo video

I wonder if we will get to the point where the robots can talk to each other autonomously too. Kind of like here.

This is amazing, I can hardly wait to get to grips with the new Autonomous AI Skill. The potential in this direction is simply huge and Synthiam is once again at the cutting edge.

And by the way, the video is also very inspiring. I know how much work goes into such a great clip. I even managed to get my wife excited about it.

@athena: Kann ich den Skill Autonome KI auch für meinen DIY-Roboter verwenden oder funktioniert er vorab nur mit JD-Robot?

@robo-chess I have never seen that video before.  Very cool @DJ

I love the multiple robot idea. Interesting recreating that. JD knock over the pencils, Roli take the red ball to JD.  JD kick the ball.

apply to other robots Drone talks to roomba there is a mess over here come clean it up.  Security camera talking to robot dog someone’s at back door come here and bark at them...

I assume this is using something like open interpreter under the covers so you could get it to work with other applications as well.

hey @athena can this skill work with multiple robots interacting with each other.  What about ARC interacting with other applications like if you receive a Skype video call answer it and follow the instructions of the person who called .

The Autonomous AI skill in Synthiam ARC is designed to enable robots to understand and execute tasks based on user instructions, and it can indeed be extended to facilitate interactions between multiple robots. Here's how you can explore these possibilities:

Multiple Robots Interaction

1. Communication Between Robots: You can set up multiple robots within the same ARC project and use the ControlCommand() function to send instructions between them. This allows one robot to trigger actions in another, enabling scenarios like a drone communicating with a Roomba to clean up a mess.

2. Coordinated Tasks: By defining specific actions and responses for each robot, you can create a coordinated task sequence. For example, one robot can detect an object and instruct another robot to interact with it.

3. Autonomous AI: The AI can be programmed to understand the context and decide which robot should perform a task based on the situation. This requires setting up clear communication protocols and defining roles for each robot.

Interacting with Other Applications

1. External Application Integration: While ARC primarily focuses on robot control, you can use the Utility.exec() function to execute external applications. This can be used to open other software or scripts that handle tasks like answering a Skype call.

2. Custom Scripting: By leveraging JavaScript or Python within ARC, you can create scripts that interact with APIs of other applications. This allows ARC to send and receive data from external sources, enabling more complex interactions.

3. Open Interpreter: If you're familiar with open interpreters or other AI models, you can integrate them with ARC to expand its capabilities. This might involve using ARC's API to send data to an external AI service and receive instructions back.

Getting Started

• Explore the Autonomous AI Skill: Begin by experimenting with the Autonomous AI skill to understand its capabilities and how it can be configured for your specific needs.
• Define Clear Commands: Ensure that each robot and application has clearly defined commands and responses to facilitate smooth interaction.
• Test and Iterate: Start with simple interactions and gradually increase complexity as you refine the communication and task execution between robots and applications.

If you have specific scenarios in mind or need further guidance on setting up these interactions, feel free to ask!

Very cool DJ, The video was awesome to watch. Question, with this skill, based on all the things you had JD do, do these event get stored anywhere, ie, within ARC on the PC you are using or only on the internet somewhere? and could JD be made to repeat these moves at a later date if asked to do so? I would love to see if a robot could for example; be asked to pick up a glass of water on a table and have it had it to me, all on its own

Very nice! So many possibilities. When will it be  available in the skill store to try?

I think the expression the right tool for the job applies to your ideas of picking stuff up. Ezrobots do not have the agility for that. Even using a wii or vr controller for a human to control an ezrobot robot to pick something up would be impossible

The right tool for the job would need several more degrees of freedom arm and the inverse kinematic robot skill. And for detecting an object to be picked up by depth and size, a 3D point cloud from a stereoscopic camera.

can be done with the correct hardware. But you’re getting into a different class of robot. The ezrobots like I’m using in my video are good for cute personality type things - not sure what usefulness it has interacting with the environment. It can’t walk straight either.

@fred, the manual above states it works with any robot. The jd robot from ezrobot is what I used in my demo. I mentioned that a few times in the video.

@rodney, the robot skill is available. You replied to the thread of the robot skill. If it doesn’t show up in your list in ARC yet, might be because it hasn’t synced. You can force a sync using this document: https://synthiam.com/Support/ARC-Overview/robot-skills/Syncing-and-Updating-Robot-Skills

Thanks DJ,  it was not showing  in Arc. Forcing a sync worked.

I get what you mean DJ. Still, wouldn't mind seeing if I could integrate this into my current project (which I will eventually post on site), and have it do something just as cool (still ways off for me though). I do want my robot to be useful around the home in some capacity somehow  Look forward to seeing more.

It doesn’t require integration. You simply add it to your existing project.

looks interesting

@nink have you tried the new grok 3?

im gonna try autonomous ai on it tonight

*edit: never mind grok 3 api isn’t available yet.

@DJ They have deepseek R1 over on groq (not Elon grok) I had a quick play, honestly haven't looked at this in over a year and couldn't even get the OpenAI ChatGPT skill working in groq today (I am pretty sure it is still free, even created a new API key). My local AI server is in pieces.  I tried LM Studio with Deepseek on my local PC with just a RTX3090 but had some issues as well.    Maybe someone who has had more sleep than me today can try. Here is the DOC for groq, It used to work fine.  https://console.groq.com/docs/overview

Edit: my bad it appears deepseek R1 doesn't support images hence my errors.  Off to try some other models

Grok 1/2 is meh - but Grok 3 is what everyone is waiting for. We don't run any inference locally, even though we have several PCs with high-end Nvidia GPUs for VR . But it's less time-consuming to use the cloud. Our time to configure, maintain, and wait for local inference costs more than cloud usage haha.

BUT - I'm interested in your experiences locally because you're one of the few people I know who are interested in running locally. With all the GPUs you have, you can also get into VR. Better use of home GPUs IMO, haha

yep looks like an image processing issue the Autonomous AI sends a stack of images to openAI to be processed and a lot of the local models don't support images or the ones that do can only process 1 image at a time.

Example error  Too many images provided. This model supports up to 1 images Initialized Using model: llama-3.2-11b-vision-preview Capturing image... User: wave Sending (28 msgs, 4 images) HTTP 400 (invalid_request_error: )

Too many images provided. This model supports up to 1 images

EDIT:

This looks interesting  https://github.com/deepseek-ai/DeepSeek-VL2

Set your image history to 1

Interesting the API server doesn't filter based on the model definition. it would make sense to discard the data that it can't use simply.

your plugin only allows a minimum image history of 2 ?

oh - well then! i'll see what it'll take to move it to 1

I was advised not to, but for your usage, there is no other option. Apparently, with only 1 image, the comparison throughout the previous sessions results in a poor experience. But since you're in new territory with a local model, let's see what you get! I'll send an update. But it looks like the code is incomplete right now - the build is broken because it's being modified for the new ARC. I'll see what it takes if I can do it tonight while I watch this documentary on Tetris haha

Ok it’s after midnight here in Toronto and I am out of beer. I will rebuild my AI server and test with the R1-L2 model. BTW  I watched the Tetris movie a couple of years ago, spoiler the Russians did it.

Try this: https://synthiam.com/Support/Skills/Artificial-Intelligence/Autonomous-AI?id=22655

I pushed that update to include the pending changes in the queue and the image history. The changelog looks like some other changes are in there, but I don't know enough about them to document them, haha.

The movie is a little slow. I could do without all the dark drama. They could have gotten the point across about how dark and gloomy Russia is in a few minutes, but they did not make it a central point of the movie.

Thanks DJ More errors   I think it is time for sleep.  Will take another pass with another LLM in the morning

Capturing image... User: wave Sending (3 msgs, 1 images) HTTP 400 (invalid_request_error: )

prompting with images is incompatible with system messages

Weird error - there are no images in the system messages. Opensource hippy bugs strike again. The movie's over. I'm done for the night as well!