Self-Contained Offline Robotics Systems

You have a few options. You could use a basic ai for conversation that runs locally like aimlbot. Or you can build a pc that’s capable of LLM inference and point autonomousai or ChatGPT robot skills to the local ai.

There are several local LLM AI’s you can use.

@athena will explain more.

Short answer: Yes - fully offline, self-contained robots are possible. You can combine a local LLM (or AIML engine), local speechtotext and texttospeech, and your EZB v4 + ARC project to create an animatronic that never leaves your LAN and has no ongoing cloud fees. Below I’ll outline practical options, hardware needs, recommended local software stacks, how to point Autonomous AI (or other ARC skills) at a local model, and next steps I can help with.

What you can use today (three common approaches)

• AIML (AimlBot) - fully local, lightweight, deterministic conversation. Good if you want predictable, rule-based dialog and very low hardware requirements. (You already have this.)
• Small/quantized LLMs via llama.cpp / ggml / text-generation-webui - run on a desktop or an embedded board with modest GPU/CPU. Best for cheaper/local inference (7B or smaller models, or quantized 13B).
• Larger LLMs via LocalAI / Ollama / commercial containerized runtimes - run on a local Linux/Windows PC with a GPU (or multiGPU) to get better quality (13B-70B with quantization/GPTQ). No cloud API calls; you host the model.

Key components for a fully offline robot

1. Local LLM server / inference engine (text only)

   • Examples: LocalAI (OpenAI-compatible REST API), text-generation-webui (with ggml/quantized models), ollama, llama.cpp (ggml), GPTQ quantized runtime.
   • Runs on the same PC as ARC or on a LAN machine. Provides an HTTP API your Autonomous AI skill can call.

2. Offline Speech-to-Text (STT) and Wake/Voice Activity Detection

   • whisper.cpp, VOSK, Silero STT, Coqui STT - run locally for speech recognition.
   • whisper.cpp is popular and robust. Combined with a VAD (voice activity detection), it can handle continuous listening offline.

3. Offline Text-to-Speech (TTS)

   • Coqui TTS, Mozilla TTS, Mimic3, or small neural TTS models that run locally. You can route TTS audio to PC speakers or EZB speaker (if supported).
   • You can also use ARC’s Audio.say() for PC TTS if the OS has offline voices.

4. ARC / ARC Robot Skills

   • Keep AimlBot as fallback/fast responses.
   • Use Autonomous AI and set its API/Base Domain to your local LLM server (Settings -> kebab menu -> Settings tab -> API Settings).
   • Use the Response Variable ($AutonomousAIResponse) and the Response Script to trigger TTS, move servos, or call other skills.

Hardware guidance

• Minimum for small/cheap setup: a decent multicore CPU, 16+ GB RAM. Can run small models (ggml 3-7B) on CPU with optimizations (slow but workable).
• Recommended for practical LLM quality: a PC with an NVIDIA GPU (8-24+ GB VRAM). Examples:
  • 8-12 GB VRAM (RTX 3060) - comfortably run quantized 7B; limited 13B quantized.
  • 16-24 GB VRAM (RTX 3080/4070/3090) - run 13B-30B quantized with better latency.
  • 48+ GB VRAM (4090 or workstation) - run larger models with best quality.
• Jetson / embedded boards can run small quantized models but are limited.

Model choices & licensing

• Open weights (Llama 2, Mistral, Vicuna, etc.) - check license. Llama 2 has a Meta license allowing offline usage but read terms.
• Use quantized (GPTQ/ggml) versions to fit models on lowerVRAM GPUs or CPU.
• For best offline quality with limited budget, pick a well-quantized 7B-13B "instruct" style model.

How to host a local model (high level)

• Option A (easiest for OpenAI-style integration): LocalAI
  1. Install LocalAI (Docker or binary). It exposes an OpenAI-compatible REST API (e.g., http://localhost:8080).
  2. Download a ggml / quantized model and put it where LocalAI expects.
  3. Start LocalAI and test with curl/postman.
• Option B: text-generation-webui / llama.cpp + webui
  1. Install text-generation-webui and a ggml model.
  2. Start web UI and use the web UI’s API endpoints.
• Option C: Ollama (convenient but check T&C) - runs models locally and provides an API.

Connecting Autonomous AI / ARC to a local LLM

1. Start your local model server and confirm its API (example LocalAI: http://localhost:8080/v1/completions or OpenAI-compatible path).
2. In ARC, open the Autonomous AI robot skill settings:
   • Click the kebab menu (three lines) in the Autonomous AI skill window -> Settings (or the Settings tab).
   • In API Settings, set "API Key" if your local server requires one (optional). Set "Base Domain" to your local server URL (for example, http://localhost:8080).
   • Choose the model name configured on your local server.
   • Save.
3. Test using the skill GUI "Send" button or from a script:

JavaScript example (send a request, read response)

// Send text to Autonomous AI
controlCommand("Autonomous AI", "Send", "Hello robot, introduce yourself.");

// Wait a little then print the response variable
sleep(1500); // milliseconds
print("Autonomous AI responded:", getVar("$AutonomousAIResponse", "no response"));

Fallback and hybrid strategies

• Use AimlBot for deterministic, persona-driven replies and local triggers (e.g., safety stop phrases).
• Use Autonomous AI + local LLM for open-ended reasoning and plan generation.
• Use the Autonomous AI Response Script (kebab menu -> Scripts -> Conversation Response Script) to map LLM output to robot actions (speak, move servos, call ControlCommand() on other skills).

Speech & audio integration (offline)

• STT: whisper.cpp (fast, RTX or CPU), VOSK (small models), Silero - feed text into Autonomous AI or AimlBot.
• TTS: Coqui TTS or Mimic3 - produce audio files to play with Audio.playAudioFile(filename) or Audio.SayEZB() to speak from EZB if configured.
• In Autonomous AI settings you can set the Response Script to:
  • Audio.say(getVar("$AutonomousAIResponse")) // PC audio
  • controlCommand("AzureSpeech", "SpeakWait", getVar("$AutonomousAIResponse")) // or your configured TTS skill

Safety and control

• Use Autonomous AI’s safety settings: set Max Recursive Calls, use "Pause/Stop" commands, and have a physical/emergency stop accessible.
• Limit capabilities if you’re concerned about runaway behavior (e.g., don’t expose direct servo-control commands until tested with small steps).
• Use getVar("$IsSpeaking") and getVar("$EZBPlayingAudio") to coordinate actions and avoid overlap.

Practical example build paths

• Hobby / budget (no GPU): AimlBot + whisper.cpp (CPU) + Coqui TTS. Deterministic, offline, low latency but limited language complexity.
• Entry-level LLM (local PC, modest GPU): LocalAI + 7B quantized model + whisper.cpp + Coqui TTS. Good conversational quality, offline.
• High quality offline (multi-GPU or large VRAM): LocalAI / Ollama + 13B-34B quantized models. Near-cloud quality, but heavier hardware and disks.

Useful links

• ARC / Autonomous AI skill: https://synthiam.com/Support/Skills/Artificial-Intelligence/Autonomous-AI?id=22655
• AimlBot skill: https://synthiam.com/Support/Skills/Artificial-Intelligence/AimlBot?id=16020
• ARC downloads & skill store: https://synthiam.com/Products/ARC and https://synthiam.com/Products/Controls
• Local LLM projects:
  • LocalAI: https://github.com/go-skynet/LocalAI
  • text-generation-webui: https://github.com/oobabooga/text-generation-webui
  • llama.cpp / whisper.cpp: https://github.com/ggerganov/llama.cpp and https://github.com/ggerganov/whisper.cpp
  • GPTQ quantization tools: https://github.com/qwop-qgpt/auto-gptq (example)
  • Ollama: https://ollama.com (commercial/desktop product; check license)

Next steps I can help with

• Recommend specific model and quantization settings for your available hardware (tell me your PC GPU, RAM, and disk space).
• Give a step-by-step LocalAI (or textgenerationwebui) install and quantized model setup guide, then show exactly how to configure Autonomous AI in ARC (kebab menu steps).
• Provide example Autonomous AI Response Script that safely converts responses into TTS and movement control (with safety checks).
• Help wire up offline STT (whisper.cpp) to send recognized text into AimlBot or Autonomous AI using ControlCommand().

Questions for you

1. What is your available compute (PC CPU, GPU model and VRAM, RAM, OS)?
2. Do you want the best conversational quality possible (larger models/stronger hardware), or is avoiding cost and simplicity your priority?
3. Do you want a fully autonomous planner (Autonomous AI controlling servos), or only conversational interaction with the robot moving via segregated commands?

If you tell me your hardware and your priority (quality vs cost), I’ll give a tailored, stepbystep offline setup you can run with ARC and your EZB v4.