Haha you’ll be able to setup a server and charge us

I’m on Rogers internet and my wife makes me turn the computer off at night to save power. I can guarantee 0.0001% uptime.

I continued to play with LM Studio and ARC. I purchased a couple of second hand 3090 GPUs and this thing flies with very large models.  One 3090 GPU is probably fine for most models (about 600-700USD second hand or 800-900 CAD if you look around).    It is a pretty powerful and popular application that works out of the box with ARC and there are hundreds of different models you can choose from.  There are uncensored models so you don't have any bias, games and a range of models with various expertise like coding.  You can add your own data by retraining models or if you have some Documents you want to query you can use RAG and LangChain (experimental at the moment).

If you are sick of giving money to OpenAI or just want speed and privacy for the queries, give it a try. You need a NVIDIA GPU with a decent amount of VRAM Minimum 16GB but more the better (3090 or 4080-4090) or a very recent AMD GPU with ROCm 6 support (RX 7900XT  etc).   Also need a PC with AVX2 support

How fast are replies with your setup? And by couple do you mean two or three in the computer?

I purchased 2 cards but only one in computer at the moment as I am waiting for risers from china. The cards use 3 slots so are too close together so they overheat so I am moving to an open rack with the PCIe 16x risers. You could water cool if you want them in a case but I just stick my server in the cold cellar   My initial goal is 4 GPU's (96GB of VRAM) so I can run and train some very large models like mixtral and eventually go to 8 cards (192GB of VRAM) .   I get about 50+ tokens per second with one GPU. Response are pretty much instant especially when streaming on a model of about 10GB with 13Billion parameter model.  You have to load 100% of the model in GPU VRAM to get any decent speeds if you try to run from CPU it takes about 1-2 minutes to start response then about 4TP/s to stream. I have Ryzen 9 24 core + 32000Mhz DDR4 RAM, you apparently get about double that on DDR5 6000Mhz RAM and AM5 CPU's.  People are using MACs M3 as computer and get about 30TP/s so you can get 128GB computer or even 192GB Studio MAC if you have the cash (They share Video and GPU RAM and don't require a degree in computer science to setup). The  4090 cards are getting 80+ Tokens per second but cost a lot more.

I haven't built or trained my own custom models yet or used RAG that is my next todo.

Tried training a model on ARC data with RAG.  You get some interesting responses and some complete garbage. I am using LMStudio with the model "solar-10.7b-instruct-v1.0-uncensored.Q8_0.gguf" because it is not biased but I am sure there are better ones and another program called AnythingLLM that reads the support pages and tries to find an answer.  So here are some basic Q&A with model. Hey @athena what do you think can we use Retrieval Augmented Generation (RAG)  with a local general knowledge / programming language model to replace you? 

what is Synthiam ARC Synthiam ARC is a software platform that allows users to easily create, program and control robots using technologies from industry experts. It enables the creation of robot behavior by connecting pre-built modules in an intuitive graphical environment. With its ability to handle multiple concurrent tasks, it can be used for various applications ranging from simple robot navigation to advanced industrial automation. ARC also includes a library of CAD models that can be imported into the system and used as design templates for building physical robots.

what programming languages does it support

Synthiam ARC supports several high-level scripting languages, including JavaScript, Python, and Scratch 3.0. It also allows users to write their own custom code using C++ or Java if they have experience with those languages. With these options available, you can choose the language that best suits your skill level and project needs.

how do I add a skill to the desktop

To add a new skill to your Synthiam ARC project, follow these steps:

1. Open Synthiam ARC and click on the Project tab at the top of the screen. You will see all your existing projects listed here. If you don’t have any projects yet, click the New Project button to create one.

2. Select the project you want to add a skill to by clicking on it in the list. This will open up the project workspace, where you can edit and configure the robot project.

3. Click on the Add Skill button located at the bottom left corner of the screen. This will open a window that displays all the available skills within your Synthiam ARC installation.

4. Browse through the list to find the skill you want to add, and then click on it. The selected skill will appear in your project workspace.

5. To configure the new skill, click on its title bar, which is located at the top of the skill control, to open the configuration menu. In this menu, you can customize various settings to suit your needs for that particular skill.

6. Once the skill has been added and configured, you’ll need to save the changes by clicking the Save button on the upper right corner of the Synthiam ARC window. Now you can run your robot project with the newly added skill.

Haha that's a lot of nonsense to add a robot skill.

Are you adding synthiam data for training, embedding or finetuning on the model? Because you want to use Embedding. I doubt training is an option, is it? if so how long does that take

No I am not training a new model, I was going to try that when I get more GPU cards running. This method just takes the Synthiam support data and then breaks it up into chunks and then searches that for information relevant to the question then sends the question and the relevant chunks of support data to the LLM and says give me an answer based on this information I just gave you.    There is some Python code version that also does RAG and LangChain that I will try and shoehorn into a Script and then feed into the chatgpt skill.  https://github.com/ruddythor/mick.ai