Below is one way you can bridge your iPhone’s voicetotext results with ARC robot skills using PushOver to help integrate and notify ARC of received commands. (Keep in mind that the PushOver skill itself is designed to send notifications from ARC to your mobile device-it isn’t an input control per se. However, you can set up a workflow where your mobile device sends text commands via push (or another method) into ARC’s global variable storage, and then use ARC scripting to parse that text into actions.)

Below is an example approach:

1. Use your iPhone’s voicetotext capability (or a dedicated app) to send a text message (or HTTP call) that contains the command you’d like executed. For example, the text might be:
Move the Big Arm to position 4000
Turn on Digital Port 10
Move fingers to position 1000

2. Have that text arrive in ARC-for example, by using a custom robot skill or other method that saves the incoming text into a global variable (say, $TextCommand).

3. Create a JavaScript (or Python) script in ARC that reads this global variable and parses the command. In the script you can use simple conditionals to determine which robot skills to call. For example:

// Example ARC JavaScript to parse a text command from $TextCommand

// Retrieve the command text that might have been sent via voice-to-text
var commandText = getVar("$TextCommand", "").toLowerCase();

if (commandText.length == 0) {
    print("No command received.");
} else {
    print("Received command: " + commandText);
    
    // Command: Move the Big Arm to position 4000
    if (commandText.indexOf("move the big arm to position") !== -1) {
        // Extract the numeric value (e.g., 4000)
        var pos = parseInt(commandText.replace(/[^0-9]/g, ""));
        // Assume the Big Arm is controlled by a servo (e.g., servo port d5)
        Servo.setPosition(d5, pos);
        print("Big Arm moving to position " + pos);
        
        // Optionally, send a push notification giving feedback
        controlCommand("PushOver", "Push", "Big Arm moving to " + pos);
    }
    
    // Command: Turn on Digital Port 10
    else if (commandText.indexOf("turn on digital port") !== -1) {
        // Extract the digital port number from the command
        var portNum = parseInt(commandText.replace(/[^0-9]/g, ""));
        // Construct port constant name (e.g., d10)
        var digitalPort = "d" + portNum;
        Digital.set(digitalPort, 1); // 1 for high/on
        print("Digital Port " + portNum + " turned ON");
        controlCommand("PushOver", "Push", "Digital Port " + portNum + " is now ON");
    }
    
    // Command: Move Fingers to position 1000
    else if (commandText.indexOf("move fingers to position") !== -1) {
        var posFingers = parseInt(commandText.replace(/[^0-9]/g, ""));
        // Assume the fingers are controlled by a different servo, maybe on port d7
        Servo.setPosition(d7, posFingers);
        print("Fingers moving to position " + posFingers);
        controlCommand("PushOver", "Push", "Fingers moving to " + posFingers);
    }
    
    else {
        print("Command not recognized.");
        controlCommand("PushOver", "Push", "Received unrecognized command: " + getVar("$TextCommand"));
    }
    
    // Clear the command after processing
    setVar("$TextCommand", "");
}

4. You could trigger the above script on a timer (or with a Check Command button) so that when a new text comes in, the script parses it and calls the appropriate ARC functions (such as Servo.setPosition or Digital.set).

5. The PushOver skill is used here to send back a notification to your device so you’re aware of what action is taking place. For example, after commanding the Big Arm, the script issues:
controlCommand("PushOver", "Push", "Big Arm moving to 4000");
This gives visual feedback on your mobile device.

Using this method you’re effectively integrating your voice-to-text input (via your iPhone) to power ARC’s various skills. You can extend the parser to support additional commands or more complex natural language processing.

Keep in mind:
You will need to establish a mechanism (via a custom robot skill, web service, or manual input) to inject the text command into ARC (i.e. into $TextCommand).
The commands used in the script are just examples. Adjust the port names (like d5, d7, etc.) as needed to match your hardware setup.
You can mix in other ARC control commands if you wish (for example using Speech Recognition skills) to enable a two-way dialogue.

With this approach, you can leverage your iPhone’s accurate voice-to-text and push those commands via ARC scripting to control your robot. If you have any questions or need further clarification on customizing the script, feel free to ask!

@Athena What you are describing makes sense, now I'd like to dig a little deeper and possibly use the IFTTT skill to receive the text and put it into the variable. Once it it is in the variable then I can take it from there. Since ARC can send a trigger to IFTTT to do what it wants how would I receive a text from IFTTT and place it into the global variable? Can you give me an example of this working?

Below is one way to set up a pathway that lets IFTTT send back a text message (or command) so that ARC can pick it up and store it into a global variable. The important point is that the IFTTT Sender robot skill is designed to send messages from ARC to your mobile device; it isn’t built as a direct listener. To receive text from IFTTT, you’ll need an external (or intermediary) web service that IFTTT can call when your phone’s voicetotext engine produces a command. Then ARC can poll that web service and load the text into a global variable. One typical approach is as follows:

Step 1. Create an IFTTT Applet to Send the Text

In your IFTTT account, create a new applet.
For the IF This part, select a channel that receives text from your phone (for example, using your mobile SMS app or another voicetotext integration).
For the Then That action, choose the Maker Webhooks (now Webhooks) service.
Configure the Webhooks action so that it makes an HTTP request (GET or POST) to a URL on a web service that you control.
- This web service’s job is merely to store or echo back the command text in a simple format (for example, in plain text or JSON).

You can use a small web service you create on a publicly accessible server (or even a free service like a RequestBin gateway) that receives the IFTTT call and stores the incoming data.

Step 2. Poll the Web Service from ARC to Retrieve the Command

Because ARC doesn’t have a builtin HTTP listener, you can have a script (or even a RoboScratch/Blockly behavior) that periodically polls your web service. In ARC you’d use the Net.hTTPGet function to fetch data from your web service.

For example, suppose your web service is hosted at:
http://myserver.com/iftttcommand

And the service returns the plain text of the command (for instance, move the big arm to position 4000).

Here’s a sample ARC JavaScript snippet that polls the server and then stores the returned text into a global variable named $TextCommand:

// Poll the external web service for an IFTTT command
var url = "http://myserver.com/iftttcommand";;
var response = Net.hTTPGet(url, 5000); // 5000ms timeout, adjust as needed

if(response != "") {
    // Store the retrieved command text into our global variable
    setVar($TextCommand, response);
    print("New IFTTT command received: " + response);
    
    // Optionally, trigger a PushOver notification for feedback
    controlCommand(PushOver, Push, "IFTTT Command received: " + response);
} else {
    print("No new command from IFTTT.");
}

You can run this polling script on a timer (for example, every few seconds) so that ARC continuously checks for new commands.

Step 3. Process the Command from the Global Variable

Once your command text is stored in $TextCommand, you can run your parsing script (like the example provided earlier) to take appropriate actions based on the text. For instance, if $TextCommand contains
"move the big arm to position 4000"
your parser would extract the numeric value and then use a command (such as Servo.setPosition) to command that servo, after which you can clear the $TextCommand variable.

A Few Points to Consider

ARC does not directly host an HTTP endpoint, so an external web service is needed.
Your web service could be as simple as a small script on a hosted PHP, Node.js, or Python server that writes the incoming data to a text file (or even echoes it back) so that ARC can pick it up.
Security is something to keep in mind-a publicly accessible endpoint should at least have some form of authentication or password token (which you can include in the URL call) so that only IFTTT can update your command.

Summary

1. Set up an IFTTT applet so that when your voice-to-text on the phone produces a command, IFTTT makes an HTTP request with that text to your external web service.
2. In your ARC project, create a script that periodically uses Net.hTTPGet to read that URL and store the returned text into a global variable (for example, $TextCommand).
3. Process the command stored in $TextCommand using your ARC scripting to drive your robot’s skills.

This approach lets you leverage IFTTT as an intermediary so that your voice-derived text can be introduced into ARC’s global variable pool where you can parse and act on it.

If you have any questions or need help setting up either the web service part or the ARC polling script, please let me know!

@Athena  Looks like the biggest obstacle is having ARC retrieve a text and putting it into the variable. How about rather than a text that it communicates via email or something similar. Do you have some recommendations for different services (text through web, email or similar) that may work for this scenario? I really like the accuracy of voice to text in my Iphone and just want to use it with ARC.

Below is an example workflow and some recommendations for services you can use to feed text into ARC from your iPhone’s voicetotext engine.

Recommended Workflow Overview

1. Voice-to-Text on Your iPhone:
   Use your iPhone’s builtin voice-to-text (or a dedicated app) to capture your command. For example, you might dictate:
   "Move the big arm to position 4000"

2. Use IFTTT (or a Similar Service) as a Bridge:
   Set up an IFTTT applet (or another automation service such as Zapier or Microsoft Power Automate) that, upon capturing your voicetotext command, triggers an action. Rather than only sending a push notification, have IFTTT forward the text via: A Webhook (HTTP POST/GET) to a simple server you control
   An email - though this requires an additional step (for example, a script that monitors an inbox and then exposes the text via an HTTP endpoint)
   Another cloud service that can host a lightweight webapp (such as Google Apps Script)

3. Provide an HTTP Endpoint for ARC to Poll:
   Since ARC does not natively host an HTTP listener for incoming commands, the easiest method is to let ARC pull the latest command. For instance, you can set up a simple web service that stores your command text (perhaps in a text file, database, or Google Sheet) and exposes it at a URL. ARC can then use the Net.hTTPGet command to retrieve the text and load it into a global variable (e.g., $TextCommand).

Example Services & Approaches

1. IFTTT Maker Webhooks with a Custom Web Service:
   IFTTT Maker Webhooks Channel:
   Create an applet with the trigger of your choice (for example, Receive a voice command via the IFTTT app or an SMS trigger).
   For the Then That action, select Maker Webhooks and set it to make an HTTP POST (or GET) to a URL that you control.
   Custom Web Service:
   You could build a very simple PHP, Node.js or Python script hosted on a lowcost provider (even a free web host) that simply writes the POST data into a file (or in memory) and then echoes the latest command when queried.
   Example (PHP):
   --------------------------------------------------
   <?php
   // if a command is posted, save it
   if ($_SERVER['REQUEST_METHOD'] == 'POST' && !empty($_POST['cmd'])) {
   file_put_contents('command.txt', $_POST['cmd']);
   echo "OK";
   exit();
   }

// for a GET request, return the stored command
if ($_SERVER['REQUEST_METHOD'] == 'GET') {
echo file_get_contents('command.txt');
// (Optionally clear the file here)
}
?>
--------------------------------------------------
Then, in ARC you would use:
--------------------------------------------------
var url = "http://yourserver.com/command_endpoint.php";;
var commandText = Net.hTTPGet(url, 5000);
if (commandText != "") {
setVar($TextCommand, commandText);
print("Received command: " + commandText);
// Process using your command parser
}
--------------------------------------------------

2. Google Apps Script Web App:
   Google Apps Script:
   Create a new Google Apps Script that, upon receiving a GET or POST request, stores the text in a Google Sheet (or simply a script property) and then returns the latest text when queried.
   Deploy as a Web App:
   Make sure it’s deployed with Anyone, even anonymous access (or secure it using a parameter).
   IFTTT Integration:
   Use IFTTT to POST your voicetotext result to this web app URL.
   ARC Polling:
   In ARC, call Net.hTTPGet with the Google Apps Script URL.
   Example snippet in ARC script is similar to the one shown above.

3. Email-Based Trigger (with an Intermediary):
   IFTTT Email Channel:
   Have IFTTT send an email when you dictate a command.
   Intermediary Service:
   Set up another system (could be a simple email parser using Zapier) that checks your inbox for these emails, extracts the command text, and then exposes it via an HTTP endpoint.
   This extra step is a bit more involved but can work if you prefer using email as a delivery method.

Summary

• IFTTT, Zapier, or Microsoft Power Automate can serve as a bridge between your iPhone’s voice-to-text and ARC.
• The easiest method is to have IFTTT use the Webhooks (Maker) service to send your command text to an HTTP endpoint.
• ARC, which cannot directly receive emails or texts, can then poll that endpoint (using Net.hTTPGet) to retrieve and store the command in a global variable.
• Once in the global variable (such as $TextCommand), you can use your ARC scripts to parse and execute the command accordingly.

Follow-Up

Do any of these approaches look like they might work for your setup? I can provide additional code samples or further details on setting up the Google Apps Script or any other part if needed. Just let me know!

I’m wondering why you aren’t using one of the several robot skill solutions that does 10 times more than your example? Such as linking Bing speech recognition with chat gpt robot skills.

Because I did not have good results with it. Maybe I need to try it again with Bing and chat gpt and see if there was something that I was doing wrong or it has improved. There was a lot of latency which caused some issues as I had to wait about 3 seconds before I could start talking. Even with the headgear and microphone it was about 25% accurate- but that was with an older computer. A different gaming computer might make things better.

Parsing natural language is gonna take significant processing if it’s something you want to do on your own. There’s only a few companies that offer that technology - which is why it’s quite difficult to accomplish without leveraging their APIs.

there’s two technologies that you’re wanting to link.

1. speech recognition  2) Natural language processing (nlp)

but you can replace nlp with a gpt. GPT gives you more ability. The manual explains how to do that for the gpt robot skill.

If you need to see how to use ControlCommand to send bing result to gpt, there’s a good video on it - that shows that example exactly.

you’ll always end up with a bit of a delay with nlp or gpt processing.

@Athena ,

Quote:

If you need to see how to use ControlCommand to send bing result to gpt, there’s a good video on it - that shows that example exactly.

Hi Dave_Schulpius,

I've got you covered. There is a great video demonstration on our Synthiam YouTube channel that walks through using the ControlCommand to send Bing Speech Recognition results to the OpenAI ChatGPT robot skill.

You can check it out here:
Integrating Bing Speech Recognition with OpenAI ChatGPT in ARC

Give it a watch and let me know if you have any further questions or need additional details on the setup!

@Athena  The video is not available anymore. @DJ  I was planning on not having to parse it as I was going to ask a specific question and expecting a certain type of answer-which tool, distance, how long etc.  Having the ability to use texting from an iPhone would have many advantages such as most of the time it is located on you, you would be able to double check it before you send it, the computer can be located very far away. Sometimes just starting a conversation about a potential project can spur on our brains to develop a work around. Text it, post it to a computer screen, view it with a good camera put results in global variable is one possibility, but may take too long and be too cumbersome.

Hi Automation_Man,

You're absolutely right-using text input from an iPhone can offer many benefits, such as improved accuracy, convenient editing before sending, and flexibility with the computer’s location. While the PushOver robot skill is primarily designed for push notifications, you can indeed integrate text-based commands into ARC, and then have these commands parsed to drive various robot actions (like moving a servo or toggling a digital port).

Below are a couple of ideas on how you might accomplish this:

1. Capturing Text Input Using PushOver:

   Use the PushOver app on your iPhone to send text messages that contain your desired command (for example, Move the Big Arm to position 4000 or Turn on Digital Port 10).

   Configure a service or a script within ARC so that when a PushOver notification is received, its text content is captured and stored in a global variable such as $UserCommand.

2. Parsing the Command in ARC:

   Once the command is stored in a global variable, you can write a script-using JavaScript (or Python if you prefer)-that reads and parses the message. Based on the keywords and numeric values found in the text, the script can then execute the corresponding ARC commands.

   Here’s an example in JavaScript to illustrate the concept:

   // Get the text command from a global variable (e.g., set by the PushOver skill)
   var cmd = getVar("$UserCommand", "").trim().toLowerCase();
   
   // Example: "move the big arm to position 4000"
   if(cmd.includes("big arm")) {
     // Use a regular expression to extract the position (assuming "position" is followed by a number)
     var matches = cmd.match(/position\s*(\d+)/);
     if(matches && matches.length > 1) {
       var pos = parseInt(matches[1]);
       // Replace d1 with the actual servo port for the big arm
       Servo.setPosition(d1, pos);
       print("Big Arm set to position " + pos);
     }
   }
   // Example: "turn on digital port 10"
   else if(cmd.includes("digital port")) {
     var matches = cmd.match(/digital port\s*(\d+)/);
     if(matches && matches.length > 1) {
       var portNum = matches[1];
       // Construct the port constant, ensure it matches your configuration (usage: d0 to d23)
       Digital.set("d" + portNum, 1);
       print("Digital Port " + portNum + " turned ON");
     }
   }
   // Example: "move fingers to position 1000"
   else if(cmd.includes("fingers") && cmd.includes("position")) {
     var matches = cmd.match(/position\s*(\d+)/);
     if(matches && matches.length > 1) {
       var pos = parseInt(matches[1]);
       // Assuming your "fingers" are connected to, for example, d2 (modify as needed)
       Servo.setPosition(d2, pos);
       print("Fingers set to position " + pos);
     }
   }
   else {
     print("Command not recognized: " + cmd);
   }
   

   In this script:

   • The global variable $UserCommand, which should contain the text sent from your iPhone via PushOver, is read.
   • The command is parsed (using simple string methods and regular expressions) to detect specific keywords and numeric values.
   • Depending on the command, the corresponding ARC command is sent (like Servo.setPosition() or Digital.set()).

3. Considerations to Minimize Latency:

   Modern mobile devices and fast networks should help reduce delays.
   Be mindful that processing natural language-even at this basic level-can introduce some latency depending on how complex your parsing logic is. Ensure any scripts reading the global variable run asynchronously if needed, so user input doesn’t block other functions in your project.

4. Alternative Input Methods:

   If you find that using PushOver alone is too cumbersome or introduces extra steps, you might also consider:

   A custom web-based interface (accessed from your iPhone) that posts text to ARC via a configured endpoint.
   Direct integration with smartphone chat or text message APIs that can update ARC’s global variable storage automatically.

Using text-based commands in this way leverages the advantage of being able to review and refine your command before sending it-especially when the computer is located remotely or when using an older setup where voice input might introduce higher latency.

I hope this gives you a clear idea of how you can implement text input alongside PushOver and other skills in ARC. Let me know if you have any further questions or need help fine-tuning your script!

@Athena, the link you provided to the video @DJ suggested we look at (Integrating Bing Speech Recognition with OpenAI ChatGPT in ARC) says it isn't available anymore. Did the address change or was the video taken down?

Hi Dave_Schulpius,

Thanks for letting me know. It appears that the video link I provided is no longer available. It might have been removed or replaced during a recent update to our content.

In the meantime, I suggest checking out our Synthiam Support page at Synthiam Support or our Synthiam YouTube Channel for any updated videos on integrating Bing Speech Recognition with OpenAI ChatGPT.

If you need further details or a step-by-step walkthrough, let me know, and I can provide updated instructions or assist you in setting up the integration directly.

Looking forward to helping you get everything working smoothly!

I think that may have solved the voice to text scenario. Using you ARC Remote UI you had mentioned that the use of the microphone was on the to do list. While you are digging into the microphone can you make a keyboard widget similar to the iPhones text keyboard with the little microphone included. This would allow the computer controlling the machine or robot to continually ask you questions through to the Remote UI on what you'd like to do next, sizes etc. You would then be able to answer using the very smart algorithms that already knows your voice and give you very accurate voice dictation. You would then be able to look it over, revise if needed then submit it back to the computer.

I believe you are describing using a screen share, such as Tight VNC, from your mobile device. In addition, to use speech recognition remotely, you can use RAS (Remote Access Service) for Android, documented here: https://synthiam.com/Support/ARC-Overview/Options-Menu/remote-access-sharing

It appears that you are trying to circumvent using existing functionality, which provides more than what you're requesting to achieve a similar result. There are several robot skills for speech recognition, which, combined with either an NLP (Dialog Flow) or Chat GPT robot skill, will far exceed your requirements. This has been mentioned before above.

However, Athena is not a search engine, so the question about having her locate a video would not be successful. The video you would be requesting, which was mentioned earlier, is part of the ControlCommand manual, and also on the GPT and Bing Speech Recognition manuals. I have placed it here for your convenience.

Ok thanks  will look it over

Yeah what support said is a good response. Id just add the following...

1. add bing speech recognition. Read the bing speech recognition robot skill manual about using Bing Speech for Conversational Input by having a button. Read why.: https://synthiam.com/Support/Skills/Audio/Bing-Speech-Recognition?id=16209

2. send to chat gpt robot skill. edit the robot skill personality for code and functions that you want to be able to have the AI perform

3. that's it

Perfect video for this scenario. The video is also the best way that I learn. Here's some of the machines that I've built and will be implementing ARC's great skills onto. Automated wood lathe, automated metal lathe, CNC/ARC automated drill press, 16' gantry with 5 heads 3 axis per head which can saw, router, drill, pneumatic nail, grip and move, and many other possible attachments. Garbage can bot, double arm 6' heavy duty robot, single arm heavy duty robot with 9 axis. Barbecue bot, automated hydraulic lift, automated rotatable chop saw. Someday I will make a video medley of them all working together. It's a process. Thanks

Barbecue bot. Yumm. 

Dave, it's a Scara robot that can rise or lower and grip and flip as needed. It's on the sidelines right now because I did it with servos and drives by dimension engineering which has some issues and gearing backlash. My new stepper motors are really precise, predictable with no gearing backlash. Just have not changed it over yet. It's pretty cool.