AI Robot Chat

This tutorial shows how to connect speech recognition to an AI chatbot so your robot can hear what you say, send that text to a PandoraBot, and speak the response. You can follow these steps in an existing project or start from a blank project.

Robot conversing with user demonstration screenshot
Example of a robot conversation using speech recognition and PandoraBot.

Requirements

  • Synthiam ARC installed
  • Working microphone
  • Sound card or an EZ-B with audio support (see EZB hardware overview to check compatibility)
  • An active internet connection

Robot Skills Used

The tutorial uses these robot skills. Click any item for the skill manual:

Programming Concepts Used

This tutorial demonstrates these ARC programming concepts and APIs:

Steps

  1. Add the Bing Speech Recognition skill

    In ARC, add the Bing Speech Recognition robot skill from the top menu: Project > Add Robot Skill > Speech Recognition > Bing Speech Recognition.

    ARC menu to add Bing Speech Recognition skill
    Selecting the Bing Speech Recognition skill from the Add Robot Skill menu.

  2. Add the PandoraBot skill

    From the top menu, add the PandoraBot skill: Project > Add Robot Skill > Artificial Intelligence > PandoraBots.

    ARC menu to add PandoraBots skill
    Selecting the PandoraBots skill from the Add Robot Skill menu.

  3. Open the Bing Speech Recognition config

    Open the Bing Speech Recognition skill and press the Config button in its menu bar to open its configuration window.

    Bing Speech Recognition skill config button in ARC
    Click the Config button to access the Bing Speech Recognition settings.

  4. Edit the "All Recognized Scripts"

    Click the All Recognized Scripts button. This script runs for every phrase the speech recognition translates. Note the variable shown — you will reference it shortly.

    All Recognized Scripts button in Bing Speech Recognition config
    The All Recognized Scripts area handles every recognized speech phrase.

  5. Switch to the JavaScript editor

    When the script editor opens, switch to the JavaScript tab to insert code that will forward recognized text to PandoraBot.

    JavaScript tab in script editor
    Select the JavaScript tab to write the forwarding script.

  6. Insert the PandoraBot SetPhrase command

    Right-click in the editor, choose PandoraBot, then select the SetPhrase option. This inserts the ControlCommand call used to send text to the chatbot.

    Insert PandoraBot SetPhrase command into editor
    Use the context menu to insert the PandoraBot SetPhrase command into your script.

  7. Replace the placeholder phrase

    With the inserted code visible, highlight the placeholder text ("[phrase]"). You will replace it with the speech variable that contains the translated text.

    Highlighted placeholder [phrase] in code editor
    Highlight the placeholder text to prepare for the variable insertion.

  8. Insert the $BingSpeech variable

    Open the Global Variables tab, find the $BingSpeech variable, and click it. This inserts the variable at the cursor location, replacing the placeholder. The variable contains the text recognized by the Bing skill.

    Global Variables tab showing $BingSpeech variable
    Insert the $BingSpeech global variable so the recognized text is forwarded to PandoraBot.

  9. Resulting code

    The final code instructs ARC to send the contents of $BingSpeech to the PandoraBot skill using ControlCommand. It should look like this:

    ControlCommand("PandoraBot", "SetPhrase", getVar("$BingSpeech"));

    This line uses ControlCommand() to call the PandoraBot skill's SetPhrase action with the recognized speech text.

  10. Save the script

    Press SAVE in the script editor to store your changes.

    Save button in script editor
    Save your JavaScript changes in the editor.

  11. Enable voice activity detection (optional but recommended)

    Check Auto Record using VAD to let ARC automatically detect human speech and begin recording when you speak. Hover the blue question mark for details on how VAD works.

    Auto Record using VAD option in Bing Speech config
    Enable Auto Record using VAD to automatically start recording when speech is detected.

  12. Save the Bing Speech configuration

    Press SAVE in the Bing Speech Recognition configuration to apply all changes to the robot skill.

    Saving Bing Speech Recognition skill configuration
    Save the configuration to activate the script and settings.

  13. Optional: Use the EZ-B speaker for PandoraBot TTS

    If you have an EZ-B with supported audio (for example EZ-Robot EZ-B v4 or IoTiny), you can configure PandoraBot to play speech through the EZ-B speaker. Open the PandoraBot config (three dots in the title bar), check Use EZ-B v4 Speaker, and press save.

    PandoraBot config option to use EZ-B v4 speaker
    Enable the EZ-B speaker option to have PandoraBot responses play through the EZ-B device.

  14. You're done — test the conversation

    Speak to the robot. The Bing Speech Recognition skill will translate your speech to text, the script will forward that text to PandoraBot, and PandoraBot will respond. Verify recording and audio output are working on your system.

    Successful robot conversation with PandoraBot response displayed
    When configured correctly, your robot can converse by forwarding recognized speech to PandoraBot and playing the response.

Notes & Troubleshooting

  • Ensure your microphone and audio output are working before testing.
  • If responses are not heard, confirm the PandoraBot TTS settings and your audio routing (local speakers or EZ-B speaker).
  • Use the global variable $BingSpeech whenever you need the translated text from Bing Speech Recognition.