Windows Speech Recognition skill: detect custom phrases via PC mic, trigger configurable scripts/actions with adjustable confidence.
How to add the Speech Recognition robot skill
- Load the most recent release of ARC (Get ARC).
- Press the Project tab from the top menu bar in ARC.
- Press Add Robot Skill from the button ribbon bar in ARC.
- Choose the Audio category tab.
- Press the Speech Recognition icon to add the robot skill to your project.
Don't have a robot yet?
Follow the Getting Started Guide to build a robot and use the Speech Recognition robot skill.
How to use the Speech Recognition robot skill
The Speech Recognition robot skill uses the built-in Microsoft Windows Speech Recognition Engine to listen for known phrases using your computer’s default audio input device (microphone). Phrases are configured manually in the Settings, and each phrase can trigger a custom action using scripts and commands.
Most robots generate significant noise from motors, servos, fans, and room echo. Because of this, placing the microphone on the robot is often impractical. For best results, place the microphone on the controlling PC/laptop, on yourself, or elsewhere in the room closer to your mouth and farther from the robot. Increasing microphone gain can help detect speech across larger rooms, but it also increases false positives, so keeping the microphone closer to your mouth is ideal. A headset or Bluetooth microphone will generally provide far better results than a built-in laptop microphone.
Main Window
1. Pause Button
Pauses audio detection. While paused, the skill will not recognize phrases and no configured actions will execute.
2. Phrase List Button
Opens the phrase list configured in the Settings. This allows you to quickly review what phrases the skill is currently listening for.
3. Audio Waveform
Displays live audio waveform feedback to confirm your microphone is configured correctly and actively receiving sound.
4. Response Display
Shows detection and execution feedback. When a phrase is recognized, the log will display the matched phrase (and confidence),
along with the action/script that was executed.
Settings
1. Confidence Drop-down
Sets the minimum confidence required for a phrase to be accepted. Phrases detected below this threshold are ignored.
If your phrases are not being detected reliably, reduce the confidence setting. When a phrase is detected, the confidence value
appears in brackets in the Response Display.
Note: Lower confidence thresholds increase the chance of false positives.
2. Setup Microphone Button
Opens the Windows microphone device properties dialog. Use this to verify the correct input device is selected and to confirm the audio meter responds when you speak.
3. Recognition Scripts
Scripts that run when a phrase is recognized at or above the confidence level (All Recognized),
and when a phrase is detected with low confidence. These are useful for logging, diagnostics,
or responding differently based on recognition quality.
4. Enable / Disable Phrase Fields
Phrases used to pause and unpause speech detection. These act as voice-controlled toggles for enabling or disabling recognition.
5. Enable / Disable Command Scripts
The scripts executed when the Enable/Disable phrases are recognized. This is commonly used to control robot behavior
such as entering a “listening mode” or disabling speech during loud activities.
6. Language Drop-down
ARC uses the speech recognition capabilities built into Windows. Any language supported by Windows Speech Recognition is also supported by ARC.
ARC will default to EN-US if installed, otherwise it will use the first installed language that supports speech recognition.
If multiple supported languages are installed, select the desired language here.
For more information on installing additional speech recognition languages, view:
https://www.tenforums.com/tutorials/120631-change-speech-recognition-language-windows-10-a.html
Here is how to add a new language pack:
- Go to Start and open Settings.
- Select Time & language > Language.
- Select the language you want to add speech to, then select the Next button.
- Select the speech options you want included with the language.
- Sign out and sign back in for the new speech pack to be added to speech options.
- Go back to Settings > Time & language > Language, select your new language, and move it to the top to make it default.
- Go to Speech and ensure the Speech language setting matches your new default language.
- Sign out and sign back in for the new settings to take effect.
- Select the desired language from the ARC Speech Recognition configuration menu.
7. Confidence Variable
The variable that stores the confidence value (decimal format) of the last recognized phrase.
8. Phrase Variable
The variable that stores the text of the last recognized phrase.
9. Phrase List
The list of phrases to recognize. You can customize the defaults and add additional phrases.
10. Command List
The command/action corresponding to each phrase in the same row. You can customize commands and add additional rows.
11. List Management Buttons
Buttons for managing phrase rows: move up/down, insert, append, and delete.
How to Use Speech Recognition
- Install, configure, and test your audio input device (see Resources below).
-
Add the Speech Recognition skill to your ARC project:
Project → Add Skill → Audio → Speech Recognition.
Note: This is different from Advanced Speech Recognition. - In Settings, configure your phrases and the corresponding commands/scripts.
- Save your settings, then speak your configured phrases into the microphone to trigger the actions.
Requirements
Headset or External Mic
A headset or external microphone produces better results than a built-in PC/laptop microphone. It helps the recognition engine capture your voice clearly with less background noise. Laptop fans, motors, radio interference, and room echo can cause false positives (the skill recognizes an incorrect phrase). An external mic also helps prevent the speech engine from hearing the robot’s own speaker output.
Resources
Configure Audio Input Device
You may need to adjust your microphone input volume/gain. Use the Windows Volume Mixer and ensure you have selected the correct input device. Some systems have multiple microphones (for example: webcam microphone, headset microphone, Bluetooth microphone). Follow these steps:
- Right-click the speaker icon in the system tray.
- Select Open Sound Settings.
- In the Input section, confirm the correct microphone is selected and that the VU meter moves when you speak.
- Click Device Properties and adjust the volume slider. We often use a value around 78, but your setup may differ.
- Adjust volume so normal speech peaks near the middle of the VU meter. If the gain is too high, audio distorts and recognition quality drops.
Voice Training
You can train Windows Speech Recognition using the built-in training wizard. Open the Windows Control Panel, search for Speech Recognition, and run the training wizard to improve accuracy for your voice.
Troubleshooting
- If you receive an error that the input device could not be opened, Windows privacy/security settings may be blocking microphone access for Synthiam ARC. Follow this guide to enable microphone access: https://synthiam.com/Support/troubleshooting/camera-audio-microphone-issues .
-
If you receive an error stating Voice Recognition was unable to start (invalid OS or missing device), verify:
- A microphone is set as the default recording device in Windows sound settings.
- A Windows language pack is installed that supports Windows Speech Recognition.
To confirm Windows Speech Recognition is working, open the built-in Windows Speech Recognition application: click the Start button and type Speech Recognition. Launch Windows Speech Recognition to verify your microphone, operating system, and language support. This Microsoft tool includes diagnostic dialogs that ARC does not provide.
Video
Related Tutorials
Randomizing Answers In Speech Recognition
Speech Recognition Say It Didn't Understand You
Speech Recognition Tutorial
Synbot Plugin Tutorial - Interface To Syn Bot Software
Vision Training: Object Recognition
Related Hack Events
Related Questions
Brazilian Speech Recognition
Speech Recognition Screen Popping Up.
Upgrade to ARC Pro
Unlock the true power of automation and robotics by becoming a proud subscriber of Synthiam ARC Pro.
@FANT0MAS, You may need to enable Synthiam ARC in the microphone input privacy settings for Windows. This can be done per the instructions on this link: https://synthiam.com/Support/troubleshooting/camera-audio-microphone-issues
Have been using the typical speech recognition with headset and it has been working great, except for when I want Blockly to ask for users input for a word or number so it can put it into a variable. It seems like it is shutting off the headset and then it says there is no audio signal (seems like it's conflicting). Have tried to put a ControlCommand to pause off in speech rec but it doesn't fix it. It also shuts off when you close the computer for a restroom break and when you turn it back on it says that it does not have an audio signal, so I have to shut down ARC and bring it back up again. I have made sure the settings in Windows was allowing ARC to use microphone as you have mentioned. Tried to turn off headset and turn back on -thought it might be the bluetooth but it did not regain connection with the speech rec skill.
Could it be the headphones themselves turning off or going into power saving mode? If so maybe there's a setting on the headphones to keep them from doing that?
Thanks for your input Dave, Tried it again after closing computer for a few minutes and coming back and it worked ok this time by putting in starting words to enable pause off-thought I tried that before. Now if I can just get the headset to not shut off when asking for a word or number in Blockly should be all set.
Update Tried to bring it out to my shop and open it up again but the headset doesn't work. Pushed the headset on/off button and it mention powering off which meant it was on the whole time. Tried powering back on it mentions connected but when I try speak even with enable phrase it doesn't reconnect without closing ARC and reopening.
That's because the headset most likely disconnects and reconnects, which changes the default input device. Unfortunately, it's not something ARC can solve because that's controlled by Microsoft Windows (a different company than us). Still, you can solve it using a wired headset or discover why your wireless headset disconnects. Also, ensure your headset is connected before launching ARC so it's the default device.
Windows has default input devices. That's nothing to do with Synthiam ARC - it's a design by Microsoft Windows. We can't change that. If the device disconnects and reconnects, windows will change and remove the default input device.
So, you must figure out how to constantly keep your wireless headset connected.
Ok thanks for the explanation and will keep headset on and ARC on when going out for a little while-that one is solved. Is there any recommendation when having Blockly ask questions to put it into a variable? It seems to turn off the headset when asking questions.
You can look at the CODE that Blockly generates. Blocky does not run anything. Blockly is not a programming language. Blockly generates JavaScript code. It's the JavaScript code that runs. Look at the CODE tab and you can see the code that is generated.
Blockly does not have anything like features of speech recognition or anything. Blockly has blocks that represent code commands. Blockly generates code; that's all it does. It's a graphical representation of the code that it generates.
So, the code commands you refer to do not "turn off" anything. They use the same speech recognition engine that we're discussing here. Microsoft makes the speech recognition engine and the javascript command executes it.
Hope that helps
Yup you're right and I understand. I need to dig into it and will figure it out.