How to add the Speech Recognition robot skill
- Load the most recent release of ARC (Get ARC).
- Press the Project tab from the top menu bar in ARC.
- Press Add Robot Skill from the button ribbon bar in ARC.
- Choose the Audio category tab.
- Press the Speech Recognition icon to add the robot skill to your project.
Don't have a robot yet?
Follow the Getting Started Guide to build a robot and use the Speech Recognition robot skill.
How to use the Speech Recognition robot skill
Using the Microsoft Windows Speech Recognition Engine, this skill uses your computer's default audio input device and listens for known phrases. Phrases are manually configured in the Settings menu, and custom actions (via script) are assigned to your phrases.Most robots make a lot of noise, so locating the audio input device on a robot is impractical. It is best to find the microphone on the controlling PC/Laptop, on yourself, or somewhere in the room (away from the robot). Turning the gain higher on the input device will allow voices to be recognized across large rooms and increase false positives. Test with different audio volume gains for the best resolution. Experiment with varying locations of the microphone and volumes for the best setup of your environment. Ideally, use a headset or Bluetooth mic rather than your laptop microphone.
Main Window
1. Pause Button
This button pauses the detection from the audio input device. No configured commands will be executed while the skill is paused.
2. Phrase List Button
Displays the phrases configured in the Settings menu to be available for detection. Here's the default phrase list:
3. Audio Waveform
This gives visual feedback that your audio input device (microphone) is configured correctly and is picking up voice/sounds.
4. Response Display
This will display feedback from the Speech Recognition skill. It will show the detected text from the phrase list or actions executed in the skill.
Settings
1. Confidence Drop-down
This drop-down is the minimum percentage of confidence that the computer detects from your voice. This robot skill will ignore any detected phrases with less confidence. If your voice isn't detected well, decrease the confidence value. A confidence value will appear in brackets beside the phrase in the Response Display when a phrase is detected. *Note: Adjusting the confidence to a lower value can produce false positives.
2. Setup Microphone Button
This button is a shortcut to the properties of your installed audio input devices. Verify that your device works by watching the soundbar for movement when you speak into that device.
3. Recognition Scripts
These scripts will execute every time a phrase is detected at the set confidence level or higher (All recognized) and when a phrase is detected with low confidence.
4. Enable/Disable Phrase Fields
These fields will set up the phrases to be detected to Pause/Unpause speech detection.
5. Enable/Disable Command Scripts
These scripts will execute when the Enable/Disable phrases are detected.
6. Language Drop-down
ARC uses the Microsoft Speech Recognition included with Windows. All languages supported by Windows Speech Recognition are also supported in ARC. You can configure Windows to listen to any language. ARC will default to EN-US (English) language if installed. Otherwise, ARC will default to the first installed language. You may select a language with this drop-down if more than one language is installed. For more information on installing additional speech recognition languages, Microsoft has a support document here: https://www.tenforums.com/tutorials/120631-change-speech-recognition-language-windows-10-a.html
Here's how you can add a new language pack:
- Go to Start and open Settings.
- Select Time & language > Language.
- Select the language you want to add speech, then select the Next button.
- Select the speech options you want to be included with the language.
- Sign out and then sign back in for the new speech pack to be added to speech options.
- Go back to Settings > Time & language > Language, select your new language, and then use the up-arrow button to move it to the top of the list and make it the default.
- Go to Speech, and make sure the Speech language setting is aligned with the previous settings.
- Sign out and then sign back in for the new settings to take effect.
- Select the desired language from the ARC Speech Recognition configuration menu.
7. Confidence Variable
This variable holds the value of the confidence percentage (in decimal format) of the last recognized phrase.
8. Phrase Variable
This variable holds the recognized phrase.
9. Phrase List
This is a list of default phrases that can be customized and add more phrases.
10. Command List
This is a list of default commands corresponding to the phrases in the same row, which can customize and add more commands.
11. List Management Buttons
These buttons manage the rows of phrases. They move the rows up and down, insert them, add more to the bottom, and delete them.
How to Use Speech Recongition
1) Install, configure, and test your audio input device (see instructions below).
2) Add the Speech Recognition Skill to your ARC project (Project -> Add Skill -> Audio -> Speech Recognition). *Note: Not to be confused with Advanced Speech Recognition.
3) Set up the scripts, phrases, and commands you'd like to use in the Settings menu.
4) Save your Settings and then speak your phrases into the audio input device to activate speech recognition.
Requirements
Headset or External Mic
A headset or external mic will produce better results than the internal PC/Laptop mic. A headset or mic will enable the recognition engine to "hear" your voice much clearer with less background noise. The background noise of the laptop, motors, radio, and room echo will cause the recognition software to return False Positives. This means the software recognizes an incorrect phrase. An external mic will also prevent the recognition software from hearing the robot speak. In short, it is important to use a Mic Headset or external Mic for a positive Speech Recognition experience.
Resources
Configure Audio Input Device
You might have to adjust the microphone input volume/gain. To adjust the mic volume, use the Microsoft Windows volume mixer, and first, make sure you have selected the correct input device. Your laptop or computer may have a few different mic devices. Maybe one is on a remote camera. Find the mic you'd like to use and adjust the volume. To find the volume settings that are ideal on your computer, follow these steps:
1) Right-click on the little speaker on your system tray
2) Select "Open Sound Settings."
3) In the "Input" section of the Sound Settings, you'll notice a little VU meter beside the active device. Make sure your active device is indeed the microphone you want to use. By making sounds, the VU meter should move.
4) Click on the "Device Properties" and locate the volume slider for the microphone. We usually have our volume set for 78. Play around with different volumes until you see your voice being picked up by the VU meter. Adjust the volume input level/gain to display your voice's regular volume near the middle of the VU Display graph. The recognition software will not work if the level/gain is too high because the input audio will be distorted.
Voice Training
You may train your computer for speech recognition by using the training wizard. Find the training wizard under Speech Recognition within the Windows Control Panel.
Trouble-shooting
If you receive an error that the input device could not be opened, the Windows privacy/security may have the feature disabled for Synthiam ARC. You can enable this privacy feature by following the directions on this support document: https://synthiam.com/Support/troubleshooting/camera-audio-microphone-issues
Video
Related Tutorials
tutorial
Randomizing Answers In Speech Recognition
Want to make your robot respond more naturally and not the same thing every time? Heres how to make it answer...
tutorial
Speech Recognition Say It Didn't Understand You
The speech recognition uses a confidence level to determine if it understood your verbal command. This tutorial...
tutorial
Speech Recognition Tutorial
Speech recognition is becoming a very popular way to control robots. This tutorial will explain the EZ-Builder speech...
tutorial
Synbot Plugin Tutorial - Interface To Syn Bot Software
How To install bot software, EZ builder Synbot plugin and example SIML project files
tutorial
Vision Training: Object Recognition
Learn how to program a robot to learn and memorize an object.
Related Hack Events
Related Questions
question
Brazilian Speech Recognition
Good afternoon everyone, I would like to know how do I work the command of Voice in Portuguese,? Ive tried everything...
question
Speech Recognition Screen Popping Up.
So I have been tinkering with speech recognition and enjoying it. Recently after a speech interaction completes I get...
Upgrade to ARC Pro
ARC Pro is more than a tool; it's a creative playground for robot enthusiasts, where you can turn your wildest ideas into reality.
All is good now, always did work, just windows going it's thing, oh well
Speech recognition uses the built-in microsoft speech recognition engine. There is nothing synthiam can do to enhance or change the engine, as it's not opensource and owned by Microsoft. Follow the instructions above
All is good now
just did the old windows reboot, now works
Hi there,
Sometimes when I open ARC, my speech recognition box doesn't detect the soundwaves. It shows the soundwave box as grey and not the red and green lines, even when I have connected the program with my microphone.
The person I am working with came to the conclusion that it might be because I didn't start with the JD Bare file when I opened the project. He has been starting with the JD Bare project every time we open the program and merging it with the previously saved program, rather than opening the saved program directly because of this issue with the speech recognition. But I thought it was inconvenient to do that and that surely I should be able to open previously saved projects and have them work.
I have run out of ideas. Any suggestions on how to fix this are much appreciated.
Can you verify the speech recognition skill that you’re using is the bing speech recognition? Or just speech recognition?
General speech recognition, not bing.
Okay - I’ll see if I can reproduce it. Stay tuned
Thanks, DJ
This is what the Speech Recognition window looks like for me sometimes when I open my previously saved programs. Notice how the soundwaves cannot be seen up the top right? I'm baffled!
Hi, I am not trying to interfere between you and DJ (he knows the best) but it looks like to me you don't have a mic setup.
If you are using Windows 10:
Did you check your windows program and make sure it is setup like the information above?
Also check your in windows, Resources - Configure Audio Input Device
in Control Panel\All Control Panel Items - there is sound item for you to adjust to the correct levels (playback and recording)
Make sure, in the "sound item" you microphone is "enabled" in "recording"
I did the test, disabled it and it looked your your example above,
then "restart ARC" - I enabled the mic in sound, all good in speech recognition. ( you can do this also in the speech recognition, setup mic)
EzAng
Do you have an icon in the system tray that shows the mic being used?
windows controls what apps can use the mic and shows when it’s in use.
Thankfully, it's not ARC that cancels. It's the speech recognition script I am running. When the pause button on the speech recognition tab is not activated, background noise is still able to be picked up by the software and in some cases, the "robot stop" command is accidentally triggered causing my voice recognition program script to terminate before it has completed its run.
At the time this snapshot below was taken, I was not talking at all, yet as you can see, for some reason, robot stop was still detected. I have been avoiding this by clicking the pause button and that way the background noise isn't detected.
Yes - that’s a challenge with speech recognition across the world
are you using a lapel style mic as described in the speech recognition manual?
a Microphone that listens to the entire room is going to have a very very very very difficult time understanding noise vs commands. There’s no consciousness that can relate the current activity to what noises to focus on in a computer vs animal life. We take for granted the number of things our brain does to focus on particular events around us and filter out things that don’t matter to the current situation.
you can help A robot focus by limiting the sensory input. In this case using a microphone that has shorter range such as a lapel or Bluetooth headset etc
also, if you weren’t talking and stuff is being picked up - looks like the mic volume might be cranked because you’re attempting to use a laptop microphone
this conversation should be in the correct thread, not soundboard. This should be in the speech recognition skill for proper categorization. I moved it for you
Is there a way to script the pause to unpause per say ? every time i boot the robot and ARC autoloads the pause is activated on the app, i tried ControlCommand("Speech Recognition", "UnpauseListening") it doesnt work ?
Thanks :-)
You can see the list of controlcommands for each robot skill. There is one to unpause but I’m on my phone so can’t look it up. Here’s the manual page for how to see the list: https://synthiam.com/Support/Programming/control-command
strange that it’s paused though. I know there was a bug with Microsoft Windows for a while that caused something similar. Hopefully the bug didn’t return
Try manually unclicking the pause button then save your project and reboot the computer.
Additionally you can place the ControlCommand unpause command in an INT script that starts automatically every time a windows desktop shortcut is clicked.
when installing, starting, writes the following error:
The microphone works, in the settings I see a green bar jumping.
@FANT0MAS, You may need to enable Synthiam ARC in the microphone input privacy settings for Windows. This can be done per the instructions on this link: https://synthiam.com/Support/troubleshooting/camera-audio-microphone-issues
Have been using the typical speech recognition with headset and it has been working great, except for when I want Blockly to ask for users input for a word or number so it can put it into a variable. It seems like it is shutting off the headset and then it says there is no audio signal (seems like it's conflicting). Have tried to put a ControlCommand to pause off in speech rec but it doesn't fix it. It also shuts off when you close the computer for a restroom break and when you turn it back on it says that it does not have an audio signal, so I have to shut down ARC and bring it back up again. I have made sure the settings in Windows was allowing ARC to use microphone as you have mentioned. Tried to turn off headset and turn back on -thought it might be the bluetooth but it did not regain connection with the speech rec skill.
Could it be the headphones themselves turning off or going into power saving mode? If so maybe there's a setting on the headphones to keep them from doing that?
Thanks for your input Dave, Tried it again after closing computer for a few minutes and coming back and it worked ok this time by putting in starting words to enable pause off-thought I tried that before. Now if I can just get the headset to not shut off when asking for a word or number in Blockly should be all set.
Update Tried to bring it out to my shop and open it up again but the headset doesn't work. Pushed the headset on/off button and it mention powering off which meant it was on the whole time. Tried powering back on it mentions connected but when I try speak even with enable phrase it doesn't reconnect without closing ARC and reopening.
That's because the headset most likely disconnects and reconnects, which changes the default input device. Unfortunately, it's not something ARC can solve because that's controlled by Microsoft Windows (a different company than us). Still, you can solve it using a wired headset or discover why your wireless headset disconnects. Also, ensure your headset is connected before launching ARC so it's the default device.
Windows has default input devices. That's nothing to do with Synthiam ARC - it's a design by Microsoft Windows. We can't change that. If the device disconnects and reconnects, windows will change and remove the default input device.
So, you must figure out how to constantly keep your wireless headset connected.
Ok thanks for the explanation and will keep headset on and ARC on when going out for a little while-that one is solved. Is there any recommendation when having Blockly ask questions to put it into a variable? It seems to turn off the headset when asking questions.
You can look at the CODE that Blockly generates. Blocky does not run anything. Blockly is not a programming language. Blockly generates JavaScript code. It's the JavaScript code that runs. Look at the CODE tab and you can see the code that is generated.
Blockly does not have anything like features of speech recognition or anything. Blockly has blocks that represent code commands. Blockly generates code; that's all it does. It's a graphical representation of the code that it generates.
So, the code commands you refer to do not "turn off" anything. They use the same speech recognition engine that we're discussing here. Microsoft makes the speech recognition engine and the javascript command executes it.
Hope that helps
Yup you're right and I understand. I need to dig into it and will figure it out.
Hi, can anyone tell me how to change the language in speech recognition? I have installed 'Cantonese,' but I can't find it in the dropdown menu. Am I missing any steps in the process?
You can read the manual above rather than repeating it here. There's a whole section on selecting the language and what languages are supported. Scroll up to the menu
Hi DJ,
How are you?
Perhaps, I haven't explained my question clearly. I have successfully installed some languages in the drop-down menu of "Speech Recognition," but I am facing issues with the "Cantonese" language. Despite checking and installing all the features for Cantonese, it still does not appear in the drop-down menu. Is there something missing? I apologize for any inconvenience and misunderstanding.:)
Is it possible the name for Cantonese could be listed as something else in the drop down? I found this about other names for Cantonese:
Hi Dave,
Thank you for your message!
In Microsoft, only "zh-HK" is recognized as Cantonese, and the voice setting should be selected as "Tracy." It works seamlessly with "AI Chat GPT" and "Bing Speech Recognition" in ARC program. Then, I have installed all the Cantonese features, but for some reason, it does not appear in the drop-down menu in the Speech Recognition settings. I'm wondering if it is not supported in this application?
Those are voices, not speech recognition. Voices are the sounds made to speak. So you can select Tracy from the list of voices and speak. But those are not speech recognition.
The window you want looks like this..
I used google to find this for you: www.tenforums.com/tutorials/120631-change-speech-recognition-language-windows-10-a.html
Thank you DJ, Cantonese does not appear in the drop-down menu of "Speech Recognition" in my Windows 11. It should not be supported! However, I use Cantonese dictation for the text box in Windows 11, and it can recognize Cantonese. Therefore, I assumed that Cantonese Speech Recognition has been installed. Anyway, thanks for your help! Cantonese Dictation