Dual Hbridge icon Dual Hbridge Two-channel Dual H-Bridge movement panel to control two DC motors (on/off) for forward, reverse, left, right via EZ-B digital ports; no PWM speed. Try it →
Australia
Asked
Resolved Resolved by Athena!
Speech Recognition Mic Variable

Speech Recognition Mic Variable

,

I'm using the Speech Recognition robot skill in ARC. From a Python script running in that Speech Recognition Skill, I'm attempting to start Object Recognition in the Camera Robot Skill. I converted the EZ-Script code from the "Object recognition" example by DJ Sures to Python. The script executes, but it does not capture the speech text from the user microphone in response to the prompt "What is the name of the Object?"

It seems the Speech Recognition Skill does not expose a variable equivalent to the EZ-Script example's "$BingSpeechPhrase". I have tried using "SpeechPhrase", but that always contains the previously spoken robot voice (TTS), NOT the text captured from the user microphone. Is there a variable I can use to obtain the captured speech text from the Speech Recognition Skill?

Note: I have a bespoke variable named "$CurrentSpeechPhrase" that another script uses to manage the robot voice and jaw animations.

My Python script (converted from EZ-Script) is shown below:

import time

# Ask for the object name
setVar("$CurrentSpeechPhrase", "what is the name of the new object")
setVar("$IsSaying", 1)
while getVar("$IsSaying") == 1:
    time.sleep(3.0)

# Reset speech variable and start listening
setVar("$SpeechPhrase", "")
controlCommand("Speech Recognition", "PauseOff")
time.sleep(0.25)
setVar("$SpeechPhrase","?????????")  #

# Confirm object name with the user first test
object_name = getVar("$SpeechPhrase")   # This does not match the input from the microphone #
setVar("$CurrentSpeechPhrase", "I heard you say " + str(object_name))
setVar("$IsSaying", 1)
while getVar("$IsSaying") == 1:
    time.sleep(0.1)

# Wait for Bing Speech to get a response
while getVar("$SpeechPhrase") == "":
    time.sleep(0.25)

# Confirm object name with the user
object_name = getVar("$SpeechPhrase")
setVar("$CurrentSpeechPhrase", "Excellent. Let me know when you are ready to teach me what the " + str(object_name) + " is.")
setVar("$IsSaying", 1)
while getVar("$IsSaying") == 1:
    time.sleep(0.1)

# Wait for 'OK' or 'Cancel' voice confirmation (Timeout: 60 seconds)
# Note: waitForSpeech is a native ARC global function
resp = Audio.waitForSpeech(60, "OK", "Cancel")
if resp != "OK":
    setVar("$CurrentSpeechPhrase", "ok, I will not learn the " + str(object_name))
    setVar("$IsSaying", 1)
    # Stop script execution
    exit()

# Disable current tracking to prepare for training
controlCommand("Camera", "CameraObjectTrackingDisable")

# Train the new object
controlCommand("Train Vision Object By Script", "Learn", str(object_name))
time.sleep(1.0)

# Wait while the camera is actively learning
while getVar("$CameraIsLearning") == True:
    time.sleep(0.25)

# Success confirmation
setVar("$CurrentSpeechPhrase", "Great, I know what a " + str(object_name) + " is.")
setVar("$IsSaying", 1)
while getVar("$IsSaying") == 1:
    time.sleep(0.1)

# Re-enable regular camera tracking
controlCommand("Camera", "CameraObjectTracking")

Thanks.



Related Hardware (view all EZB hardware)
EZ-B V4 by EZ-Robot
EZ-B v4/2 robot controller: dual Cortex ARM, Wi-Fi, audio/video, 24 servo/digital ports, I2C/UART, camera and Synthiam ARC control for custom robots
Wi-Fi / USB
Servos 24
Camera
Audio
UART 3
I2C
ADC 8
Digital 24

Related Robot Skills (view all robot skills)
Train camera vision objects via controlcommand(), attach camera, start learning, monitor progress, and return object name for scripts.
Windows Speech Recognition skill: detect custom phrases via PC mic, trigger configurable scripts/actions with adjustable confidence.

ARC Pro

Upgrade to ARC Pro

ARC Pro will give you immediate updates and new features needed to unleash your robot's potential!

Author Avatar
PRO
Canada
Twitter YouTube GitHub
#1  

Yes - the Speech Recognition skill does not provide a "$BingSpeechPhrase"-style variable for the recognized user phrase in the way the old Bing speech examples did.

What you want is usually the last recognized phrase variable exposed by the Speech Recognition skill itself, which in ARC is typically:

  • $SpeechPhrase = the last recognized phrase
  • $SpeechConfidence = confidence of the last recognition

However, in your case the reason you’re seeing the robot’s TTS text instead of the user’s microphone input is likely because your own custom variable ($CurrentSpeechPhrase) and/or the speech/jaw logic is updating $SpeechPhrase or your script is reading it at the wrong time.

Important distinction

  • Speech Recognition skill listens to the microphone
  • Audio.say... / TTS is the robot speaking
  • Those are separate paths

So if "$SpeechPhrase" is showing the spoken robot text, then that variable is likely being used by another script as a shared "current phrase" variable, not the Speech Recognition skill’s recognized input.


Better approach

Use the Speech Recognition skill’s built-in recognized phrase variable from that control, not your custom phrase variable.

If you’re using the Speech Recognition robot skill, check its settings:

  • Open the skill’s kebab menu (three lines in the top-right of the skill)
  • Look at the Phrase Variable setting
  • Also check the Confidence Variable

You can point those to unique variable names, for example:

  • "$RecognizedPhrase"
  • "$RecognizedConfidence"

That avoids collisions with your TTS/jaw script.


1) Change the Speech Recognition skill variables

In the Speech Recognition skill settings:

  • Set Phrase Variable to something unique like:
    • "$RecognizedPhrase"
  • Set Confidence Variable to something like:
    • "$RecognizedConfidence"

2) Read that variable in Python

Then your script should use:

object_name = getVar("$RecognizedPhrase", "")
print(object_name)

Why your current script is failing

This line is the key issue:

setVar("$SpeechPhrase", "")

If "$SpeechPhrase" is already being used by another part of your project, clearing it right before listening can cause conflicts. Also, your jaw/TTS logic is likely writing to a phrase variable that is not the microphone result.


Suggested Python pattern

Here’s a cleaner version of the speech capture portion:

# Clear only the dedicated recognition variable
setVar("$RecognizedPhrase", "")

# Enable speech recognition if needed
controlCommand("Speech Recognition", "PauseOff")

# Wait until a phrase is recognized
while getVar("$RecognizedPhrase", "") == "":
  sleep(100)

object_name = getVar("$RecognizedPhrase", "")
print(object_name)

If you need to wait for a specific prompt response like OK or Cancel, you can do:

resp = Audio.waitForSpeech(60, "ok", "cancel")

if resp != "ok":
  setVar("$CurrentSpeechPhrase", "ok, I will not learn the " + str(object_name))
  exit()

What I’d recommend in ARC

Since you have multiple scripts interacting, the safest setup is:

  • Use $CurrentSpeechPhrase only for robot speech / jaw animation
  • Use $RecognizedPhrase only for microphone recognition
  • Use $RecognizedConfidence only for confidence filtering

That separation will prevent the two systems from stepping on each other.



If you want, I can help you rewrite your whole script into clean ARC Python so it:

  1. prompts for the object name,
  2. captures the mic phrase correctly,
  3. confirms it,
  4. then starts the camera learning skill.
I am a customer support AI and still learning. Please verify important answers in our support documentation. For advanced AI features, use the AI Script Agent built into ARC.
Author Avatar
Australia
#2   — Edited

@Athena, thanks, yes a rewrite will save me time. Thanks. I did not know that there was a variable = $RecognizedPhrase One question: Can I still use $SpeechPhrase in other scripts or do I now have to change all other uses to $RecognisedPhrase as that would seem to just transfer the clash of variables to $RecognisedPhrase. Thanks again

Author Avatar
Australia
#3   — Edited

@Athena, @Synthiam Support, the problem remains. The Speech Recognition Skill does not populate the variable $RecognizedPhrase with the input from the microphone. By the way, some of the code you provided caused errors in Python - this line while getVar("$RecognizedPhrase", "") == "": results in an error "invoke(requires one argument not 2". This Python code runs but does not give the intended result.

import time
# Reset Speech variable and start listening
# Clear only the dedicated recognition variable
setVar("$RecognizedPhrase", " ")
# Enable speech recognition if needed
controlCommand("Speech Recognition", "PauseOff")
# Wait until a phrase is recognized
while getVar("$RecognizedPhrase") == "":
     #time.sleep(0.25)
  object_name = getVar("$RecognizedPhrase")
  print(object_name)
# Confirm object name with the user first test
object_name = getVar("$RecognizedPhrase")
setVar("$CurrentSpeechPhrase", "I heard you say " + str(object_name))
setVar("$IsSaying", 1)
while getVar("$IsSaying") == 1:
    time.sleep(0.1)
# Wait for Speech to get a response
while getVar("$RecognizedPhrase") == "":
    time.sleep(0.25)
# Confirm object name with the user
object_name = getVar("$RecognizedPhrase")
setVar("$CurrentSpeechPhrase", "Excellent. Let me know when you are ready to teach me what the " + str(object_name) + " is.")
setVar("$IsSaying", 1)
while getVar("$IsSaying") == 1:
    time.sleep(0.1)
# Wait for 'OK' or 'Cancel' voice confirmation (Timeout: 60 seconds)
# Note: waitForSpeech is a native ARC global function
resp = Audio.waitForSpeech(60, "ok", "cancel")
if resp != "ok":
  setVar("$CurrentSpeechPhrase", "ok, I will not learn the " + str(object_name))
  exit()
# Disable current tracking to prepare for training
controlCommand("Camera", "CameraObjectTrackingDisable")
#4  

There is no such variable as the one you commented on. The Speech recognition robot skill refers to the recognized phrase as $SpeechPhrase

However, speech recognition only recognizes the phrase configured in the list.

Because of that, you should only have the scripts associated with the respective phrase. Athena can explain more.

If you want free text for any phrases, use a different speech recognizer.

Author Avatar
Australia
#5  

@Synthiam Support. I'm not happy with your response, given that @Athena suggested code that might work and unfortunately it did not. I think the AI expected it would (as I did).

Here's my take on what you have said - Phrases entered in the "Phrase list" are attached to the SpeechPhrase variable (or whatever variable you put in the box on the first screen in the Skill), but once a script starts execution, there is no recognition of Microphone input (this is as you said line 2 above). Bing Speech recognition does this - attaches input to a variable but the in-house Speech recognition does not. Why not? One might think that as Bing Speech is restricted in use unless you are a PRO user, Synthiam have limited functionality in the Speech Recognition Skill to have makers use Bing with the 10/day phrase restriction or subscribe to PRO and use Bing. I'd like this question marked unsolved.

#6   — Edited

The speech recognition uses the Windows built-in speech recognition system, which requires pre-defined phrases. It does not convert "any spoken words into text". The detected phrase is stored in the variable mentioned above. Bing speech recognition uses a third-party paid service that converts any spoken words into text.

In 99.9% of cases, the variable of the Speech Recognition robot skill is not quite useful because the recognized phrase executes the respective code associated with it. For example...

Phrase: "Say Hello" will execute the script associated with that phrase. Phrase: "Move Forward" will execute the script associated with that phrase.

Hope that helps!

Read more here:

Windows Speech Recognition skill: detect custom phrases via PC mic, trigger configurable scripts/actions with adjustable confidence.

#7  

Oh, and one more thing. We attempted to make a free version of a full speech recognition system using the "built-in speech recognition," and it is called Total Speech Recognition. Because localized speech recognition technology is very limited, it is incredibly unreliable, unfortunately.

You can find more about it here:

Offline Windows speech recognition with open dictionary, confidence filtering, and scriptable phrase actions for ARC robots

Author Avatar
Australia
#8  

@Synthiam Support - Thanks. I guess I should direct my complaint to Bill Gates!