Bing Speech Recognition Wake-Word Pause Issue

Question

I have a question about the ARC Robot Skill "Bing Speech Recognition". I am using the Bing Speech Recognition skill with wake-word enabled. The wake word works correctly and the skill plays the wake-word ding. After that, it starts listening and normally sends the recognized phrase to my script via the $BingSpeech variable. I want to prevent the robot from hearing its own spoken replies, so I tried using this control command during or after the robot’s response: controlCommand("Bing Speech Recognition", "PauseListening"); I also tested similar listener control commands such as StopListening. After PauseListening or StopListening is called from a script, the Bing Speech Recognition skill no longer works correctly with the wake word. The wake word still appears to trigger (the ding plays), but the skill gets stuck at "sending data" and never completes recognition or runs the recognized-speech script. Manual operation still works. If I manually press Start Recording, speak a phrase, and then press Stop Recording, recognition works and my script runs correctly. The issue only appears after using the listener control commands from a script. For troubleshooting, I replaced my recognized-speech script with a simple print of $BingSpeech, and that works correctly. I also removed the PauseListening / StopListening control commands from my script, and wake-word recognition works again. So it appears that calling PauseListening or StopListening from a script may leave the Bing Speech Recognition wake-word flow in a bad state. My question: Is PauseListening intended to be safe to call from a script while using wake-word mode? If so, is there a recommended way to pause Bing Speech Recognition during the robot’s speech and then safely resume it afterward? Environment: - ARC on Windows (PC-connected) - Robot Skill: Bing Speech Recognition - Wake-word enabled - Recognized text variable: $BingSpeech - Script invoked from the Bing Speech Recognition recognized-speech script

Athena · Answer

Yes - PauseListening / StopListening can interfere with the wake-word flow if they’re called from within the same Bing Speech Recognition workflow that is currently handling recognition. ### What’s likely happening With wake-word enabled, Bing Speech Recognition is managing a state machine: 1. wake word triggers 2. ding plays 3. it starts recording/sending data 4. it returns recognized text and runs your script If your recognized-speech script calls PauseListening or StopListening while that session is still active, the skill can end up in a partially paused state and the next wake-word cycle may get stuck at "sending data". ### Recommended approach Instead of pausing the skill itself during robot speech, use a separate guard variable in ARC to ignore recognition while your robot is talking. For example: javascript if (getVar("$RobotSpeaking", "false") == "true") { return; } print(getVar("$BingSpeech"));  Then, before your robot speaks: javascript setVar("$RobotSpeaking", "true"); Audio.sayEZBWait("Hello there"); setVar("$RobotSpeaking", "false");  If you are using Audio.say() instead of the wait version, you can monitor speech with the built-in variable: - getVar("$IsSpeaking") for computer TTS - getVar("$EZBPlayingAudio") for EZ-B audio ### Better pattern for wake-word robots A common pattern is: - keep Bing Speech Recognition running - when speech is recognized, set a "busy/speaking" flag - ignore any new recognition while the flag is set - clear the flag when the reply finishes Example: javascript if (getVar("$RobotBusy", "false") == "true") { return; } setVar("$RobotBusy", "true"); var text = getVar("$BingSpeech", ""); print(text); Audio.sayEZBWait("I heard you say " + text); setVar("$RobotBusy", "false");  ### If you really want to pause listening If you want to pause based on a button or external event, do it outside the recognized-speech callback, not from inside the recognition handler that is currently processing the phrase. Also, if you need more advanced control than Bing’s built-in behavior, ARC lets you create a custom robot skill: https://synthiam.com/Support/Create-Robot-Skill/Overview That can be useful if you want a dedicated "mute while speaking" state machine. ### My recommendation For your use case, I would avoid PauseListening and StopListening inside the recognized-speech script and use a variable flag instead. That’s the safest way to prevent the robot from hearing itself without breaking wake-word mode. If you want, I can help you write the exact JavaScript script for your setup - are you using Audio.sayEZB() or Audio.say() for the robot replies?

dbeard · Answer

Thank you, that explanation makes sense and matches what I am seeing. I tested this further and confirmed that the wake word itself is working. The skill hears the wake word, plays the ding, and $BingSpeech updates correctly when I keep the recognized-speech script very simple. The problem appears when I call PauseListening or StopListening from the script flow tied to the recognized speech. After that, the wake-word flow can get stuck at "sending data" and does not complete the recognition/script cycle correctly. Based on your explanation, I am going to avoid using PauseListening or StopListening from inside the Bing recognized-speech callback. Instead, I will keep Bing running and use an ARC guard variable such as $ROE_Voice_IgnoreUntilMs or $RobotBusy to ignore any recognition that happens while the robot is speaking. For now, I will keep the Bing recognized-speech script very small, something like: javascript print("$BingSpeech = " + getVar("$BingSpeech")); controlCommand("ROE Audio", "ScriptStart", "ROE Voice Command Router");  Then the router will decide whether to act on the phrase or ignore it based on a busy/ignore window. That seems like the safest pattern for wake-word use. I appreciate the clarification that the listener control commands should be used outside the active recognition callback flow.

Athena · Answer

That sounds like a solid approach, and your testing confirms the behavior well. Using a small Bing callback plus a separate router/guard variable is the right pattern for wake-word projects in ARC. It keeps the Bing skill stable and avoids interrupting its recognition state machine. A couple of tips to make your pattern even more reliable: - Use one ignore variable with a timestamp or short timeout window, so the robot can ignore its own speech only for the brief period it is talking. - If you are using computer TTS with Audio.say(), you can also watch getVar("$IsSpeaking"). - If you are using EZ-B audio with Audio.sayEZB(), use getVar("$EZBPlayingAudio"). Example guard pattern: javascript setVar("$RobotBusy", "true"); Audio.sayEZBWait("Hello there"); setVar("$RobotBusy", "false");  And in the Bing callback: javascript if (getVar("$RobotBusy", "false") == "true") { print("Ignoring speech while robot is busy."); return; } print("$BingSpeech = " + getVar("$BingSpeech")); controlCommand("ROE Audio", "ScriptStart", "ROE Voice Command Router");  If you want, I can help you turn your ROE_Voice_IgnoreUntilMs idea into a clean ARC JavaScript router script.

dbeard · Answer

Thank you. That matches what I found in testing. I removed all PauseListening / StopListening calls from the router and changed the Bing recognized-speech script to only print $BingSpeech and start my separate voice router. That made the wake-word flow stable again. Instead of controlling the Bing skill, my router now uses a timestamp guard variable: javascript Voice_IgnoreUntilMs  When a valid phrase is accepted, the router sets that to now plus about 15 seconds. If Bing hears the robot during that window, the router starts but immediately ignores the phrase. I also publish a few diagnostics: javascript Voice_Busy Voice_BusyReason Voice_BusyUntilMs  This seems to be working much better because Bing remains "ears only" and the router handles whether to act or ignore. I will continue testing with this pattern. Thanks for confirming that avoiding PauseListening / StopListening inside the recognition callback is the safer approach.

DJ Sures · Answer

Have you experienced the robot hearing its own replies? I ask because arc pauses and unpauses speech recognitions such as bing when it is speaking.  what command are you using for speaking? The audio.sayezb commands are the ones I believe that do that. So you shouldn’t need any code to handle it.

dbeard · Answer

Yes, I have experienced the robot hearing its own replies, but my setup is not using only Audio.sayEZB(). For ChatGPT replies, the OpenAI ChatGPT skill response script is using computer audio: javascript Audio.say(getVar("$OpenAIResponse"));  I also use several pre-recorded MP3 replies through a soundboard-style skill, for example: javascript controlCommand("Phrases", "Play", "status_normal_04.mp3");  So some replies are coming from Audio.say() through the PC audio path, and some are MP3 files played through a soundboard control. They are not all going through Audio.sayEZB() or Audio.sayEZBWait(). That may explain why Bing can still hear the robot’s own voice in my setup. If ARC’s automatic pause/unpause behavior mainly applies to Audio.sayEZB() speech, then my ChatGPT Audio.say() responses and soundboard MP3 responses may not be covered the same way. I originally tried to handle this by calling: javascript controlCommand("Bing Speech Recognition", "PauseListening");  from my voice router, but doing that inside the Bing recognized-speech flow caused problems. After that, Bing could get stuck at "sending data" on later wake-word attempts. The current workaround is that I no longer control Bing directly from the router. Bing stays running, and my router uses a short timestamp guard window. If Bing hears the robot during that window, the router ignores it instead of trying to pause Bing. So the short answer is: yes, I have seen the robot hear itself, but my speech output includes Audio.say(getVar("$OpenAIResponse")) and soundboard MP3 playback, not only Audio.sayEZB().

dbeard · Answer

Hi Synthiam team, I’m trying to better understand how ARC handles speech recognition pause/unpause behavior between the OpenAI ChatGPT skill and the Bing Speech Recognition skill. In my project, I’m using Bing Speech Recognition as the main "ears" for the robot, and the OpenAI ChatGPT skill for fallback/open-ended responses. During testing, it appears that when the OpenAI ChatGPT skill speaks a response, ARC/ChatGPT can pause Bing listening and then unpause it afterward. That behavior seems to work cleanly and does not appear to cause Bing to hang. However, when I try to pause/unpause Bing Speech Recognition myself through controlCommand() from a script, Bing sometimes appears to hang at the "sending data" step. So my question is: Is the OpenAI ChatGPT skill using a different internal pause/unpause mechanism than the public controlCommand() commands for Bing Speech Recognition? I’m not trying to work around ARC incorrectly. I’m trying to understand the recommended pattern so I can avoid destabilizing Bing recognition. If the correct approach is to let ARC/OpenAI manage Bing pause/unpause internally and avoid manually controlling Bing from scripts, that would be helpful to know. My current goal is to prevent the robot from hearing its own MP3 or speech responses without causing Bing Speech Recognition to lock up. Any guidance on the safest supported way to temporarily prevent Bing from processing robot speech would be appreciated. Thank you.

DJ Sures · Answer

I’m having it looked at and a support document created. Stay tuned.

Synthiam Support · Answer

A fix has been added to the next release of ARC, which will prevent the "sending data" hang. A fix has also been added to the pause unpause for using the Say commands toward any speech recognition robot skills.

dbeard

Upgrade to ARC Pro