Script To Activate Bing Speech Recognition

Question

I've got the Bing Speech Recognition working and WOW it's great! I have been trying to come up with a script that will activate it - so I don't have to click on the Start and Stop Recording tab - no luck. Is something like this possible? Also, can Bing Speech be coupled with the Talk Servo Control? Thanks

fxrtst · Answer

Oh I hope someone comes up with this... will make me a happy person!

bhouston · Answer

Will, how do you have the jaw on Alan rigged to move with the speech? It looks great!

fxrtst · Answer

I use a separate voice board made by Audio Servo Controller. I just bought a Scary Terry board to test on as well. But my results have been great with ASC. I also have a very fast servo which helps.

bhouston · Answer

Thanks, I am really happy with the way the Talk Servo Control works for my InMoov jaw. It would be great if the Bing Speech plugin could work the way the Speech Recognition Control works.

thetechguru · Answer

You can start and stop the plugin with control command. The trick will be knowing when speech is done to stop and send the data. Timer would not be good because it would be waiting too long for short utterances, and could cut you off on longer ones. You could have a "secret" button that triggers a digital port to start and stop the script. Probably other similar solutions. Need to think about this one. Alan

Andy Roid · Answer

I'm not clever enough but wasn't there a Frequency detector plugin that maybe someone can modify to determine an external or internal "sound" to allow the script to start or stop? Ron

thetechguru · Answer

@Ron, That is an interesting idea. Wouldn't work with speaker independence since we speak at different frequencies, and if you act on anything between 300hz to 3.4Khz (enough to catch all human voices, this is what the telephone network narrow band operates on) it would also pick up virtually any other environmental noise, but might be able to narrow it down to a specific users common frequency range. Alan

fxrtst · Answer

Interesting ideas! The end recording does seem to be difficult to nail down. You could have key work to activate recording like the Echo devices. Like "hey Alan"...then start recotding... but ending the recording challenging!

Dunning-Kruger · Answer

PC Mic uses the variable $SoundValue... Maybe it can be used monitor your voice level while you are inputting speech... When the sound drops below a certain pre-determined level (because you stopped speaking) it can be used to stop data being sent to Bing?

Andy Roid · Answer

Hey Will and Alan, What about taking that old Scary Terry board Will has. Maybe use it to put a signal back in to the EZB from the speaker output. (Just for a test). I think you can set a threshold value which drives the "LED EYES" maybe this can be used as an input to the EZB? Just a thought? Ron

fxrtst · Answer

@Ron, But wouldn't that be Alan finishing talking, not the speaker? But that i a great idea for something else......not sure yet what that is...but now its going to lay an egg and I'm going to sit on it til it hatches!

thetechguru · Answer

Quote: Hey Will and Alan, What about taking that old Scary Terry board Will has. Maybe use it to put a signal back in to the EZB from the speaker output. (Just for a test). I think you can set a threshold value which drives the "LED EYES" maybe this can be used as an input to the EZB? Just a thought?  Not sure what you are getting at here. That would detect when EZ-B has stopped speaking, but there are many ways to do that. What we need is to detect when the user has stopped speaking so that we can send the voice to Bing. We could ask DJ for an update. The way we do this in Telephone Interactive Voice Response is to set a silence detection timeout to detect when speech is complete. But, the speech is being detected by the voice recognition engine, so is not impacted by background noise. In the plugin, EZ-B is telling Bing when to stop listening. The API may support detection of end of speech. I haven't read anything about it yet. Alan

Andy Roid · Answer

LOL Hook it up to the mic? LOL , I know I am a hacker, but Hey, if it stimulates brain cells to get an answer it's worth it. Ron

fxrtst · Answer

Good points Alan. @Ron, thats what forums are for! Exchanging ideas!

bhouston · Answer

So ultimately, what would be great is a cross between the Bing speech and the current Speech Recognition Control. A control that could; - activate with your speech - like the Speech Recog. now - put specific statements in and specific response will occur - like it does now - will search for an answer if the statement is not recognized - like Bing does thru pandorabot - can 'hear' like Bing - personally speaking, this is the most impressive part of Bing - will work the Talk Servo.

fxrtst · Answer

Absolutely agree. As I said before..game changer for this kinda robot/human exchange.

bhouston · Answer

DJ has been quite on this thread - so i'm thinking he's busy working on it. hint hint wink wink.

Dunning-Kruger · Answer

Stupid question was @DJ just using pandorabot (and you guys too) in that JD Chat video in conjunction with BingSpeech?.... is this all you need to get a chatbot up and running are these two controls?

kamaroman68 · Answer

Not a stupid question, I want to fully understand this also.

thetechguru · Answer

Yes, JD was just using a Pandorabot. If you download the sample EZ-B project on the plugin page, it is pre-configured to use the EZ-Robot pandorabot, you just need to add your Bing API key and it starts working. It is really quite amazing. Alan

bhouston · Answer

I have it working with the following controls; Bing Speech Recognition Plugin Pandorabot Control Speech Settings Control It works without an EZB connected. you can chose whether the sound comes thru your PC or an EZB. Follow the tutorial for the setup to get the Bing code.

bhouston · Answer

I can't seem to get it to work today. I get this message when I say something to it.  I tried deleting the Pandorabot Control and reinstalling it into the project - didn't help. Any thoughts?

Dunning-Kruger · Answer

Using @DJ's sample program I am getting a Pandorabot error: system.net web exception, the remote server returned an error 502 bad gateway... Perhaps pandorabot is down right now? @Bob... I am getting that same error

bhouston · Answer

Ya, must be Pandorabot. I'll try later.

Dunning-Kruger · Answer

Good to know I was doing it right... couldn't figure out why it wasn't working....

Andy Roid · Answer

Is that why I can't load the Plugin too? Ron

Dunning-Kruger · Answer

@ANDY ROID No I don't think that's why unless you are getting the same error as Bob and I... You need to give us more info on what "won't load" means.... we need details

Andy Roid · Answer

Hi Richard, I went back to the SteveS thread "Down and Out". I am having the same issue as he is. Thanks for the reply Ron

thetechguru · Answer

Ron, 2 people (me and someone else) already provided solutions to the 'down amd out' thread. There is a missing Visual Studio dependency (and it won't work at all in Windows 7) Alan

Andy Roid · Answer

The C++ update didn't work, Tried the Visual Studio, All is good. Loaded and running fine. Thanks to all. Ron

DJ Sures · Answer

I updated the plugin with the file that is needed without having to install ALL of visual studio: https://synthiam.com/redirect/legacy?table=plugin&id=179

Andy Roid · Answer

Thanks Dj. Ron

lizpoir · Answer

I'm currently facing the same problem, you could start Bing when the answer from Pandora was issued. About a Soundservo. But how do you stop Bing? Time would be inappropriate.

thetechguru · Answer

I'll be setting up a a script to stop listening and send to Bing (or Google, I plan on trying that plugin too) using my Android Wear smartwatch. See: https://synthiam.com/Community/Questions/9125 Alan

DJ Sures · Answer

Bing now supports controlcommand() for start and stop Also has an auto record function, which of course will use up a lot of data in a noisy environment... https://synthiam.com/redirect/legacy?table=plugin&id=179

fxrtst · Answer

You are brilliant...again! Thanks!

lizpoir · Answer

Hello DJ Sures, Now the Bing speech recognition is perfect. And it also works. Many Thanks

DJ Sures · Answer

The latest update includes an energy calculation for auto recording. This will prevent false positives. The level configuration is different, as it's a decimal value. however, a suggested range to try is provided in the config menu. I prefer 0.03 on my laptop. Although my other laptop prefers 0.02. https://synthiam.com/redirect/legacy?table=plugin&id=179

fxrtst · Answer

Once again hats off to you good sir!

bhouston · Answer

I've been away for a bit but I have been following the development of the Bing Speech recognition plugin. The way Bing 'hears' what I'm saying is great. Is there a way for it to use the EZ Robot Speech Recognition instead of the Pandorabots? My EZ Speech Recognition file is huge and it would be nice to use it rather than having to develop a Pandorabot for the answers to my questions.

DJ Sures · Answer

You don't have to use pandoraBot - that's an example, as stated in the plugin description. Here's a link to the Bing plugin description: https://synthiam.com/redirect/legacy?table=plugin&id=179 Use IF conditions Explore blockly which is a good ui for learning programming with IF conditions. It's pretty easy and fun!

bhouston · Answer

Thanks DJ, I've been playing with it but still not luck. I was thinking that I should be able to use something like;
 ControlCommand("Speech Recognition", SetPhrase, $BingSpeech)

but I get an error due to the SetPhrase to replace;
 ControlCommand("PandoraBot", SetPhrase, $BingSpeech)

DJ Sures · Answer

You can find out about ControlCommand() by clicking on that word. It's highlighted in this response. All commands that are available for a control are displayed in the Cheat Sheet. If a command, is not displayed, it doesn't exist. The command SetPhrase does not exist for the Speech Recognition control. I can add it - because i see what you're attempting to do. However, be aware that the bing speech recognition will recognize any phrase, including a period at the end of the statement, or a question mark. So your existing speech recognition will not work without removing the period or question mark from the $bingspeech variable. I'll add the SetPhrase for ControlCommand() in the next ARC release. In the meantime, the IF condition's i'm referring to is coding. For example, this is what my video uses for the script in the bing speech recognition plugin...
 if ($BingSpeech = "What do you see?") ControlCommand("Microsoft Cognitive Vision", Detect) ELSEif ($BingSpeech = "Hello.") ControlCommand("Auto Position", AutoPositionAction, "Wave") ControlCommand("PandoraBot", SetPhrase, $BingSpeech) ELSEif ($BingSpeech = "Take a seat.") ControlCommand("Auto Position", AutoPositionAction, "Sit Down") ELSEif ($BingSpeech = "How am I feeling?") ControlCommand("Microsoft Cognitive Emotion", Detect) ELSEif ($BingSpeech = "How do I feel?") ControlCommand("Microsoft Cognitive Emotion", Detect) ELSEif ($BingSpeech = "Say hello.") ControlCommand("Auto Position", AutoPositionAction, "Sit Wave") ELSE ControlCommand("PandoraBot", SetPhrase, $BingSpeech) endif

bhouston · Answer

Hi DJ, I didn't explain what I was doing very well but I figured you would get it. I knew why the SetPhrase wasn't working. Thanks for the sample of the code it helps a lot. Looking forward to the next update.

Dave Schulpius · Answer

I'll add the SetPhrase for ControlCommand() in the next ARC release.

Has this been added yet?

DJ Sures · Answer

Yes. Read about the controlcommand so you can see what commands are available. You can see for yourself what commands are available for a control. I prefer the right click method rather than cheat sheet. But it's up to you.

Dave Schulpius · Answer

Yep, first thing I did was look at both of thouse methods looking for this new command. Theses options were listed under Speech Rec for both right click and cheat sheet:  Am I looking in the wrong place? confused

Dunning-Kruger · Answer

@Dave... Are you asking this for Bing speech plugin or the built in ARC speech rec? What you posted is for the built in speech rec control... EDIT never mind I just re-read part of the above thread...

Dave Schulpius · Answer

Thanks for stepping in RR. I just now found time to start playing with the Bing SR. Actually I'm playing with both the EZ Robot SR control and and Bing and trying to find the best way to get result I'm looking for. I'm not useing the Pandorabot control but rather I want to trigger sound files located on several EZB soundboards installed in ARC and other movement scripts I've already written. Kinda like the present SR control does I really don't know yet what I'm doing but am having some success and learning as I go. I was hoping to trigger scripts I've already written located in EZ's SR control with the Bing SR control. After reading this thread it sounded to me that DJ was going to add a SetPhrase option so this can be done. I may be misunderstanding that this can be done, not writing the script properly or not looking in the right place. I also want to "wake up" the Bing control so it will start listening and execute a sound file or script. I have succeed in doing this by adding a wake up word in the EZ Robot SR control like you would do with Alexis and adding the following command in it's script section:
 ControlCommand("Bing Speech Recognition", StartListening)

This actually works pretty good. So far here's what I've been able to succeed at: *Moved my old Speech Rec control out of my present project. *Installed a new SR control with only a wake word listed with it's script commanding Bing to start listening like mentioned above. *I've copied a few scripts out of the old SR control and wrote "If" statements with controlcommand commands pointing to these scripts or just to a sound file on one of my installed EZB sound boards. *When Bing hears and recognizes a phrase, it stores it in the variable I've got entered in the Bing control ($BingSpeech). *The "if" statement script I have in Bing control will then run. When it finds the phrase stored in the variable it will execute the ControlCommand below it. Here's a look at a sample from my script inside the Bing control that runs when Bing detects a phrase:
 IF ($BingSpeech = "Do you know where you are?") ControlCommand("Head Section", ScriptStart, "Radar 1/4 Sweep") Sleep(5000) ControlCommand("Soundboard v4", Track_18) ELSEIF ($BingSpeech = "Tell me a Joke") ControlCommand("Soundboard v4", Track_19) ELSEIF ($BingSpeech = "Two steps forward") ControlCommand("Soundboard v4", Track_20) ENDIF

To me this seems like above is the way to do what I want done. Is there a better way? At first I thought by having a way for Bing to use my existing SR scripts would be easier but I now I see that I may not be able to do that. Any thoughts? Thanks!

bhouston

Upgrade to ARC Pro