Voice Variable in VR

Question

Hi all

Is there a way to fire a voice command... like say Hi, then the script will say what is your name... then you say your name, and the VR holds that variable (e.g. your name) and the EZ says the variable back?

Proteus · Answer

Yes, its very easy. Try it.

Dave Schulpius · Answer

A couple questions first, To clarify, you want the robot to repeat back the name spoken to him. The name could be any name?  Are you using Bing VR? I haven't thought this through yet but when you speak a work (or in your case a name) to Bing is will return what it hears to you in text form. You will see what is returned in the control window. Perhaps there is a way to use that returned text to make the robot say what is returned? Perhaps by some some how capturing Bing's returned text and then using the Say( text to speech ) command? Speaks the text that is specified within the brackets out of the PC Sound Card in the background. This command does not block, the script will continue to execute. Example: Say("Hello, I am a robot") I think I see why you are asking about capturing the returned name into a variable. You could use that variable to use the Say command to repeat it.

TerryBau · Answer

Hey Dave... so I am grabbing API data, it shows different data per a variable (in this instance a name)... that name is tacked onto the json string... so I would like B9 to 1) Say what is your name (that is easy lol) 2) EZ wait (that is easy) for reply 3) you speak your name could honestly be any word in this example 4) that spoken word is stored into a variable 5) I append that variable to my json string I already know how to have EZ wait for a verbal command (e.g. like a yes no, or a multiple name thing) and then based on the input (e.g. if response = no) do something...

TerryBau · Answer

Ahhh maybe i will look at the Bing VR, that may do what I want, i was using the standard VR from EZ

DJ Sures · Answer

You’re on the right track with bing speech recognition. If you’re having conversation, also look into the open ai robot skill  but I do want to clarify.  VR = virtual reality  Speech Recognition = speech recognition  also, there is something called Voice Recognition sort of. It’s the process of matching sound to a voice. So the robot would know who’s speaking. That technology has mostly renamed to Speaker Recognition over the years. Not sure why lol so if you do say voice recognition, you’re talking about identifying the voice and that’s very different than speech recognition.  Try Bing Speech Recognition robot skill. It can detect anything.

Dave Schulpius · Answer

Woops, yes. VR. Voice. Sorry. Slip of the mind/keystroke.

Dave Schulpius · Answer

You're into an area I really don't know about.  I think DJ is pointing to this skill: https://synthiam.com/Support/Skills/Artificial-Intelligence/OpenAI-Chatbot?id=20207

DJ Sures · Answer

Ya that open ai works pretty good.  We’ve been circling around the speaker recognition from azure for a while. Not sure an eta though

TerryBau · Answer

thanks all for your help I found what i was looking for based on this thread and another thread. Not looking for a full blown convo as my robot (as Dave knows) has limited phrases, so I wanted to run a call word (like hey siri) in this case it would be Hey B9... then listen to my voice input and store that voice input, then based on what I said, run a script or sound... i got it working, this is what i pieced together, obv. just testing based on two simple words, but I will expand greatly on this... this is using Bing saywait("How can I help you") sleep(1000) controlCommand("Bing Speech Recognition", "StartListening") controlCommand("Bing Speech Recognition", "UnpauseListening") sleep(3000) controlCommand("Bing Speech Recognition", "PauseListening") controlCommand("Bing Speech Recognition", "StopListening") #sleep(2000) print($BingSpeech) sleep(1000) if ($BingSpeech = "Tom") ControlCommand("Convo", "Track_4") print("track 4") Elseif ($BingSpeech = "Terry") ControlCommand("Convo", "Track_3") print("track 3") endif

Proteus · Answer

Nice work! You got it!

DJ Sures · Answer

Because you know what to expect, it might be easier to use the WaitForSpeech command. All you do is pass the parameters that you want to wait for, and you'll get the response. Quote: WaitForSpeech( timeOut Seconds, phrases ) Pauses and waits for one of the specified phrases to be spoken. Returns the phrase that was spoken in lowercase. Will return "timeout" if no word is detected in the specified timeout length. Example: WaitForSpeech(30, "Yes", "No") Example: $value = WaitForSpeech(30, "Yes", "No")  Here's an example from your code. I added an ELSE so that the robot would speak it didn't understand if the timeout occurred.
 $resp = WaitForSpeech( 10, "Tom", "Terry") if ($resp = "Tom") ControlCommand("Convo", "Track_4") print("track 4") ELSEif ($resp = "Terry") ControlCommand("Convo", "Track_3") print("track 3") else Say("I didn't understand you") endif

Dave Schulpius · Answer

Outstanding Terry! Nicely done. I know how exciting this is.  I love having a wake word. I found early on that if I let Bing just run and listen all the time for phrases I wrote in the control I would get a lot of false positives. I also had trouble with it understanding me if there was background chatter from other people. Another thing I didn't like about letting a SR run and listen to everything being said in the room and sending it off to God knows where or who. I solved this by having two SR's installed. The Speech Recognition Skill and the Bing Skill. *Like you start the project with Bing paused from listening but have the Speech Recognition skill listening and listening for only word, "Robot". *Once the SR skill hears Robot it will run a script I wrote in and unpause Bing. It will also start the belly lights in a Scanner sequence for about 5 seconds. After 5 seconds that script will pause Bing from listening again. *During those 5 seconds Bing will be listening for a phrase and run a script depending on if it understands or not.  It works really well and I love it even in noisy rooms. However I'm always trying and looking for better ways. I'm actively working on my B9 now and will be looking at your and others methods for using a Wake Word.  When you have time it would be great to see a vid of your method in action. I saw your Stock Price video and that was really amazing.  Here's a vid of my wake word method in action I took about 5 months ago.

TerryBau · Answer

thanks Dave... i am using VAD (in the Bing Voice), but yes, he was picking up too much noise... i tried throwing in a PauseListening, but then realized i had to turn off my other EZ Voice module (lol) But here is a snippet of what I am doing using Bing and Aimlbot (if we had deep fake audio, we can take the else statement output the Aimlbot words into B9's voice... I cropped out all the other ElseIf to make the code easier to see... so far he is working well about 90% of the time, does our vid show how to code the wake word, that was my next section controlCommand("AimlBot", "SetPhrase", $BingSpeech) controlCommand("B9 Voice", "Stop") controlCommand("Bing Speech Recognition", "PauseListening") sleep(2000) ControlCommand("Scripts", "ScriptStart", "RadarMoveL") if ($BingSpeech = "How are you") # I am swell ControlCommand("BingChat", "Track_0") Elseif ($BingSpeech = "Tell me a joke") ControlCommand("BingChat", "Track_13") sleep(15000) Else sleep(6000) ControlCommand("BingChat", "Track_10") sleep(3000) # SayEZB($BotResponse) # sleep(5000) endif controlCommand("Bing Speech Recognition", "UnpauseListening") controlCommand("Bing Speech Recognition", "StartListening")

TerryBau · Answer

Dave, thanks for the kind words on the Stock Quote, i looked at your date and weather, modified those, then tweaked it into a stock API

TerryBau · Answer

Dave, i would also like to understand how you ran the motors (uart i think) into the EZB, mine are just on a random sequence on an Adruino, but love how you did them (e.g. the ears)

Dave Schulpius · Answer

Hi Terry, I started using H-Bridges to control motors back and forth. However I had limited position and speed control and no feedback of where the motors were. I was able to set speeds with the PWM skill. I soon realized I needed better precision and control over most of my motors. I started using Sabertooth Motor controllers with a Kangaroo daughterboard attached. This gave me precise position and speed control. The Kangaroo can use either a pot or an encoder for feedback. I found that for my needs an encoder was the way to go.  I started switching my motors over to ones that already had encoders attached to the motor shaft. ServoCity has a great selection. I controlled the Sabertooth/Kangaroo by sending SimpleSerial commands through the Uart ports of an EZB from ARC.  I do have a couple small DC motors (like the ears sensors) that I run directly off the EZB's digital pins. I simply turn on and off the signal pin to run the motor. A PWM skill in ARC lets me adjust the speed of these little motors.  I hope this helps.

TerryBau · Answer

thanks Dave, good to know I can wire my ear motors directly to the EZB, I may do the same. I saw the write up on the Sabre/Kang so I may go that route on the torso... I also finally (posted in our group) the call word and bing work... it's pretty cool and stable...

DJ Sures · Answer

Eak careful about connecting motors directly to any ezb. I wouldn’t recommend it If you need small form factor, consider something small and affordable like this: 3pcs 2-Way DC Motor Driver Module 1.5A 2V-10V PWM Speed Regulation Dual H-Bridge Replace L298N https://www.amazon.ca/dp/B087X9KBC7/ref=cm_sw_r_cp_api_glt_i_Y5X4HFVSD064EJ5ARC47?_encoding=UTF8&psc=1 they are cheap for 3 and very small little units for small motors

TerryBau · Answer

i have a few of those motor drivers, so the IN1 or 2 or 3, just goes to one of the pins?

DJ Sures · Answer

Oh nice - yeah the IN connectors go to what ever EZB pins you have available. Then, you use the Dual HBridge robot skill. Do not use the movement panel version. Use the one from the HBridge tab.  I'll make a quick change to that robot skill to control the two channels separately. Right now they're setup to control the motors as movement. In the meantime, you don't need to use that robot skill and instead can control them like this with code Assume the they're connected like such... IN1 is D1 IN2 is D2 IN3 is D3 IN4 is D4 Make one motor turn one direction in EZScript. I noticed youre using ezscript...
 Set(d1, on) Set(d2, off)

And the other direction...
 Set(d1, off) Set(d2, on)

And stopping...
 Set(d1, off) Set(d2, off)

And the other channel one direction
 Set(d3, on) Set(d4, off)

And the other channel on other direction
 Set(d3, off) Set(d4, on)

And the other channel stop
 Set(d3, off) Set(d4, off)

So the IN1 and IN2 is for the first channel (first motor), and the IN3, IN4 is for the second channel (second motor). The direction of the polarity determines the direction of the motor.

TerryBau · Answer

awesome, so one last dumb question you mention Motor 1 IN1 = D1 IN2 = D2 so the D1 and D2, etc.... or whatever ones i use that are open, have a black, pos, and signal.... which pin from IN1 to D1 do I use, same for D2? thanks for such great help

DJ Sures · Answer

Oh yeah use the signal pin. Each IN will use a different signal pin from each port. I made a quick change to ARC that will let you specify the direction. I'll release it tonight for you. You can use ControlCommand to specify the direction of each motor. See this screenshot i just took with the newest version that you'll have tonight.

TerryBau · Answer

wow super helpful!

Dave Schulpius · Answer

Humm, DJ is completely correct (as usual. LOL) about motors not going directly to the EZB . DC motors tend to send a lot of voltage back through the circuit when they stop. Reading back through my post I see I did give you incomplete and bad advice. Thanks @DJ. Still, For many years I have had my ear sensor's little micro gear motors connected to my EZB for power and control. However I didn't mention that I have a voltage regulator in between the EZB and the little motor but I don't think that would block any fly back voltage. I've never had any issues in the time it's been wired like this. I'm in process of upgrading my robot so I'll probably change how I power them. I have a number of little H-Bridges in my extra parts box. Also, I don't like powering anything through EZB's so I'll also probably bypass it and wire directly to my power source. In my current configuration it was just so much more covenant to go to the EZB.  I read a thread a while back about reversing little geared micro motors that B9 builders use for the ear sensors. One or two of the knowledgeable club members were doing tests with speed and direction. If I remember correctly they came to the conclusion that sudden reversal of these little motors tended to destroy them. My ear motors are very expensive so I take no chances and never run them in the opposite direction. That's probably why I never installed an H-Bridge. I'm going to contact David Mc. and relearn what he found in his micro gear motor tests. I'll post here what he says.

TerryBau · Answer

for my ears and crown, they are running an Adruino Uno, delay and random time sequence... works really well.. would be cool to be able to have EZ talk to Adruino and Maestro (as I love that sequence animation you can do - which my wrists and claws are running from, again all random time sequences..) would be cool to control those boards or make calls from the EZ so I had more control.. but i did what I knew how to do at the time.. but as my B9 is maturing i want more control of things lol thanks all, been spending much more time with my B9 and the EZ Robot and digging the call word from SR over to Bing to make him respond and talk back and head movement...

DJ Sures · Answer

Terry - We had to delay updating ARC until tonight. I didn't realize the release was held due to a Microsoft .Net bug that we had discovered. We isolated the issue and reported it to Microsoft. There's a work-around for the time being in the new update. More will be included in the release notes.

Dave Schulpius · Answer

Hey Terry. I talked to David Mc about the small geared motor question I had. He said the issue of a geared motor failing would only come into play if the motor was reversed quickly without letting come to a full stop. These small motors run at such a high RPM (1600 RMP for example). Then they add gears to it's motor shaft to drastically slow down the final output shaft (35 RPM for example). A sudden change of direction places an amazing amount of pressure on those little gears.  As far as my other statment about directly about plugging a motor directly into an EZB. I was completely wrong (again). I finally visually looked at my circuit. It does not plus directly to the EZB. It runs through a TIP transistor switch I built and then a voltage regulator. The transistor switch is opened and closed by a Digital Signal pin on the USB. That turns on and off the ears. I control the speed with PWM set in ARC. I really need to check over things I built years ago before I go shooting off my mouth about to do things. LOL. Sorry my friend.

DJ Sures · Answer

The update with the memory fix and your new ControlCommand for the hbridge is here: https://synthiam.com/Products/ARC/Releases/ARC-2022-02-28-00-Runtime-Early-Access-21066

EzAng · Answer

Terry,  I combined you script with DJ's  -  works good ControlCommand("Bing Speech Recognition", "StartListening") $resp = WaitForSpeech(5, "Tom", "Bob") if ($resp = "Tom" | $BingSpeech = "Tom") ControlCommand("Soundboard", "Track_0") print("track 0") sleep(3000) ControlCommand("Bing Speech Recognition", "StartListening") ELSEif ($resp = "Bob" | $BingSpeech = "Bob") ControlCommand("Soundboard", "Track_1") print("track 1") ELSE ControlCommand("Bing Speech Recognition", "StopListening") Say("I didn't understand you") endif

DJ Sures · Answer

EZAng, you should not be instruction bing speech recognition to start listening with my example. With WaitForSpeech you don't have to use the bing speech recognition at all. It should be...
 $resp = WaitForSpeech(5, "Tom", "Bob") if ($resp = "Tom") ControlCommand("Soundboard", "Track_0") print("track 0") sleep(3000) ELSEif ($resp = "Bob") ControlCommand("Soundboard", "Track_1") print("track 1") ELSE Say("I didn't understand you") endif

EzAng · Answer

Ok I will try that, thanks, works well, no bing speech,  also thanks for the new update And soundboard and speech recognition work the best for me, don't need Bing

TerryBau · Answer

EzAng Mine is a tad different... using SP I do this: ControlCommand("Speech Recognition", "PauseOn") ControlCommand("Fun", "Track_3") sleep(1350) ControlCommand("BingChat", "Track_17") sleep(2000) ControlCommand("Bing Speech Recognition", "StartListening") sleep(3000) ControlCommand("Bing Speech Recognition", "StopListening") Then using Bing after I say my call word (it opens up Bing and listens and passes my next phrase to Bing) like the below (and I pass what I said as a variable)... I am also using AimlBot for I want to get some AI logic to maybe use in the future... but my robot has a distinct voice, so I am at the moment bound to only the 200 or so phrases i have...  it works perfectly 99% of the time and still needs some tweaking... as this is new to me controlCommand("AimlBot", "SetPhrase", $BingSpeech) sleep(2000) #controlCommand("Bing Speech Recognition", "PauseListening") ControlCommand("Bing Speech Recognition", "StopListening") ControlCommand("Scripts", "ScriptStart", "RadarMoveL") sleep(2000) if ($BingSpeech = "How are you") # I am swell ControlCommand("BingChat", "Track_0") sleep(3000) ControlCommand("BingChat", "Track_1") sleep(5000) Elseif ($BingSpeech = "Stock price") # run stock script ControlCommand("Scripts", "ScriptStart", "Stock") sleep(25000) Elseif ($BingSpeech = "What is the weather") # run weather script ControlCommand("Scripts", "ScriptStart", "weather") sleep(25000) Elseif ($BingSpeech = "Who are you") # play fun sounds ControlCommand("BingChat", "Track_2") sleep(5000) ControlCommand("BingChat", "Track_3") sleep(5000) Elseif ($BingSpeech = "Your mission") # play fun sounds ControlCommand("BingChat", "Track_4") sleep(10000) ControlCommand("BingChat", "Track_5") sleep(5000) Elseif ($BingSpeech = "What day is it") # run date script ControlCommand("Scripts", "ScriptStart", "Date") sleep(25000) Elseif ($BingSpeech = "How do I look") # play fun sounds ControlCommand("BingChat", "Track_6") sleep(5000) Elseif ($BingSpeech = "Define your first law") ControlCommand("Laws", "Track_0") sleep(10000) Elseif ($BingSpeech = "Define your second law") ControlCommand("Laws", "Track_1") sleep(10000) Elseif ($BingSpeech = "Define your third law") ControlCommand("Laws", "Track_2") sleep(10000) Elseif ($BingSpeech = "Do you understand my wife") ControlCommand("BingChat", "Track_8") sleep(10000) Elseif ($BingSpeech = "Will you help me clean up") ControlCommand("BingChat", "Track_9") sleep(15000) Elseif ($BingSpeech = "Scan area") ControlCommand("Scripts", "ScriptStart", "RadarMoveL") sleep(6000) ControlCommand("BingChat", "Track_15") sleep(5000) ControlCommand("BingChat", "Track_14") sleep(5000) ControlCommand("BingChat", "Track_16") sleep(5000) Elseif ($BingSpeech = "Good Morning") ControlCommand("BingChat", "Track_11") sleep(4000) Elseif ($BingSpeech = "Goodmorning") ControlCommand("BingChat", "Track_11") sleep(4000) Elseif ($BingSpeech = "Good Night") ControlCommand("BingChat", "Track_12") sleep(6000) Elseif ($BingSpeech = "Goodnight") ControlCommand("BingChat", "Track_12") sleep(6000) Elseif ($BingSpeech = "Tell me a joke") ControlCommand("BingChat", "Track_13") sleep(15000) Elseif ($BingSpeech = "Doctor Smith") ControlCommand("BingChat", "Track_18") sleep(15000) Elseif ($BingSpeech = "Compliment my wife") ControlCommand("BingChat", "Track_20") sleep(3000) ControlCommand("BingChat", "Track_19") sleep(5000) Elseif ($BingSpeech = "Tell me if we are in danger") $Cresponse = GetRandomUnique(1,4) print($Cresponse) if ($Cresponse = 1) # no ControlCommand("BingChat", "Track_21") sleep(1000) Elseif ($Cresponse = 2) # mother in law ControlCommand("BingChat", "Track_22") sleep(5000) Elseif ($Cresponse = 3) # danger will ControlCommand("Scripts", "ScriptStart", "ArmsOut") ControlCommand("BingChat", "Track_24") sleep(3000) ControlCommand("BingChat", "Track_23") sleep(6000) ControlCommand("Scripts", "ScriptStart", "ArmsIn") Else ControlCommand("BingChat", "Track_25") sleep(6000) ControlCommand("BingChat", "Track_26") sleep(5000) endif Else sleep(6000) ControlCommand("BingChat", "Track_10") sleep(3000) #SayEZB($BotResponse) # sleep(5000) endif #controlCommand("Bing Speech Recognition", "UnpauseListening") #controlCommand("Bing Speech Recognition", "StartListening") ControlCommand("Speech Recognition", "PauseOff")

EzAng · Answer

Look good

I always use AIMlL Bot and Bing

I modified the XML code and added what I want it to say - works great

As for voices, audacity can make a robot voice, and with a little code here get many hidden Microsoft voices

TerryBau

Voice Variable In VR

Upgrade to ARC Pro