United Kingdom
Asked — Edited

Cortana Speech Recognition Integration With ARC

We have managed to integrate Cortana unlimited (same as dictation mode) speech recognition with ARC which now means we can input any speech into ARC (for no cost unlike Dragon). we are using the HTTP custom server as can be seen on the screendump below.

User-inserted image

We added a Pandorabot to ARC (to use it with unlimited speech) but it always seems to use it's own (default MS) speech recognition and we can't see a way to send our speech string into the Pandorabot.

Can DJ or anyone advise how we may do this - running Pandorabot with reliable/accurate unlimited speech recognition would be very neat!

Thanks in advance for any help.

Tony


ARC Pro

Upgrade to ARC Pro

Synthiam ARC Pro is a new tool that will help unleash your creativity with programming robots in just seconds!

#1  

Exciting! ! I hope someone can come up with something m!

PRO
Synthiam
#2  

There's a ControlCommand to send text. ControlCommand can be viewed for each control in the cheat sheet.

What does it do? Will you be making this into a plugin for easier distribution and integration?

PRO
United Kingdom
#3  

DJ, at the moment we are just trying to see if we can use Cortana as a speech recognition input for ARC giving us a diction type mode and not be stuck with grammar (limited vocabulary) mode. My plan is to integrate with Pandorabot where we can ask any questions not pre-defined ones like grammar mode forces. I have been playing with Cortana speech recognition for a few days now and get around 99% accuracy it is surprisingly good!

Can you give me a bit more info (code snippet etc) on the send text command control required. As always thanks for your help here.

Tony

PRO
United Kingdom
#5  

DJ, I think you are possibly mistaken, the Cortana speech recognition is cloud based and not PC based and I believe is a much better SR engine - my reasoning is detailed below.

"Cortana’s speech recognition is actually a cloud-based system, where blocks of speech are submitted to the cloud for translation"

The above is referenced here

http://www.develop-online.net/tools-and-tech/how-windows-10-and-cortana-are-bringing-speech-recognition-to-games/0215391

Cortana also passes speech through a NLP (natural language processor) filter which obviously would improve SR engine output.

"The natural language processing capabilities of Cortana are derived from Tellme Networks (bought by Microsoft in 2007) " from Wikipedia

Cortana has to have a better SR engine as

Talking to the Pandorabot via the ARC (PC based SR) I get about 80% accuracy and it also hears itself which causes false recognitions - if I disconnect from the net SR continues to work proving that it is (in my opinion the not very good at dictation) internal SR based on a derivative of the Microsoft SR engine 6.1.

Talking to Cortana through my app yields 99% accuracy (similar to accuracy from Dragon) - if I disconnect from the net SR stops working proving that its cloud based.

My Cortana based SR also waits for its name which is important to stop false recognitions when interacting with the Pandorabot.

I may be wrong here, but I then cannot explain the huge difference in performance that I am seeing?

Tony

#6  

Yes, cortana is 100% cloud based and runs through a service on the computer that handles this. The services is quite a pain to get turned off if you don't want it running and consuming resources on the computer. The service is quite bloated but cortana does work fairly well without training. It is also free to use if you upgraded to windows 10. If you didn't, this upgrade to windows 10 is no longer available for free. Also, if you have an older mac running bootcamp, windows 10 isn't an option.

PRO
Synthiam
#7  

That's what I originally thought, but some Microsoft documentation had led me astray sometime ago. This page: https://msdn.microsoft.com/cortana/getstarted

Says this...

Quote:

Windows speech

Windows speech is a set of UWP APIs that enable both speech recognition and speech synthesis across multiple languages on all Windows-10 based devices, including IoT hardware, phones, tablets, and PCs.

Cortana on Windows uses these speech APIs.

Perhaps what they are failing to say correctly is Cortana uses the speech API for synthesis, not recognition.

Lastly, if you want pandora bot to be disabled from listening to voice commands, simply pause the control with the checkbox. View available control commands using the Cheat Sheet as previously stated.

#8  

Interesting that Cortana uses TellMe. I thought Microsoft had sold them. Maybe just sold the commercial IVR business and kept the technology. I'll need to do some research (in my last job I did some work with M$/TellMe on a partnership that fell apart shortly after I was laid off.

Alan