One more thing that I should mention... API.AI is constantly learning from how questions are asked. It uses AI learning methods to improve how it understands how questions or statements are asked. The more examples that you can provide in your intents, the more it will be able to learn initially, but even with only two items specified in the "user says" section for my robotRaiseRightArm Intent, it was able to easily understand the statements of "Hey Robot, I want you to raise your right arm 19 degrees" and "yo, raise your right arm 1 degree".

These statements are

"raise right arm @sys.number:value @unit_rotation:unit"

and

"robot raise right arm @sys.number:value @unit_rotation:unit" 

Okay, now, what it isn't It isn't google's search engine, although you could call the search engine from the results returned by api.ai.
It isn't a great chatbot. If you want to do that, I would recommend other options. It isn't something that won't require you to do more work to use the data returned. It isn't a complete AI that can just be dropped in and used. It isn't something that requires no learning and effort It isn't something that you can just let someone else develop things in it for you.

You will have to take the initiative to learn about it and make it customized for your robot. Not every robot has arms. Moving forward 5 feet is different for every type of robot with different types of sensors. Some wheeled robots have encoders, some don't.

It is a powerful tool for breaking apart speech and then formatting it in a way that you can use it. This allows many people to use the same robot with different speech styles and ways of saying something. It will help to prevent you from having to remember exact phrases that you have programmed to do things. It will make you a much better script writer if you use it.

Oh my! This is fabulous. I've been wanting to figure this out and see how it can be worked into my robot. I've wanted to base most of much of my robot control on voice command and the present MS based voice recognition really does have limits and problems for my needs. This looks like it's the answer to my dreams.

David, your such an asset and help to this forum. Personally, I really appreciate the time and expert knowledge you're giving. Thanks!

That is a great wrap up of api.ai, I will surely check the client you included... It's also a great starting point for everyone not yet familiar with the api.ai platform. You did a great job in pointing out what api ai is, and what it is not...I could not agree more on all of your points!

When it comes to returned parameters, we could also ask @DJSures to be so kind and include all of them to be exposed within ARC, the first version of the plugin was only returning speech and later on he also included the action parameter, I do agree that it would make the work in ARC a lot simpler if we would not have to break down the returned text and the variables are directly accessible!

If you would want to use api.ai as a seach engine, you could do this in ARC or using api.ai fulfillment...same goes for smalltalk! If eg a smalltalk action is returned you could pass the speech to your chatbot within ARC, I would also guess you could use api.ai fulfillment to handle this, but I did not look into this yet!

Great intro, lets start digging into this...its worth it!

https://synthiam.com/Community/EZCloud/RobotAppDetails.aspx?id=4795

This is an example ARC project that leverages the Bing Speech API, API.AI and Pandora Bot.

It could be cleaner, just wanting to put out something that could be used to start things off for people if they wanted to use it.

I will discuss this project and API.AI on this weeks Technology Today episode.

Very cool.

After installing your project I said "Robot move backwards" when the proper phrase should have been "Robot Reverse". It understood what I wanted and responded.

Very cool.

@CochranRobotics This is great! Thanks for sharing! It already involves a great deal of coding, and having the speech recognition already triggered and ready to go is a cool thing!

To see how you coded the response parse is great! I still did not get pandora bot to respond, but I guess I can figure that out somehow...your example should send the speech from Bing to PandoraBot if the returned action is blank right?

I think it would be kind of cool to have your api.ai client access token so the testing enviroment will be level for everyone...that should be save right?

I already deleted an EZ-Robot project that I posted because I read your thread on the Bing Speech key being included...but for the api.ai key this should not matter right?

Good starting point, I am really happy to have this, since you already worked with api.ai for quiet a bit, it is so cool seeing how you manage to piece it all together in ARC! Yaaaaay!

edit I was just not getting the point that the provided example was obviously to be run on the Rafiki Client which can be installed from the .zip file that was uploaded earlier in this thread, it provides some interesting approaches on api.ai robot control and can be tailored to your own needs...just stating this in case someone else might also be confused about it!

The new version of the plugin works well. I will post the example again this afternoon and describe what I am doing and where in a notepad control. I am away from the house for a bit but will get this out this afternoon hopefully.

Here is the jist of the example project

The init script just pauses the PandorBot control and the Bing Speech Recognition plugin, along with making sure that the Speech Recognition control is not paused.

In the Speech Recognition control there are a couple of different phrases that will unpause the Bing Speech Recognition control. They are "Hey Robot", "I want to ask you something" and "Hey Spock". Feel free to change these to whatever you need. They all run the same script which causes the Bing Speech Recognition plugin to start listening.

After the user says something, the Bing Speech Recognition plugin pauses the Speech Recognition control, calls ControlCommand("API.AI", Send, $BingSpeech) and then pauses the Bing Speech Recognition plugin. The converted text is returned to the $BingSpeech variable and then launches the API.AI plugin to send the $BingSpeech variable to API.AI.

The API.AI plugin starts the ResponseParse script which then decides what to do with the variables returned from API.AI. If it doesnt know what to do with the text, it sends the $BingSpeech variable to PandoraBot after unpausing Pandoria bot. After two seconds, Pandora Bot is paused again by the script.

If API.AI returns values that can be used, it calls the appropriate script to execute the actions necessary.

Okay, I will make sure that everything is working as intended this afternoon and publish it.

https://synthiam.com/Community/EZCloud/RobotAppDetails.aspx?id=4795

Example project here. You will have to supply your own API.AI key and Bing Speech key.

@David, glad to see you getting Back to the api, I just finish my robot and will be trying this example along with DJ's plugin soon.

Thanks for your work David.

@David or anyone who knows.
I'm trying to understand who this works, on David's API-AI example it has this in the script .

if(length($APIAction) > 0)
  if($APIAction = "cancel")
    say("Cancelling Action")
  endif
  if(SubString($APIAction,0,5) = "small")

I think I understand the >0 means if no text or nothing heard, right? just not sure what the 0,5 means.

Thanks,

@David, I know you have your hands full these days, but have you tried your example with the move forward using your API.ai example plug-in you created ? When I try it, I get "input.unknown" under the $Apiaction in the variable watch. Also it does not speak the return response to my speaker when I say "robot moveforward" but if I just asked a generic question then it speaks the response.

Any help would be most appreciated when you have time of course. Thank you, mike

@merne The ">" is the greater than comparison... what that line of code means is if $APIAction is not empty. Meaning variable contains data

 if(length($APIAction) > 0)

Substring command is looking for a specific set of characters in a longer string of characters like a sentance... In this case the word "small" starting at character 0 and reading for 5 characters (the length of the word "small")...

(SubString($APIAction,0,5) = "small")

All these commands are listed in ez script including examples on how to use them...

To use this example, you have to import the zip file into API.AI which tells API.AI what the Move Forward action is. Did you do this step?

In api.ai terms the zip file contains the actions that I used in the example.

I guess I'm confused here, when you say import the zip file, do you mean unzip it in the API.AI folder where the plug-in resides this is where I'm confused.

I re read your tread and still confused. Maybe I just don't get the stuff because I see Dave s. has it installed and working. So I must be missing something.

I have the zip file not sure where to unzip it to.

Thanks David,

in API.AI website (where you got your API key from) you would import the zip file into your api.ai project. This will give you the actions you need for this example to work.

@David, that makes sense I have imported it into the API.AI website. I think I'm still missing something I'm sorry I'm a visual person more than I am reading person.

I think I still have to do something I'm not sure, when I say hey robot robot move forward it's starts the response script that you made but it does not start the movie script.

I will hunt and peck like I said I'm a visual person I have a hard time reading and understanding some things especially this techie stuff but thanks again for pointing me to the right direction David I appreciate it !

OK, the zip file has been imported to the api.ai web site. IS there more stuff I need to do in api.ai web site? This is what I have in the api.ai example that David created. When I say robot moveforward it still gives an "input.unknown" in the $APIAction variable watch.

#code here to move the robot
#$unitinfo = APIGetParamByIndex(1) + " " + APIGetParamByIndex(0)
if($APIAction = "robot.moveForward")
  #place code here to move the robot forward the specified distance of units
  saywait("Moving forward " + APIGetParamByIndex(1) + " " + APIGetParamByIndex(0) )
  #add whatever code you want to put in here
  forward()
  Sleep(1000)
  Stop()
endif
if($APIAction = "robot.moveBackward")
  #place code here to move the robot forward the specified distance of units
  saywait("Moving backward " + APIGetParamByIndex(1) + " " + APIGetParamByIndex(0) )
  #add whatever code you want to put in here
  Reverse()
  Sleep(1000)
  Stop()
endif
ControlCommand("Bing Speech Recognition", PauseListening)
ControlCommand("Speech Recognition", PauseOff)

I add the forward() command in David's move script as you see above. When I say Robot moveforward it starts the script ResponseParse but it does not start the moverobot script.

Do I need to do something or add something on David's example code in the api.ai web site, for this below?

move forward @sys.number:value @sys.unit-length-name:unit

I was hoping Dave S. wound respond as I know David is very busy and has stated we need to do the leg work. I just need a start how ever small, as this is how I have learned what little I know by others examples. I am not asking for someone else to do my work, just asking for a visual example. I hope this makes sense.

Thanks in advance,

What happens when you say "move forward 5 feet"?

@David, OK that worked, boy I am dense I was looking at them parameters on the API.AI website and thought I had to change something there after your last post. It now makes sense thank you very much David. It is now working. Thank you very much for all your time and sharing your project with others David. This is very cool!

Thanks and cheers!

@RichardR, thanks Richard for your reply.

This Api.ai project looks very good for EZ-Robots having fully functional AI pretty quickly