Welcome to Synthiam!

The easiest way to program the most powerful robots. Use technologies by leading industry experts. ARC is a free-to-use robot programming software that makes servo automation, computer vision, autonomous navigation, and artificial intelligence easy.

Get Started
USA
Asked — Edited

Api.Ai Example To Learn A Little Bit From

I have been working in API.AI for a while and jumped back into it yesterday some. I want to describe what API.AI is and what it isn't so that there is a clearer picture of it. I am attaching a sample api.ai project that you can use to start building from if you would like to do so.

First, what is it? API.AI is great at taking speech and breaking it down to give you the intent behind the speech. For example, in the example, you can say something like "Hey Robot, I want you to raise your right arm 19 degrees". The returned text (what would be returned to EZ-Builder) is "[Movement] [RightArm] [up] [19] [deg]".
You can use a script to break down what the robot is being asked to do from the information above. The phrase "yo, raise your right arm 1 degree" would also work and return "[Movement] [RightArm] [up] [1] [deg]" for you to parse and use.

There are some chatbot features called "SmallTalk". This works like any other chatbot and takes questions or statements from the user and returns text that you want it to return. This is the simplest form of using API.AI and probably is the easiest part of it, but is also not the most useful.

There are some prebuilt agents. These agents each use their own API key. Because of this, I don't recommend using them because the plugin only allows one API key, and you will quickly run out of allowable uses. It is far better to build a single customized agent which contains everything that you want your robot to use.

The use of this tool is to break apart and parameterize language. This allows you to use completely different speech for specific commands in EZ-Builder. Currently, the plugin only sets 2 variables in EZ-Builder. This requires that you pass the parameters in API.AI back in the Text response field that would match the layout of what you want to parse.

In addition, you can add what are called integrations. This is where you could tie into many different things like Actions on Google, Microsoft Cortana, Amazon Alexa skills, telegram, skype, twitter, facebook messenger and many other one-click integrations. There are also SDK's for Android, IOS, Ruby, HTML5, JavaScript, Node.JS, Cordova, Unity, C#, C++, Xamarin, Python, PHP, Java, Botkit and Epson Moverio that allow you to write whatever you want with whatever language you want for the most part. These integrations allow you to run code instead of simply returning the meaning of what was stated back to EZ-Builder.

The example here doesn't use integrations, but is designed more to have the information sent back to ARC for you to do something with.

This is a very scaled down version of the Rafiki client that I had been working on.
Rafiki-Client.zip
You can take this and import or restore it into your api.ai project.

I hope this example helps people see what API.AI can be used for and better understand where it fits in your robot brain.


ARC Pro

Upgrade to ARC Pro

Synthiam ARC Pro is a cool new tool that will help unleash your creativity with programming robots in just seconds!

AI Support Bot
Related Content
Synthiam
Based on your post activity, we found some content that may be interesting to you. Explore these other tutorials and community conversations.
#1  
One more thing that I should mention... API.AI is constantly learning from how questions are asked. It uses AI learning methods to improve how it understands how questions or statements are asked. The more examples that you can provide in your intents, the more it will be able to learn initially, but even with only two items specified in the "user says" section for my robotRaiseRightArm Intent, it was able to easily understand the statements of
"Hey Robot, I want you to raise your right arm 19 degrees" and "yo, raise your right arm 1 degree".

These statements are

Code:

"raise right arm @sys.number:value @unit_rotation:unit"

and

Code:

"robot raise right arm @sys.number:value @unit_rotation:unit"
#2  
Okay, now, what it isn't
It isn't google's search engine, although you could call the search engine from the results returned by api.ai.
It isn't a great chatbot. If you want to do that, I would recommend other options.
It isn't something that won't require you to do more work to use the data returned.
It isn't a complete AI that can just be dropped in and used.
It isn't something that requires no learning and effort
It isn't something that you can just let someone else develop things in it for you.

You will have to take the initiative to learn about it and make it customized for your robot. Not every robot has arms. Moving forward 5 feet is different for every type of robot with different types of sensors. Some wheeled robots have encoders, some don't.

It is a powerful tool for breaking apart speech and then formatting it in a way that you can use it. This allows many people to use the same robot with different speech styles and ways of saying something. It will help to prevent you from having to remember exact phrases that you have programmed to do things. It will make you a much better script writer if you use it.
#3  
Oh my! This is fabulous. I've been wanting to figure this out and see how it can be worked into my robot. I've wanted to base most of much of my robot control on voice command and the present MS based voice recognition really does have limits and problems for my needs. This looks like it's the answer to my dreams.

David, your such an asset and help to this forum. Personally, I really appreciate the time and expert knowledge you're giving. Thanks!:)
#4  
That is a great wrap up of api.ai, I will surely check the client you included...
It's also a great starting point for everyone not yet familiar with the api.ai platform.
You did a great job in pointing out what api ai is, and what it is not...I could not agree more on all of your points!

When it comes to returned parameters, we could also ask @DJSures to be so kind and include all of them to be exposed within ARC, the first version of the plugin was only returning speech and later on he also included the action parameter, I do agree that it would make the work in ARC a lot simpler if we would not have to break down the returned text and the variables are directly accessible!

If you would want to use api.ai as a seach engine, you could do this in ARC or using api.ai fulfillment...same goes for smalltalk! If eg a smalltalk action is returned you could pass the speech to your chatbot within ARC, I would also guess you could use api.ai fulfillment to handle this, but I did not look into this yet!

Great intro, lets start digging into this...its worth it!:)
#6  
Very cool.

After installing your project I said "Robot move backwards" when the proper phrase should have been "Robot Reverse". It understood what I wanted and responded.

Very cool. :D
#7  
@CochranRobotics This is great! Thanks for sharing!:D:D:D
It already involves a great deal of coding, and having the speech recognition already triggered and ready to go is a cool thing!

To see how you coded the response parse is great! I still did not get pandora bot to respond, but I guess I can figure that out somehow...your example should send the speech from Bing to PandoraBot if the returned action is blank right?

I think it would be kind of cool to have your api.ai client access token so the testing enviroment will be level for everyone...that should be save right?:)

I already deleted an EZ-Robot project that I posted because I read your thread on the Bing Speech key being included...but for the api.ai key this should not matter right?

Good starting point, I am really happy to have this, since you already worked with api.ai for quiet a bit, it is so cool seeing how you manage to piece it all together in ARC! Yaaaaay!:D

*edit*
I was just not getting the point that the provided example was obviously to be run on the Rafiki Client which can be installed from the .zip file that was uploaded earlier in this thread, it provides some interesting approaches on api.ai robot control and can be tailored to your own needs...just stating this in case someone else might also be confused about it!:)
#8  
The new version of the plugin works well. I will post the example again this afternoon and describe what I am doing and where in a notepad control. I am away from the house for a bit but will get this out this afternoon hopefully.

Here is the jist of the example project

The init script just pauses the PandorBot control and the Bing Speech Recognition plugin, along with making sure that the Speech Recognition control is not paused.

In the Speech Recognition control there are a couple of different phrases that will unpause the Bing Speech Recognition control. They are "Hey Robot", "I want to ask you something" and "Hey Spock". Feel free to change these to whatever you need. They all run the same script which causes the Bing Speech Recognition plugin to start listening.

After the user says something, the Bing Speech Recognition plugin pauses the Speech Recognition control, calls ControlCommand("API.AI", Send, $BingSpeech) and then pauses the Bing Speech Recognition plugin. The converted text is returned to the $BingSpeech variable and then launches the API.AI plugin to send the $BingSpeech variable to API.AI.

The API.AI plugin starts the ResponseParse script which then decides what to do with the variables returned from API.AI. If it doesnt know what to do with the text, it sends the $BingSpeech variable to PandoraBot after unpausing Pandoria bot. After two seconds, Pandora Bot is paused again by the script.

If API.AI returns values that can be used, it calls the appropriate script to execute the actions necessary.

Okay, I will make sure that everything is working as intended this afternoon and publish it.
#10  
@David, glad to see you getting Back to the api, I just finish my robot and will be trying this example along with DJ's plugin soon.

Thanks for your work David.
#11  
@David or anyone who knows.
I'm trying to understand who this works, on David's API-AI example it has this in the script .

Code:


if(length($APIAction) > 0)
  if($APIAction = "cancel")
    say("Cancelling Action")
  endif
  if(SubString($APIAction,0,5) = "small")


I think I understand the >0 means if no text or nothing heard, right?
just not sure what the 0,5 means.

Thanks,
#12  
@David, I know you have your hands full these days, but have you tried your example with the move forward using your API.ai example plug-in you created ? When I try it, I get "input.unknown" under the $Apiaction in the variable watch. Also it does not speak the return response to my speaker when I say "robot moveforward" but if I just asked a generic question then it speaks the response.

Any help would be most appreciated when you have time of course.
Thank you, mike
#13  
@merne The ">" is the greater than comparison... what that line of code means is if $APIAction is not empty. Meaning variable contains data

Code:

 if(length($APIAction) > 0)


Substring command is looking for a specific set of characters in a longer string of characters like a sentance... In this case the word "small" starting at character 0 and reading for 5 characters (the length of the word "small")...

Code:

(SubString($APIAction,0,5) = "small")


All these commands are listed in ez script including examples on how to use them...
#14  
To use this example, you have to import the zip file into API.AI which tells API.AI what the Move Forward action is. Did you do this step?

In api.ai terms the zip file contains the actions that I used in the example.
#15  
I guess I'm confused here, when you say import the zip file, do you mean unzip it in the API.AI folder where the plug-in resides this is where I'm confused.

I re read your tread and still confused. Maybe I just don't get the stuff because I see Dave s. has it installed and working. So I must be missing something.

I have the zip file not sure where to unzip it to.

Thanks David,
#16  
in API.AI website (where you got your API key from) you would import the zip file into your api.ai project. This will give you the actions you need for this example to work.
#17  
@David, that makes sense I have imported it into the API.AI website. I think I'm still missing something I'm sorry I'm a visual person more than I am reading person.

I think I still have to do something I'm not sure, when I say hey robot robot move forward it's starts the response script that you made but it does not start the movie script.

I will hunt and peck like I said I'm a visual person I have a hard time reading and understanding some things especially this techie stuff but thanks again for pointing me to the right direction David I appreciate it !
#18  
OK, the zip file has been imported to the api.ai web site. IS there more stuff I need to do in api.ai web site?
This is what I have in the api.ai example that David created. When I say robot moveforward it still gives an "input.unknown" in the $APIAction variable watch.

Code:


#code here to move the robot
#$unitinfo = APIGetParamByIndex(1) + " " + APIGetParamByIndex(0)
if($APIAction = "robot.moveForward")
  #place code here to move the robot forward the specified distance of units
  saywait("Moving forward " + APIGetParamByIndex(1) + " " + APIGetParamByIndex(0) )
  #add whatever code you want to put in here
  forward()
  Sleep(1000)
  Stop()
endif
if($APIAction = "robot.moveBackward")
  #place code here to move the robot forward the specified distance of units
  saywait("Moving backward " + APIGetParamByIndex(1) + " " + APIGetParamByIndex(0) )
  #add whatever code you want to put in here
  Reverse()
  Sleep(1000)
  Stop()
endif
ControlCommand("Bing Speech Recognition", PauseListening)
ControlCommand("Speech Recognition", PauseOff)


I add the forward() command in David's move script as you see above. When I say Robot moveforward it starts the script ResponseParse but it does not start the moverobot script.

Do I need to do something or add something on David's example code in the api.ai web site, for this below?

Code:


move forward @sys.number:value @sys.unit-length-name:unit


I was hoping Dave S. wound respond as I know David is very busy and has stated we need to do the leg work. I just need a start how ever small, as this is how I have learned what little I know by others examples. I am not asking for someone else to do my work, just asking for a visual example. I hope this makes sense.

Thanks in advance,
#20  
@David, OK that worked, boy I am dense;)
I was looking at them parameters on the API.AI website and thought I had to change something there after your last post. It now makes sense thank you very much David. It is now working. Thank you very much for all your time and sharing your project with others David. This is very cool!

Thanks and cheers!

@RichardR, thanks Richard for your reply.
Australia
#21  
This Api.ai project looks very good for EZ-Robots having fully functional AI pretty quickly :)
Login to post a comment