Hey guys, this morning I realized it will take some time until I will be putting up a YouTube clip, since I am just too slow in getting this type of stuff done...I am also still kind of in the middle even getting my robot ready for the testing environment!

So I thought to pace things up a little I will just upload a working version of the script to my serverspace for you to test it out... it is just a simple bot with the native smalltalk agent, no training whatsoever!

The only thing I did to bridge it with ARC is to add a httpGet(theUrl) fuction after the respond(spokenResponse) funtion which sends the variable that is returned by the query within the action parameter, in this case it would be smalltalk.greetings or smalltalk.greetings.hello etc , over to the server to trigger a command for the robot!

What I am still trying to accomplish would be getting data into api.ai...eg to store data lets say via IFTT MakerWebhooks to the server in a correct format so the fullfiment in api.ai will be able to read the data correctly. Any information on this would be highly appreciared!

I uploaded the code for testing, and could also leave it there if we would decide to collaborate in building and agent, or just as a testing enviroment!

http://www.downtown-tattoo.de/robotics/test123.html

edit same problem on the link...please remove https and put http to your browsers address bar...

Here you go: https://synthiam.com/redirect/legacy?table=plugin&id=193

@DJSures Man you are the best!

I will check the plugin right now, this is awesome!

The plugin works great! I would love to do some copy and paste hacking on it if you would mind to share the Visual Studio Project...

I tried to follow the instructions on how to create a plugin but had trouble with the .net version and with compiling the project, but I guess if I had a working file I could try to get into how it has been set to work!

What I would like to get done are basically two things, one would be to send some data to api.ai from within ARC...the other thing is that it would be great to also have the other returned variables exposed in ARC rather than just the speech variable! (this should not be too much of a problem I guess?)

If for instance the actions variable would be exposed, we could use it to drive our robots actions which are be tied to the variables value...eg if actions would return smalltalk.greetings, this could be used to trigger a certain movement!

Thank you so much for this great plugin! It is awesome to have this integrated within ARC! I know that you are super busy so it is very much appreciated!

1. use the ControlCommand() to send data within ARC

2. I can add the other variables

There you go: https://synthiam.com/redirect/legacy?table=plugin&id=193

@DjSures You are awesome!

This is supercool, it all works as a charm!

The only thing is that I am still a little puzzled on how to send data over to api.ai, it appears to be a little difficult...if webhooks are enabled api.ai seems to be needing a response which includes a header and a body...something like this!

Headers:
Content-type: application/json

Body:
{
"speech": "Barack Hussein Obama II is the 44th and current President of the United States.",
"displayText": "Barack Hussein Obama II is the 44th and current President of the United States, and the first African American to hold the office. Born in Honolulu, Hawaii, Obama is a graduate of Columbia University   and Harvard Law School, where ",
"data": {...},
"contextOut": [...],
"source": "DuckDuckGo"
}

So how would I send my variable to api.ai using the ControlCommand() function? a more detailed description on webhooks is documented here...

https://docs.api.ai/docs/webhook

Read the example in the description of the plugin. Forget about webhooks. Forget about json. Forget about wget. Read the plugin page. I took time to write how to use it AND provided an example.

Just use the plugin and send the text using controlcommand()

@DjSures I am really grateful and I am sorry if I am too bad to understand how this works! It might also be something that is not as easy as I thought it is! My idea was to store data to a variable within ARC...eg my sleep being logged from my Fitbit, and this data would be accessible to api.ai! Maybe this approach is super wrong...I will think about it before I keep on making random requests here!

The plugin works great, speech can be send via the Bing Speech Recognition and is returned properly, the ControlCommand() is working super easy to send speech over to api.ai! The action paramenter is returned and can be used to drive the robot!

Thanks so much for getting this done, I guess the best approach is to do the logic within ARC rather than sending the variable out to api.ai?

edit

I was just reading this post again, and just wanted to state that by no means I meant to say the plugin is useless for what I want to achieve! It works great, I just need to make a recap of my approach and find the right way to get things done!

Thanks again for putting in the effort to write a plugin! This is great!

@DjSures Thanks for the plugin, this is obviously the key to get started with api.ai on ARC! It can send natural language over to api.ai to be processed and the returned variables can be used to drive the robots actions and to output speech!

But api.ai is so much more than just a chatbot, and it would be kind of a shame to let go all the other functions implemented!

The example on the api.ai site on how to integrate fulfillment using webhooks is a bit limited so I found a very nice tutorial and a database which is very easy to use and to learn how to extract json data and get it send to api.ai!

I think without being able to connect to outside data api.ai is nothing more than a brain in a jar...

The approach to get this working is to use Amazons AWS Service, which is free for one year, and to use lambda to run Node.js, which seems to be the preferred application to process this type of requests these days. I would be grateful to have an aproach to host this on my server space which does not run Node.js at the moment, but I guess I will have to take it as I can get it done right now!

The Star Wars Bot will be able to use data from a Star Wars data base and extract the height of any Star Wars character, this could be turned into a fun trivia game asking Darth Jader questions! It is still be bit buggy and returns Luke Skywalker anytime it is asked for an unknown character, but it is good enough to get the point, right? Any data can be obviously retrieved from the data base...so lets have fun with it! Everyone is a Star Wars Fan right?

The Data Base with detailed instructions on how to extract data is to be found here... http://swapi.co/

This is a live version of the Star Wars Bot http://www.downtown-tattoo.de/robotics/StarsWars.html

A detailed instruction on how to set everything up to work, step by step is to be found here...kudos belongs to Raizlabs! https://www.raizlabs.com/dev/2017/01/build-ai-assistant-api-ai-amazon-lambda/

This setup is very flexible and will work with ARC, Actions on Google, Alexa and also as a standalone Webpage...

Thanks for helping me on this @DJSures, I hope I could also put something back to the community with this EZ-Robot project!

Have fun with it and try asking Darth Jader how tall Yoda is...

This is the EZ-Robot Project... edit I had to delete this because my Bing speech key was saved in the project, I could upload again if anyone is interested though...

Send data to the API.AI plugin by using the provided ControlCommand(). There is an example in the plugin. Programatically send any string to the api using the controlcommand(). You do not need node.js. You do not need any 3rd party applications. Simply use the provided ControlCommand() and send and receive strings to the API.AI server using the ARC API.AI plugin.

There are no 3rd party components necessary.

For an example of how to use the ControlCommand() with the API.AI plugin, click here: https://synthiam.com/redirect/legacy?table=plugin&id=193

You can send any data to the API.AI system using the ControlCommand() for the plugin. I repeat, there is no need for node.js or any third party components. Simply use the ControlCommand() for the api.ai to send data.

Configure your API.AI using their web interface on what to do with the data that you submit. You can submit any data to the API.AI using the controlcommand()

Use the ControlCommand() to send data to the api.ai. You can use the ControlCommand() to send data to their server. Any data can be sent to the API.AI server using the ControlCommand() for the plugin.

That would of course make things a lot simpler than what I did... But how would you integrate something like a weather forecast?

I just chose the StarWars database to have a fun thing to try...

How would your approach on this look like? I am by no means a coder, so anything that would make live easier for me is appreciated!

Could you take a look at the project and maybe give me a hint on what to change? That would be awesome!

edit This is the described Star Wars trivia bot... it needs an input of "how tall is yoda" or something alike!

edit I had to delete this because my Bing speech key was saved in the project, I could upload again if anyone is interested though...

Edited

Never mind, thanks