Thumbnail

Dialogflow

Use Google's DialogFlow to create an artificially intelligent NLP chatbot for your robot.

+ How To Add This Control To Your Project (Click to Expand)
  1. Make sure you have the latest version of ARC installed.
  2. Select the Get button in this page to download the archive file.
  3. Double click the downloaded archive file to execute installer.
  4. The installer will add this control to ARC.
  5. Load ARC and press the Project -> Add Control button from the menu.
  6. Choose the Artificial Intelligence category tab.
  7. Press the Dialogflow icon to add the control to your project.

Manual

Google's DialogFlow is a chatbot NLP (natural language processor). Either manually enter conversational phrases into the input field, or send phrases programmatically using the ControlCommand(). This plugin requires an internet connection, which means your computer must be connected to both the robot and the internet at the same time. 

The theory behind NLP is to have a conversation with a user to determine intent. The intents are defined by yourself in the DialogFlow configuration web console. The intent may require parameters, which can be prompted to the user until all of the required parameters are specified. Once all of the parameters are specified by the user, the intent can then be executed.

Here is an example of a conversational ChatBot that you can configure with DialogFlow and incorporate other ARC plugins, such as cognitive services for Vision and Emotion detection...



Programmatically send phrases using the following ControlCommand() syntax...

Code:


ControlCommand("DialogFlow", "Send", "Hello There")


The response from the server is stored in global variables. Additionally, a script can be configured to execute when the server responds.

Google Credential Service Account
DialogFlow uses a Google Credential Service JSON file, which can be downloaded and saved to your local hard drive. Reference the file using the config menu of this robot skill. See how to create a Google service account here. To visit the google cloud console directly, press the button on the config page.

1) Press the View Sevice Account button
User-inserted image


2) Press the Create Service Account option. Fill out the required information to create the service account.
User-inserted image


Download Service Account File

1) Click the View Services Account button in the robot skill's config menu
User-inserted image



2) Click the selected service account.
User-inserted image


3) Select Add Key and choose the Create New Key option. The JSON file will be downloaded.
User-inserted image


4) Select the JSON file using the config menu option
User-inserted image



Google Projects
Additionally, you can view DialogFlow projects directly by pressing the respective button on the config page. The Google project ID must be inserted into the respective textbox.

1) Press the "View DialogFlow Projects" button
User-inserted image


2) Press the Gear Icon next to the drop-down list of projects to view the Project Id
User-inserted image




Parameter Queries
Some DialogFlow Intents may require parameters that will be configured to prompt the user. Checking the $APIComplete variable will determine if the intent has been successfully fulfilled and completed. If the intent has been completed, the array $APIParameters will contain a list of the user-defined parameters.

See this DialogFlow Intent configuration for "Add Numbers".
User-inserted image



In this example javascript code will use an intent that requires 2 numbers and will add them together.

Code:


if (getVar("$APIIntent") == "Add numbers") {

if (getVar("$APIComplete")) {

var answer = getVar("$APIParameters[0]") + getVar("$APIParameters[1]");

Audio.say(getVar("$APIParameters[0]") + " plus " + getVar("$APIParameters[1]") + " equals " + answer);

} else {

Audio.say(getVar("$APIResponse"));
}
}


Example ChatBot - Create An Intent For Moving
Once you have some familiarity with creating a new Agent and adding the key to this plugin, we can walk you through how to create an intent. The verbiage used by DialogFlow may not be clear from the beginning. However, once you understand what an Intent and Entity is, it begins to make sense.

The process that we will take in these steps will use text input to teach the bot by typing sentences around a specific action. In this case, we're going to teach the bot how to move a direction (forward or reverse) for a number of seconds. We'll do that by continually typing phrases into the Intent and the bot will create the necessary variables.

Step 1
Load ARC and add two controls to your project. We'll add VARIABLE WATCHER and DialogFlow plugin. Because we'll be having the robot move FORWARD and REVERSE, you'll want to add a movement  panel to an existing project so you can actually see the robot move. If you add this robot skill to a blank project, such as this example shows, you'll never have the robot actually move. 
User-inserted image



Step 2
Load your Dialog Flow agent in a web browser
User-inserted image



Step 3
We're going to create a new Entity. An entity is essentially a variable that contains a bunch of synonyms. For example, you will see in the next few steps that we will create an identity called DIRECTION which will have two variables FORWARD and REVERSE, but a bunch of synonyms of each. This is how the AI magic understands what inputs people may provide that mean what you expect. So go ahead and create a new Entity.
User-inserted image



Step 4
Name this entity DIRECTION and create synonyms for each direction FORWARD and REVERSE as seen in the image below.
User-inserted image



Step 5
Now we will begin creating an Intent. This is done by pressing the + on the side menu next to INTENT.
User-inserted image



Step 6
Now begin adding phrases into the "User Says" box. This is the magic of AI, because the system will begin detecting the similarities of your statements and create an intent out of them. The sentences you enter must all be related to moving a direction (forward/reverse) for a number of seconds. See in this example the different entries I created. For best results, use the same entries that I did.
User-inserted image



Step 7
The seconds will now be detected as a time parameter, which normally would be a good thing. However, in this case we only want seconds. So we're going to edit the parameter and rename it to SECONDS and set the type to SYS.NUMBER.
User-inserted image



Step 8
Both parameters DIRECTION and SECONDS are required. So we'll need to set the checkbox for required on each parameter.
User-inserted image



Step 9
Now that the parameters are required, this is where the AI magic can have conversations with the user to get more information. For example, if you asked the robot to move Forward and didn't supply a number of seconds, the dialog prompt that we'll enter here will ask the user for how many seconds.
User-inserted image


Here is the prompt for DIRECTION
User-inserted image


Here is the prompt for SECONDS
User-inserted image



Step 10
Once all of the parameters have been provided to the bot during a conversation, the bot will respond. We can create our own response, which is friendly to the user. Here's an example response that uses the variables we created in the previous steps.
User-inserted image



Step 11
When the intent is successful and we wish ARC to do something, we need a way to identify what this intent is. The intent has an Action that is unique. In this case, we'll call the action move.direction.time
User-inserted image



Step 12
Save this Intent and we're done with the Dialog Flow editor and will return to ARC
User-inserted image



Step 13
Back in ARC. let's try out our Intent by typing MOVE and press ENTER in the DialogFlow plugin. Notice in the Variable Watcher that the $APIComplete variable is 0 (false). And, notice the Bot has asked us what direction to move? That is the bot executing the prompt we entered earlier because the parameter for direction is missing.
User-inserted image



Step 14
Respond with FORWARD and you will see the prompt now asking for how many seconds. The $APIComplete is still 0 (false).
User-inserted image



Step 15
Now enter a number of seconds. Notice the $APIComplete is now 1 (true) and the successful response is given.
User-inserted image



Step 16
The system now knows how to determine the direction and number of seconds to move. But, the robot doesn't know what to do with that yet. We will create a script that will execute the movement commands based on these parameters.

Click the CONFIG menu button on the DialogFlow plugin
User-inserted image



Step 17
Edit the SCRIPT
User-inserted image



Step 18
Finally, let's paste in some javascript code that will be executed.

Code:

if (getVar("$APIIntent") == "move.direction.time") {

if (getVar("$APIComplete")) {

Audio.say(getVar("$APIResponse"));

if (getVar("$APIParameters[0]") == "forward")
forward();
else
reverse();

sleep(getVar("$APIParameters[1]"))

stop();
} else {

Audio.say(getVar("$APIResponse"));
}
}

#1  
Wow great info here, I was just about to start from scratch figuring out how I would do this and bang there it is at the top of the Forum,almost like you are a psychic ,DJ!:)
#2  
Hi, my question is about integrating EZ-Robot JD Humanoid with DialogFlow. In my DialogFlow scenario the agent has a French female voice (FR-Wavenet-A). The question is:  with EZ-Robot JD Humanoid and DialogFlow integrated,  is it possible to retain the same voice for the robot? Thanks.