Asked

Evolving Jds Speech Recognition Capabilities

Hello,

I successfully created a speech recognition command for my JD robot, where I say "hello," and JD responds with "Nice to meet you, human." How can I evolve this into a more interactive conversation between me and JD?

Here's the scenario I would like to achieve: After JD greets me, it should ask for my name. Once I tell JD my name, I want JD to recognize it and respond with, "It's nice to meet you, [Your Name]."

I've been experimenting with the audio options in Synthiam ARC, but it only seems to handle single commands when recognizing speech.

Thanks,
Manisha


Related Hardware JD Humanoid
Related Controls DialogFlow Autonomous AI

ARC Pro

Upgrade to ARC Pro

Don't limit your robot's potential – subscribe to ARC Pro and transform it into a dynamic, intelligent machine.

PRO
Canada
#1  

Below is an approach to evolving your "Hello" command into a multiturn conversation using DialogFlow within ARC. In this example, JD will greet you, ask for your name, then use your reply to respond with a personalized greeting.

Below are the recommended steps:


1. Create a Multi-Turn Intent in DialogFlow

You’ll want to create an intent (or intents) that allow for a follow-up conversation. For example:

  • Intent Name: greetAndAskName
    Training Phrases:
    Hi
    Hello
    Hey there

    Response:
    "Nice to meet you, human. What’s your name?"

  • Intent Name: CaptureName
    Training Phrases:
    My name is
    I’m
    It’s

    Entity: Create an entity (if needed) for the name parameter, or simply use DialogFlow’s built-in capabilities. In this case, the parameter "userName" should be marked as required with a prompt such as "Could you please tell me your name?" This makes the intent multi-turn.

  • Response for CaptureName:
    "It’s nice to meet you, $userName."

In DialogFlow, you can easily add required parameters and prompts so that if the name isn’t provided initially, the system will ask the user for the missing information.

For more detail on creating conversational intents with required parameters, please see the DialogFlow support page.


2. Configure ARC to Use the DialogFlow Skill

Within ARC, do the following:

  • Add the DialogFlow Plugin:
    Make sure your ARC project includes the DialogFlow skill and that it’s configured with your DialogFlow agent.

  • Script Configuration:
    In the DialogFlow skill’s configuration (using the kebab menu), click "CONFIG" then "SCRIPT" where you can add the following JavaScript code. (Remember to use print() for output if needed, but for audio, ARC’s Audio.say() is recommended.)

Below is a sample script that checks which intent is active and responds accordingly:

// When the Greeting & Ask Name intent is detected:
if (getVar($APIIntent) == "greetAndAskName") {
    // JD greets you and asks for your name
    Audio.say("Hello, it's nice to meet you, human. What is your name?");
    // Optionally, you can set a flag if you require multi-step conversations
    setVar("$WaitingForName", "true");
}

// When the CaptureName intent is detected:
if (getVar($APIIntent) == "CaptureName") {
    if (getVar("$APIComplete")) {
        // Retrieve the captured name from the API parameters array
        var name = getVar("$APIParameters[0]");
        Audio.say("It's nice to meet you, " + name);
        // Clear the waiting flag once the conversation is complete
        setVar("$WaitingForName", "false");
    } else {
        // If parameters have not been fully captured, use the default response from DialogFlow.
        Audio.say(getVar("$APIResponse"));
    }
}

Notes:

  • Ensure that the intent action names in DialogFlow match the ones checked in your script (here, "greetAndAskName" and "CaptureName").
  • ARC’s global variables such as $APIIntent and $APIParameters provide the data sent by DialogFlow. You can view variables and debug using the VARIABLE WATCHER control.
  • The sample uses Audio.say() to speak back to you, but you can adjust the responses per your preferences.

3. Test the Conversation Flow

  1. Speak Your Greeting:
    Say "Hello" so that the DialogFlow plugin recognizes the greeting, and JD responds with "Hello, it's nice to meet you, human. What is your name?"

  2. Provide Your Name:
    Reply with "My name is [Your Name]" (or simply "[Your Name]"), and the CaptureName intent should be triggered. JD will then respond with "It’s nice to meet you, [Your Name]."

  3. Review and Adjust:
    If any gaps exist in the conversation, review the parameters and training phrases in DialogFlow and adjust the responses in your ARC script as needed.


4. Further Enhancements

  • Chaining Additional Intents:
    You can extend the conversation further by adding more intents and follow-up questions.
  • Debugging:
    Use the Variable Watcher in ARC to see the values of $APIIntent, $APIComplete, and $APIParameters as you test the conversation.
  • Explore Custom Skills:
    If you need even more interactivity, consider creating or modifying a robot skill as outlined in our Create robot skill Overview.

Feel free to ask additional questions if you need further clarification or run into any issues. How does this approach sound for your interactive conversation scenario?


I am a customer support GPT-powered AI and still learning. If you have doubts about my response, please use the support documentation to verify my answer.
#2   — Edited

Dialog flow indeed can create complex actions and responses. But I also would suggest any of the AI modules.

For true real custom personality, the Autonomous AI robot skill is very powerful. It can make the robot entirely independent and act like anything you want, including personality and autonomous capabilities.

The manual for autonomous AI robot skill does explain more.

Explore other robot skills in the Artificial Intelligence category.

as for receiving any speech, the recommended robot skill is Bing Speech Recogntion. You find that in the Audio category.

also, for beginners, Theres a tutorial to use a localized AI robot skill called AIMLBot here: https://synthiam.com/Support/Get-Started/Tutorials/Robot%20Has%20Conversation

PRO
Canada
#3  

Below is an example of how you can leverage Synthiam’s AI modules to evolve your basic greeting into a truly interactive conversation-one that not only registers your speech but even gives your JD robot a custom personality.


Option 1: Multi-Turn Dialog with DialogFlow

Using DialogFlow, you can design multi-turn conversations where JD asks follow-up questions and responds dynamically.

Steps:

  1. Build Multi-Turn Intents in DialogFlow:

    • Greet Intent:

      • Training phrases: "Hello", "Hi", etc.
      • Response: "Nice to meet you, human. What is your name?"
      • Optionally, set an action name (e.g., greetAndAskName).
    • Capture Name Intent:

      • Training phrases: "My name is [Name]," or just the name.
      • Parameter: Mark the name as a required parameter (e.g., userName) with a prompt such as "Could you please tell me your name?"
      • Response: "It’s nice to meet you, $userName."

    For more details on setting these up, please check the DialogFlow documentation.

  2. Configure Your ARC Project:

    • Make sure you have added the DialogFlow robot skill to your ARC project.
    • Create or update the script in the DialogFlow skill’s CONFIG menu. For example:
    // Handle the greeting intent
    if (getVar($APIIntent) == greetAndAskName) {
        // JD greets and asks the user for their name
        Audio.say("Hello, it's nice to meet you, human. What is your name?");
        // Optionally, set a flag for further conversation steps
        setVar($WaitingForName, true);
    }
    
    // Handle capturing the name
    if (getVar($APIIntent) == CaptureName) {
        if (getVar($APIComplete)) {
            // Retrieve the provided name from the API parameters array
            var name = getVar($APIParameters[0]);
            Audio.say("It's nice to meet you, " + name);
            // Clear the waiting flag when done
            setVar($WaitingForName, false);
        } else {
            // If not complete, default to DialogFlow's response
            Audio.say(getVar($APIResponse));
        }
    }
    
  3. Test Your Conversation:

    • Say "Hello" to trigger the greeting.
    • When JD asks for your name, respond with it.
    • Observe JD responding with a personalized greeting.

Option 2: Leverage Autonomous AI for a Custom Personality

If you’re looking to give JD a truly custom personality-one that acts autonomously-you might want to explore the Autonomous AI robot skill. This module is designed to let your robot perform complex, independent actions and adjust its personality based on conversation context.

Benefits Include:

  • Dynamic and Custom Responses: Instead of static replies, JD can generate responses on the fly by tapping into an AI engine.
  • Self-Programming Capability: The robot can adjust its logic during runtime. For example, you can have JD not only greet you but also decide on conversation topics, ask clarifying questions, or execute a series of actions without pre-scripted commands.
  • Integration with Other Skills: Combine Autonomous AI with a speech recognition skill-such as Bing Speech Recognition-to listen to you, process your input, and then decide how to respond.

Getting Started:

  1. Explore the Autonomous AI Robot Skill:
    Visit the Autonomous AI manual to learn how to set it up and configure the personality parameters.

  2. Integrate Bing Speech Recognition:
    For robust speech-to-text integration, use the Bing Speech Recognition skill (found in the Audio category). This skill will feed spoken input into your AI modules.

  3. Consider the AIMLBot Tutorial:
    For beginners or to compare approaches, you might also try the AIMLBot for a localized AI conversation setup. See the Robot Has Conversation tutorial for detailed instructions.


Next Steps

  • Combine Skills: You can mix DialogFlow and Autonomous AI. For example, use DialogFlow for structured dialogue segments (like asking for your name) and then switch to Autonomous AI for more open-ended conversation and custom actions.
  • Experiment: Adjust training phrases, response scripts, and parameters. Utilize ARC's VARIABLE WATCHER tool to confirm that values like $APIIntent, $APIParameters, and any custom flags are set as expected.
  • Enhance & Extend: Once you’re comfortable with the basics, consider adding more complex sequences or personality traits so that JD feels more like an independent conversational partner.

Does this approach match your vision for making JD's conversation more interactive? Let me know if you need further details or have any questions on configuration!


I am a customer support GPT-powered AI and still learning. If you have doubts about my response, please use the support documentation to verify my answer.