United Kingdom
Asked — Edited
Resolved Resolved by WBS00001!

Displaying A Robots Response In A Text Format

A bit of an unusual request, but there is reasoning behind what I'm about to ask. So my question is...

Is there a way to display a robots spoken response as text with ARC?

Let me explain. Speech synthesis for a robots response is not an option here. You speak to a robot using speech recognition and the robot responds with text displayed on the computer screen. Now using the Pandorabot control will do this showing the robots response in the debug window, but the text is too small.

If your familiar with Pandorabot, you'll know that when you have a published bot, you get your own page where your Pandorabots response is displayed and you can add HTML code to change the text layout. This is something like what I'm looking to do via ARC. I could use the Pandorabot portal, but I would need control commands which are imbedded in the bots responses to activate when the response is given.

The reasoning behind this, is for someone who is hard of hearing to interact and converse with the robot, so you see why speech synthesis can't really be used in this situation. I might be overlooking something obvious here, or there might be something that I'm unaware of, so ANY input anyone can offer on this really would be gratefully appreciated.

Cheers,

Steve.


ARC Pro

Upgrade to ARC Pro

Unleash your creativity with the power of easy robot programming using Synthiam ARC Pro

United Kingdom
#33  

@WBS.

Sorry, haven't had time to try your program yet as robots have taken a back seat for now while I sort out some family problems that have come up. One I get to try it out, I'll report back.

@DJ.

So using the SketchPad will do exactly what I asked in the thread title, whereby any verbal response given by a robot can actually be displayed in text using this control?

If so, could you share an example of this with us please.

Thanks.

#34  

@JustinRatliff It should be the size according to this:


ControlCommand("Sketch Pad", CanvasDrawString, [X], [Y], [Size], [Text], [Color])

Don't know yet why it makes no difference.

@Steve G Yes, it looks like you can use it to display text. Changing the text size will mean changing how linefeed works in that the different text sizes will require a different line spacing but that should not be too hard to work out. As far as I have been able to test the legal color values are: White,Black,Aqua,Blue,Fuchsia,Gray,Green,Lime,Maroon,Navy,Olive,Purple,Red,Silver,Teal,Yellow

Here is some code to make it cycle through all possible color combinations, beginning with White and Black:


ClearVariables
$SleepTime =2000
$MaxColors =16
$BasicColors ="White,Black,Aqua,Blue,Fuchsia,Gray,Green,Lime,Maroon,Navy,Olive,Purple,Red,Silver,Teal,Yellow"
$NextBkgndColor =0
repeatwhile($NextBkgndColor < $MaxColors)
  $TheBkgndColor =Split($BasicColors,",",$NextBkgndColor)
  $NextColor =0
  repeatwhile($NextColor < $MaxColors)
    $TheColor =Split($BasicColors,",",$NextColor)
    ControlCommand("Sketch Pad", CanvasClear,$TheBkGndColor)
    ControlCommand("Sketch Pad",CanvasDrawString,30,30,15,"The Background color is "+$TheBkGndColor,$TheColor)
    ControlCommand("Sketch Pad",CanvasDrawString,30,50,15,"The Text color is "+$TheColor,$TheColor)
    $NextColor++
    Sleep($SleepTime)
  endrepeatwhile
  $NextBkgndColor++
endrepeatwhile

I'm still looking into how the rest all works, but if you want an in-between color, you have to use this form:


#For the background:
ControlCommand("Sketch Pad", CanvasClear, [Red], [Green], [Blue])

#And the text:
ControlCommand("Sketch Pad", CanvasDrawString, [X], [Y], [Size], [Text], [Red], [Green], [Blue])

Red, Green, and Blue are in the range of 0-255.

I have code for making it cycle through all the possible colors but it takes a very long time to run even with no sleep statements.

Anyway, now that we know about all this it might just be the easiest way to go. Of course you can't change fonts or have italics, but you can do bold. At least I know enough about it for now to come up with a script to make it all happen if you like.

#35  

The size seems to make a difference for a line, but for text it does nothing for me

#36  

@JustinRatliff Yes, that's the thing in all this. DJ says to just look at the examples. There, problem solved. Nope. The examples only highlight a couple of things about it all. All I learned from the examples was that the color was words and not number values. They, and the tutorials are FAR from complete. Nothing about the other color names. Nothing about why 15 was used in the size value or why it doesn't seem to do anything. Nothing about the appropriate range of values for the Red, Blue, Green values. Nothing about changing font sizes or thickness. Maybe you have to change line thickness first, then that will change font thickness as well. Still working on that sort of thing. For something that is supposed to be easy for beginners to use, there is a lot left out that beginners would need to know.

United Kingdom
#37  

@WBS.

I finally managed to get around to try your program.

I seemed to have set it up okay, and your test script works. I'm trying to run a simple speech recognition control script to display a Say() response using the $SpeechPhrase variable, but I'm having trouble receiving the response.

I had a play with the SketchPad as well and I'm a littler clearer on what it does, but an explanation of the different values would be helpful. But I'm still having the same issue as above. I guess what I need is a kind of SpeechCommand() variable to display the speech synthesis Say() text and Pandorabot control text which I thought would exist but can't see any. What do I need to do here, as I'm a little lost and not sure what a variable to display this text would look like?

#38  

@SteveG The variable you're looking for is $DisplayText. It is used in the SendToDisplay script.

Before using the SendToDisplay script to automatically send strings to the external display (via setting $DisplayText), however, you must start the script:


CC("SendToDisplay",ScriptStart)
Sleep(2000) 

You only need to start it once and it runs forever until explicitly stopped.

Wherever you have a "Say" command, then, you also set $DisplayText to that same string. Something like this:


Say("Hello World")
$DisplayText ="Hello World"
Sleep(2000)
  #Or, using a variable:
$Phrase ToSay ="Hello World"
Say($PhraseToSay)
$DisplayText =$PhraseToSay
Sleep(2000)

That's it. The bottom line is, whatever you want to go to the display, you set $DisplayText to that text. Anywhere in any script at any time. That's all. The SendToDisplay script will automatically take care of sending that text to the external display.

Also the Sleep(2000) statement could be placed in the SendToDisplay script. It already has a Sleep(500) statement. Just change it to Sleep(2000).

OTOH, if there will naturally be at least 2 seconds between sending text to the display, then you can leave it out entirely.

Now, while I'm on the topic, let me address another point concerning the 2 second (or longer) sleep time between individual strings being sent to the external display. This could become annoying when the reader is having to read more than one line for a given response. If you wish to send multiple lines to the display and they all need to be read in quick succession, the best way to do that is to send them all at the same time. This is what I mean:


Say("Sentence 1")
Say("Sentence 2")
Say("Sentence 3")
#There would need to be at least 2 seconds between for the display
$DisplayText ="Sentence 1")
Sleep(2000)
$DisplayText ="Sentence 2")
Sleep(2000)
$DisplayText ="Sentence 3")
Sleep(2000)

That pause between what should be a rapid succession of sentences could be annoying. So it would be best to concatenate them and send all at once, like this:

 
Say("Sentence 1")
Say("Sentence 2")
Say("Sentence 3")

$DisplayText ="Sentence 1. Sentence 2. Sentence 3" )
Sleep(2000)

  #Or, using variables:
$Sentence1 ="Sentence 1"
$Sentence2 ="Sentence 2"
$Sentence3 ="Sentence 3"

Say($Sentence1)
Say($Sentence2)
Say($Sentence3)

$DisplayText =$Sentence1+$Sentence2+$Sentence3
Sleep(2000)

The display could be setup to show the sentences like this: Sentence 1. Sentence 2. Sentence 3.

Or it could break them at the periods and show then like this Sentence 1. Sentence 2. Sentence 3.

Even put a blank line in between: Sentence 1.

Sentence 2.

Sentence 3.

All that could be done via setup options.

Something I just thought of To better display the sentences in a timely manner it may be best to send them to the display BEFORE saying them. That would probably help with the sleep issue as well, since it would probably take a couple of seconds to say the sentence(s), so no Sleep(2000) statements would be needed.

EDIT You're using Say instead of SayWait so you would need the Sleep delays after all.

One advantage of using the Sketch Pad option is that there would not be an issue of needing sleep periods between the sending of text lines to it.

United Kingdom
#39  

Thanks for getting back to me. I see where I was going wrong now. So every Say() or SayWait() script needs the $DisplayText variable in it too. Any ideas on how this could be achieved with the Pandorabot control as well, as there is not a scripting option for this control?

#40  

@SteveG I had another look at the Pandorabot control but there just seems to be no way to get at the returned text from the Pandorabot site. One would have to figure out a way to intercept it as it comes in to the control or as it is displayed to the debug window. Perhaps there would be a way using the SDK, but it would require a knowledge of this control's operation. That is probably not a part of the SDK. Nor would it likely be a part of the new Plug-in system. The only other ways I can think of offhand to get to it would be:

  1. Somehow cajole DJ into putting a variable in the Pandorabot control which would contain the text send from Pandorabot (like is done with the $SpeechPhrase variable). This would be the quickest and most straightforward route and should be relatively easy for him to do, but that task is much like getting a glacier to move by pushing from the back end, being as swamped as he is. I'm surprised this isn't already a part of that control actually.

  2. To write a plug-in which would perform the same function as the current Pandorabot control but allow for the returned text to be placed into a variable for the user to send to a display other than the Debug window.

  3. Write a plug-in which would be able to get at the text of any debug window.

  4. Write a totally separate program which would provide a user interface for the users current Pandorabots and send the text to the computer's speech generator or to the ARC to be sent on to the robot's speaker.

  5. Write something which would look at the data stream going to and from the internet on the user's network and parse out those parts coming from Pandorabot for display.

None of those options are easy or something I could do quickly since I am not versed with the SDK or the plug-in system. Nor am I familiar with the Pandorabot's system of coding. Though it doesn't seem to be too difficult.

Perhaps starting a new thread focusing on just the Pandorabot control and it's lack of a text variable like the Speech Recognition control has would be the best course of action for now. No one seems to be very concerned with this thread anymore but the two of us.