United Kingdom
Asked — Edited
Resolved Resolved by WBS00001!

Displaying A Robots Response In A Text Format

A bit of an unusual request, but there is reasoning behind what I'm about to ask. So my question is...

Is there a way to display a robots spoken response as text with ARC?

Let me explain. Speech synthesis for a robots response is not an option here. You speak to a robot using speech recognition and the robot responds with text displayed on the computer screen. Now using the Pandorabot control will do this showing the robots response in the debug window, but the text is too small.

If your familiar with Pandorabot, you'll know that when you have a published bot, you get your own page where your Pandorabots response is displayed and you can add HTML code to change the text layout. This is something like what I'm looking to do via ARC. I could use the Pandorabot portal, but I would need control commands which are imbedded in the bots responses to activate when the response is given.

The reasoning behind this, is for someone who is hard of hearing to interact and converse with the robot, so you see why speech synthesis can't really be used in this situation. I might be overlooking something obvious here, or there might be something that I'm unaware of, so ANY input anyone can offer on this really would be gratefully appreciated.

Cheers,

Steve.


ARC Pro

Upgrade to ARC Pro

Experience the transformation – subscribe to Synthiam ARC Pro and watch your robot evolve into a marvel of innovation and intelligence.

#1  

Let me see if I understand. What you want is to display (in larger text) whatever the robot says via a SayEZB or SayEZBWait command. Is that right?

United Kingdom
#2  

@WBS.

Not quite right buddy. I don't want to use the "Say" commands (although this could actually be useful also), it's more about showing the Pandorabots responses.

#3  

Ok. I take it you are using the Pandorabot control then. And it, somehow, sends it's responses to the robot or PC. What's needed is a way to find or intercept those strings for display I take it. Or am I still off?

Looking at the Pandorabot control I see it has check boxes for where to send the responses, none of which is "send to a variable." Even if it had that there would still be the problem of displaying it. Would probably have to send it to an external program for actual display. The Script language doesn't even have anything for sending a variable to a script control (or reading from a Script control). Not even to a Notepad control. All of which would be handy.

I also looked at the Variable Watcher control to see if some system global variables were listed that might have been related to the Pandorabot controls, but, alas, nothing. I'm afraid I'm at a loss at the moment.

What is needed is a companion control to the Pandorabot control (and perhaps others) that can be used for displaying the response. Possible but not to likely to happen any time soon. Like you say, that would really be helpful to the hard of hearing or the profoundly deaf. The next best could be being able to enlarge the font size in the debug portion.

United Kingdom
#4  

@WBS.

You've pretty much nailed what I'm asking. Yeah Pandorabot is the main control that I'm looking at, although this would be useful for any speech coming from the robot.

The idea of being able to resize the debug font would be a possible solution, but I'm hesitant to ask for this as a feature request as staff participation on the forum has been very quiet recently to say the least, so this request could easily go unnoticed which would be a shame not just not my situation, but for others who know someone (or even themselves) with hearing problems.

You can see why I'm at a loss to think of a way to parse this information for it to be displayed. On the surface, this seems like something simple (computers with a screen do it all the time), but thinking how it would all go together shows that it may not be so simple.

#5  

Yes, there seems to be no ready solution to the Pandora problem.

That notwithstanding, if you are interested in displaying strings you would also be sending to be spoken via SAY type commands (or anything else), I could help there. I don't know what software tools you have at your disposal so my help may not be needed, but I thought I'd offer. I can write a simple program that would display that text. I would see it as working from text file update sent by a script. The Script language does have several commands for reading and writing to a file at least. Would even be possible to send formatting commands if desired. It could even be started and stopped from a script via the Script Exec command as you did before.

United Kingdom
#6  

Thanks for your kind offer. I don't know what sort of software tools I would need or indeed how to use them if I had them. I'm more about building and electronics. Networking and software... confused, which about somes it up for me, lol. Let me know what you have in mind or show me an example and I'll see if it could be something I could wrap my head around.

Thanks for the help.

#7  

Ok, be happy to do it for you. I've been working on programs to manipulate ARC data already. One that extracts the Speech Recognition data from the .EZB file and allows you to do things like alphabetize it, search it, do some editing or moving around, etc. Things you can't do in ARC. Then merges it back into the project from which it was taken. Even can save the data in a separate file for recall later. I'm figuring I can use that to load up different sets of phrases as desired. Could even have different sets of the same phrases (and scripts) for various languages or applications. Anyway, point is I've already been working with this stuff external to ARC, so what you want isn't starting from scratch for me. I can use the same base but with different bells and whistles. E-Z P-Z 1,2,3-Z! :D

One quick question, are you running Windows 7 or 8?

United Kingdom
#8  

That's great, thank you. It's Windows 8.1 I'm working with.

#9  

This would be a great new feature for DJ to add to the list. A text results box that displays variable contents with formatting.

Until or unless that comes as a feature, I see two ways to do this, both requiring some custom coding.

One would be as wbs00001 proposes in having EZ-B write to a file and than custom code display the file contents with whatever formatting you add.

The other would be to use the SDK to write a listener for a specific variable and then display the variable with formatting.

In either case you could either put a timer to remove the display after a pre-set interval, or just have new text over-write old text (or scroll it so the user can scroll back through responses if they come too fast).

Unfortunately, I don't have the time to work on either idea, but maybe wbs00001 or one of the other experienced developers could throw something together.

[edit] whoops, I was typing while wbs00001 was answering. Looks like a solution is incoming....

Alan

United Kingdom
#10  

@Alan.

Thanks for your response. It would be great if DJ would consider adding this to ARC for those who can't use speech synth. The scrolling idea would be good. That way a user could posit backtrack to view what was previously said similar to what iOS Siri does. If it was SDK based, I would probably struggle with it (never used it).

Anyway as you say, it looks like @WBS just might have a good solution for this.

#11  

If I ever have some free time I am going to take some C# lessons to get my skills up to speed with modern programming (I was handy with VB6, but have been doing architecture but not actual programming since then). Once I have the skills, I will be doing a lot of extensions and integrations to ARC with the SDK.

Alan

#12  

Hey SteveG, I asked about this same feature over a year ago. I wanted it for the same reason as you. I was planning to make a small external app for this and make it two way communication for text like a chat screen.

United Kingdom
#13  

@Justin.

That's interesting. I wondered if someone else might have asked about this (to be honest, I didn't do a forum search before I asked). I'm surprised this has not been implemented in to ARC already, especially as you asked over a year ago, as this really would be a valuable tool to have for the reasons we have stated. Do you reckon you'll give your app a go one day?

#14  

@SteveG Well, DJ is back and he has hit the ground running after some time off to recharge. He has been a fireman, going around and putting out one fire after another on the forum. Including the bug I recently posted about. He also said he has a re-write of ARC. Now I wonder if it's going to be worth the effort to do this. He may already have something like that or soon will due to the chat in this thread and, apparently, threads which have gone before. It doesn't seem it would take a whole lot to do if done within ARC itself. What do you think?

@JustinRatliff Sounds like you had interesting ideas on this with the 2-way chat thing. If you don't mind saying, I'm wondering what you meant by that? Chatting with who? Thanks.

United Kingdom
#15  

Yes leaving it for a couple of weeks or so to see what will be new in ARC sounds like a good idea. It would be great to hear from DJ to see what his thoughts are on this subject, or if he has plans to implement this, as apart from us guys who have posted here, I'm sure there are many other membership potential customers who would welcome an addition such as this.

United Kingdom
#16  

I just wanted to bump this thread so it doesn't get lost of forgotten about. Looking at the new ARC release, there doesn't look like anything has been implemented in regards to this subject, although the new plug-in idea could be used for this I guess (although I wouldn't know how to do it).

@WBS.

In regards to Justin's post about two way conversation, I think he meant a user chatting with the robot. I know we said about leaving this for a bit to see what the new release would bring, but I wanted to ask how far you got with what you were working on for this?

#17  

I got as far as a proof of concept trial. I wrote a program to simply read the data exchange file and delete the info from it after reading and displaying it. On the ARC side I made a couple of scripts. One to run a test of sending one text after another, and another script to process and send the texts to the same file.

It all worked okay but there has to be at least a 2 second delay between text sends from ARC for it all to work out. I don't think that will be a problem in this situation though. I'm kind of surprised it takes so long to reset the file for the next send. I might experiment with a virtual disk to perhaps speed that up. Maybe use another file monitor component in the external program as well. It seems to take about a half second or so for the one I'm using to recognize the file has changed. It works directly with the OS shell so I'm also surprised it takes that long.

Anyway, it would not take long to get something going. A couple of days maybe. I'm making it resizable so you can make it any size you wish. It will have a font selection button so that you can change the font, the font color, the font size and the usual (Underline, bold, strike through, Italics). I'm thinking about allowing some control characters to be sent as well to allow for the font properties to be set automatically. But that will come later after the basic program has been tried out and is working reliably. Of course there will be a clear screen button and you will be able to scroll back to past text as well. Anything else you can think of that would be good to have on it?

I also have a program I wrote some time ago to show pictures, videos, and gifs (though the gifs don't always do too well). I was thinking about using a variation of that to work on this same principle so you could send commands to have it show whatever visuals you like as well. It could stay up at least. Not have to close and open every time to show a new picture or whatever.

United Kingdom
#18  

That's great. Thanks for the update WBS. I look forward to having a play with this when its ready. :)

#19  

>>@JustinRatliff Sounds like you had interesting ideas on this with the 2-way chat thing. If you don't mind saying, I'm wondering what you meant by that? Chatting with who? Thanks.

@WBS00001, haha....very good question! I guess I assumed everyone knew I meant two way text chat with your EZ-Robot. So rather then speaking commands and requests you could type them.

United Kingdom
#20  

@WBS.

Sorry, I forgot to say... no there's really nothing else I can think of at this stage. I think the proof of concept and trials you have so far will be great as it is. One little thing springs to mind though, is reading the text layout (however you would have it formatted) on a portrait screen, ie on a tablet, so you wouldn't have to manually side scroll the screen to read said text (useful if a tablet is installed in a robot in the portrait position).

#21  

@SteveG Wait ... you want to run this on a tablet? Windows 8 tablet I suppose? I've never tried any of my software on a windows 8 tablet so I have no idea if it will run. In theory it should. I guess I should send something soon, finished or not just to see if it will work.

That notwithstanding, the format should be easy enough since it will be fully resize-able so it should be possible to make it fit properly in either mode. I don't know what will happen when you switch between the two modes, however. I'll make it compact enough such that it all controls will show up under portrait mode. As far as not having to side scroll in portrait mode, I guess that would depend on the font size selected and you can choose whatever you want there. An alternative would be word wrap. I'll put in a check box to allow for either word wrap or not.

A question though, what is the X-Y resolution of the tablet? So I can make the startup size small enough to fit in that resolution (with room to spare, of course). I'd make it initially come up sized for portrait mode. Again I have no idea just how it will really appear. All we can do is try it on your device and see what happens and make adjustments from there.

I'll post something to the cloud later tonight or tomorrow just to run tests with.

United Kingdom
#22  

Thank's my friend. Yeah, it's an Acer W3 that runs full Windows 8.1, so there shouldn't be any real difference. The w3 screen res is 1280 x 800.

#23  

@SteveG I have something ready to go for uploading but it seems you can only upload projects to the EZ Robots cloud. So I'll have to use another method. Any suggestions? Email? DropBox? FTP? What is good for you?

United Kingdom
#24  

Hey WBS.

You could put it in the cloud, but You can email me if you like. My address is in my profile.

#25  

I've sent it in an email with instructions. Let me know how it goes and what changes need to be made. I'm sure I haven't thought of everything. Maybe not even most things. Have "fun". Try not to curse me too much. :)

EDIT Well darn! The email was rejected because of the attachment. Probably because it's an EXE file. I'll try again by renaming the file so that it has a TXT extension. You can put it back to EXE on your end.

United Kingdom
#26  

Thanks buddy. I'll check it out hopefully this weekend, and get back to you. :)

EDIT.

I just saw your edit about the email. Can you not attach the file to a forum post?

#27  

Ok, I'm trying to attach the file to this post. If it seems to work, I'll put the instructions I put in the email here as well.

Didn't see anything. Zip Files are listed as a legitimate file to attach. So I zipped it up and tried again.EZDisplayInstaller.zip

Looks like that may have worked so here is what I wrote in the email:

Hi Steve;

I've attached an installer for what I'm calling the EZDisplay. It will allow for step by step setup even though there are only three files involved. There are certain directories that must be setup as well and an installer seemed like the most straightforward way to do it.

Be sure to take note of whatever directory you choose to set it up in (copy and paste is good) so you can put the same root directory in the accompanying script. A good place might be the ARC directory itself, but anything is fine. The default will be what I chose during the development process (F:\EZ_DisplayD7\TestArea).

Once installed, it will be ready to go (in theory). There is even a check box at the end to launch it for the first time if you wish and a desktop icon should be created for subsequent startups. Of course you can also start/stop it via the Exec script command.

The directory for the data transfer is called, appropriately enough, "Transfer." And the transfer file itself is called "DisplayText.txt." The DisplayText.txt file is used by both the EZDisplay program and the ARC script. Speaking of the script, here it is:


$TheFileName ="F:\EZ_DisplayD7\TestArea\Transfer\DisplayText.txt"
$StopSendToDisplay =False
FileDelete($TheFileName) #Clears the DisplayText.txt file
FileWriteLine($TheFileName,"" ) #Recreates it again
repeatwhile($StopSendToDisplay =False)
  $DisplayText =""
  WaitForChange($DisplayText)
  Print("  "+$DisplayText)
  FileWriteLine($TheFileName,$DisplayText)
  Sleep(500)
endrepeatwhile

It's fairly simple. I put it in a script control called "SendToDisplay". You can call it whatever you wish. The main variable is $DisplayText. You would use that variable in whatever script you want to send data to the Display. All you have to do it set the $DisplayText equal to what you want to send, usually what the robot says. When you do that, The "WaitForChange" command is satisfied and the script goes beyond that and sends the line out.

Of course you will want to put the path you chose at installation for the $TheFileName variable. The path in this example is: "F:\EZ_DisplayD7\TestArea"

The last part ("Transfer\DisplayText.txt" ) will always be the same. Also, never put anything else in the Transfer directory. It must contain only the DisplayText.txt file.

To set it up you have to have another script first call it via a Command Control. Here is a script I used for testing:


CC("SendToDisplay",ScriptStart)
Sleep(2000)
$TheCount =0
repeatwhile($TheCount <=100)
  $TheCount++
  $DisplayText ="AAAAAAAAAAA  AAAAAAAAAAA AA Sentence number "+$TheCount
  Sleep(2000)
endrepeatwhile 
CC("SendToDisplay",ScriptStop)

The first line calls the SendToDisplay script one time to allow it to set itself up. Once done, the SendToDisplay script will run in an endless loop until terminated by: CC("SendToDisplay",ScriptStop).

It can also be stopped by setting $StopSendToDisplay to True.

This test script simply sends out the same line with a new line number attached 101 times. Again, when you actually use the SendToDisplay script, just run CC("SendToDisplay",ScriptStart) once somewhere in an Init script and set $DisplayText equal to what you want to send each time.

I have gotten most of the functions I wanted to have in the Display program working. You can bring up the options by right clicking anywhere in the display area with a right mouse click or pressing the "Set" button at the lower left. The options are: Word Wrap - Same as in Notepad and other editors. Auto Clear - Clears the display area each time a new line comes in Show Buttons - Not implemented yet Show Vert Scroll - Show the vertical scroll bar Show Horiz Scroll - Show the horizontal scroll bar Set Font - Opens another window to set the font options

All the settings, as well as, the window size and position, and font options will be remembered when the window is closed.

As I said, I have no idea just how it will work out on the tablet, especially when you turn it from Portrait to Landscape position. Just have to try it and see. Then you can advise me on what needs to be done to make it look better.

The reason I have not implemented the "Show Buttons" option is that I'm not sure how you would get to the options if you set Show Buttons to False (no buttons or button panel shown at the bottom). That will show only display text area, but without a mouse, I don't know how you would get to the options after you hid the "Set" button.

So, give it a try and let me know what happens. :)

United Kingdom
#28  

Excellent stuff. I look forward to trying this. If I can get the time I'll try it tonight as I'm itching to have a play. Thanks again, and I'll let you know how it goes.

PRO
Synthiam
#29  

Use the Sketch Pad control: https://synthiam.com/Tutorials/Help.aspx?id=195

It has many ControlCommands() which you will find in the Cheat Sheet. One is for DrawString, which accepts colors, size and the string. Simply use the Clear ControlCommand() before drawing the text each time to start with a blank canvas of your preferred color.

Resize the control to be your preferred size as well.

#30  

@DJ Sures

Quote:

It has many ControlCommands() which you will find in the Cheat Sheet. One is for DrawString, which accepts colors, size and the string. Simply use the Clear ControlCommand() before drawing the text each time to start with a blank canvas of your preferred color.
Thanks for the info, but there's the rub. There is NO further information. Even the tutorial gives no clue you can even "write" strings in that control. There is nothing as to what form or range the numbers involved should be. HTML code? RGB code? Just numbers? What? You can't just reference the tutorial and say all you need to do is this and this and bingo! See how simple it is?.

I finally figured out what the range should be by making them too big and got an error message which indicated they needed to be between 0 and 255. Same with the number of parameters must be in each command. The error message told me that, finally. All this stuff should be in the tutorial. And that doesn't count the fact nothing mentions the numbers have to be in quotes. I still don't know what sort of number has to be in this command as to color:


ControlCommand("Sketch Pad", CanvasDrawString, [X], [Y], [Size], [Text], [Color])

Whatever I put in there seems to do the same thing. There seems to be no wrong answer for color.

When I first brought up the Cheat Sheet for this control all 3 possible commands were exactly the same. Only when I tried the one shown above, did it change to show 3 different commands.

That is why people don't even try controls like this. Don't even know this one can display text.

PRO
Synthiam
#31  

You will find examples in the examples folder of ARC. Here is an important tutorial on example files: https://synthiam.com/Tutorials/Lesson/22?courseId=6

#32  

#ControlCommand("Sketch Pad", CanvasDrawString, $x, $y, 15, "Time is: " + $time, "White")

The Example Project is pretty nifty, it provides and I think it'll prove to be robust once we all have more examples and see it used more. My question is, in the code example above $x and $y provides the location to display the text, but what does the value of "15" do? I thought it was font size at first but I can change the value and it does not appear to make a difference.

United Kingdom
#33  

@WBS.

Sorry, haven't had time to try your program yet as robots have taken a back seat for now while I sort out some family problems that have come up. One I get to try it out, I'll report back.

@DJ.

So using the SketchPad will do exactly what I asked in the thread title, whereby any verbal response given by a robot can actually be displayed in text using this control?

If so, could you share an example of this with us please.

Thanks.

#34  

@JustinRatliff It should be the size according to this:


ControlCommand("Sketch Pad", CanvasDrawString, [X], [Y], [Size], [Text], [Color])

Don't know yet why it makes no difference.

@Steve G Yes, it looks like you can use it to display text. Changing the text size will mean changing how linefeed works in that the different text sizes will require a different line spacing but that should not be too hard to work out. As far as I have been able to test the legal color values are: White,Black,Aqua,Blue,Fuchsia,Gray,Green,Lime,Maroon,Navy,Olive,Purple,Red,Silver,Teal,Yellow

Here is some code to make it cycle through all possible color combinations, beginning with White and Black:


ClearVariables
$SleepTime =2000
$MaxColors =16
$BasicColors ="White,Black,Aqua,Blue,Fuchsia,Gray,Green,Lime,Maroon,Navy,Olive,Purple,Red,Silver,Teal,Yellow"
$NextBkgndColor =0
repeatwhile($NextBkgndColor < $MaxColors)
  $TheBkgndColor =Split($BasicColors,",",$NextBkgndColor)
  $NextColor =0
  repeatwhile($NextColor < $MaxColors)
    $TheColor =Split($BasicColors,",",$NextColor)
    ControlCommand("Sketch Pad", CanvasClear,$TheBkGndColor)
    ControlCommand("Sketch Pad",CanvasDrawString,30,30,15,"The Background color is "+$TheBkGndColor,$TheColor)
    ControlCommand("Sketch Pad",CanvasDrawString,30,50,15,"The Text color is "+$TheColor,$TheColor)
    $NextColor++
    Sleep($SleepTime)
  endrepeatwhile
  $NextBkgndColor++
endrepeatwhile

I'm still looking into how the rest all works, but if you want an in-between color, you have to use this form:


#For the background:
ControlCommand("Sketch Pad", CanvasClear, [Red], [Green], [Blue])

#And the text:
ControlCommand("Sketch Pad", CanvasDrawString, [X], [Y], [Size], [Text], [Red], [Green], [Blue])

Red, Green, and Blue are in the range of 0-255.

I have code for making it cycle through all the possible colors but it takes a very long time to run even with no sleep statements.

Anyway, now that we know about all this it might just be the easiest way to go. Of course you can't change fonts or have italics, but you can do bold. At least I know enough about it for now to come up with a script to make it all happen if you like.

#35  

The size seems to make a difference for a line, but for text it does nothing for me

#36  

@JustinRatliff Yes, that's the thing in all this. DJ says to just look at the examples. There, problem solved. Nope. The examples only highlight a couple of things about it all. All I learned from the examples was that the color was words and not number values. They, and the tutorials are FAR from complete. Nothing about the other color names. Nothing about why 15 was used in the size value or why it doesn't seem to do anything. Nothing about the appropriate range of values for the Red, Blue, Green values. Nothing about changing font sizes or thickness. Maybe you have to change line thickness first, then that will change font thickness as well. Still working on that sort of thing. For something that is supposed to be easy for beginners to use, there is a lot left out that beginners would need to know.

United Kingdom
#37  

@WBS.

I finally managed to get around to try your program.

I seemed to have set it up okay, and your test script works. I'm trying to run a simple speech recognition control script to display a Say() response using the $SpeechPhrase variable, but I'm having trouble receiving the response.

I had a play with the SketchPad as well and I'm a littler clearer on what it does, but an explanation of the different values would be helpful. But I'm still having the same issue as above. I guess what I need is a kind of SpeechCommand() variable to display the speech synthesis Say() text and Pandorabot control text which I thought would exist but can't see any. What do I need to do here, as I'm a little lost and not sure what a variable to display this text would look like?

#38  

@SteveG The variable you're looking for is $DisplayText. It is used in the SendToDisplay script.

Before using the SendToDisplay script to automatically send strings to the external display (via setting $DisplayText), however, you must start the script:


CC("SendToDisplay",ScriptStart)
Sleep(2000) 

You only need to start it once and it runs forever until explicitly stopped.

Wherever you have a "Say" command, then, you also set $DisplayText to that same string. Something like this:


Say("Hello World")
$DisplayText ="Hello World"
Sleep(2000)
  #Or, using a variable:
$Phrase ToSay ="Hello World"
Say($PhraseToSay)
$DisplayText =$PhraseToSay
Sleep(2000)

That's it. The bottom line is, whatever you want to go to the display, you set $DisplayText to that text. Anywhere in any script at any time. That's all. The SendToDisplay script will automatically take care of sending that text to the external display.

Also the Sleep(2000) statement could be placed in the SendToDisplay script. It already has a Sleep(500) statement. Just change it to Sleep(2000).

OTOH, if there will naturally be at least 2 seconds between sending text to the display, then you can leave it out entirely.

Now, while I'm on the topic, let me address another point concerning the 2 second (or longer) sleep time between individual strings being sent to the external display. This could become annoying when the reader is having to read more than one line for a given response. If you wish to send multiple lines to the display and they all need to be read in quick succession, the best way to do that is to send them all at the same time. This is what I mean:


Say("Sentence 1")
Say("Sentence 2")
Say("Sentence 3")
#There would need to be at least 2 seconds between for the display
$DisplayText ="Sentence 1")
Sleep(2000)
$DisplayText ="Sentence 2")
Sleep(2000)
$DisplayText ="Sentence 3")
Sleep(2000)

That pause between what should be a rapid succession of sentences could be annoying. So it would be best to concatenate them and send all at once, like this:

 
Say("Sentence 1")
Say("Sentence 2")
Say("Sentence 3")

$DisplayText ="Sentence 1. Sentence 2. Sentence 3" )
Sleep(2000)

  #Or, using variables:
$Sentence1 ="Sentence 1"
$Sentence2 ="Sentence 2"
$Sentence3 ="Sentence 3"

Say($Sentence1)
Say($Sentence2)
Say($Sentence3)

$DisplayText =$Sentence1+$Sentence2+$Sentence3
Sleep(2000)

The display could be setup to show the sentences like this: Sentence 1. Sentence 2. Sentence 3.

Or it could break them at the periods and show then like this Sentence 1. Sentence 2. Sentence 3.

Even put a blank line in between: Sentence 1.

Sentence 2.

Sentence 3.

All that could be done via setup options.

Something I just thought of To better display the sentences in a timely manner it may be best to send them to the display BEFORE saying them. That would probably help with the sleep issue as well, since it would probably take a couple of seconds to say the sentence(s), so no Sleep(2000) statements would be needed.

EDIT You're using Say instead of SayWait so you would need the Sleep delays after all.

One advantage of using the Sketch Pad option is that there would not be an issue of needing sleep periods between the sending of text lines to it.

United Kingdom
#39  

Thanks for getting back to me. I see where I was going wrong now. So every Say() or SayWait() script needs the $DisplayText variable in it too. Any ideas on how this could be achieved with the Pandorabot control as well, as there is not a scripting option for this control?

#40  

@SteveG I had another look at the Pandorabot control but there just seems to be no way to get at the returned text from the Pandorabot site. One would have to figure out a way to intercept it as it comes in to the control or as it is displayed to the debug window. Perhaps there would be a way using the SDK, but it would require a knowledge of this control's operation. That is probably not a part of the SDK. Nor would it likely be a part of the new Plug-in system. The only other ways I can think of offhand to get to it would be:

  1. Somehow cajole DJ into putting a variable in the Pandorabot control which would contain the text send from Pandorabot (like is done with the $SpeechPhrase variable). This would be the quickest and most straightforward route and should be relatively easy for him to do, but that task is much like getting a glacier to move by pushing from the back end, being as swamped as he is. I'm surprised this isn't already a part of that control actually.

  2. To write a plug-in which would perform the same function as the current Pandorabot control but allow for the returned text to be placed into a variable for the user to send to a display other than the Debug window.

  3. Write a plug-in which would be able to get at the text of any debug window.

  4. Write a totally separate program which would provide a user interface for the users current Pandorabots and send the text to the computer's speech generator or to the ARC to be sent on to the robot's speaker.

  5. Write something which would look at the data stream going to and from the internet on the user's network and parse out those parts coming from Pandorabot for display.

None of those options are easy or something I could do quickly since I am not versed with the SDK or the plug-in system. Nor am I familiar with the Pandorabot's system of coding. Though it doesn't seem to be too difficult.

Perhaps starting a new thread focusing on just the Pandorabot control and it's lack of a text variable like the Speech Recognition control has would be the best course of action for now. No one seems to be very concerned with this thread anymore but the two of us.

United Kingdom
#41  

@WBS.

My brain is refreshed today do I gave your program another go... And I t works great. Nice job indeed, and thank you for your work on this. I am very grateful indeed. I think I was over thinking things last night (as well as being really tired) as its a lot easier to display text than what I thought. eyeroll

If I could make a couple of suggestions though if that's cool.

One would be to be able to change the background colour or add an image so it blends in with any set ARC wallpapers that might be set.

And the other, not so much of a suggestion but more of a question, is there a way of adding a script to bring the display to the foreground? When the display is up but you then use an ARC control, the display is pushed to the background (as Windows does), so having a line of code to bring the display back to the foreground after another computer/robots response would be great. I can't think how to do that with a script.

In regards to the Pandorabot control, I thought that might be an issue. But like you say, it shouldn't be that difficault to add as the returned text is already displayed in the debug window. I don't think it's worth starting a new thread as we already have DJ's attention. I hope he see's this and think it's worthy of adding a vairable for this. I really hope so, as this would be quite an important addition for me personally (as I'll mention below).

I am surprised that there has not been much interest or input from other members on this entire subject. For such a simple tool, I can see so many uses for this like for kids education in robotics where some children my be hard of hearing and/or poor vision. This would be a great way to get this kids to interact with a robot platform. And, as in my case, family members who are hard of hearing who want to interact with a robot, for example, a scripted reminder "it's time to take your pills" would be better as large colourful font displayed text where speech syntheses wouldn't be adequate for this. I appreciate this may not be suitable for every robot, especially ones without onboard computers, but as ARC has to use a computer, then the option would be there for any type of robot using the EZ-Robot platform.

There are many more examples I can think of, and I'm sure there are other current or future members who are hard of hearing who would appreciate having this ability, but I guess the lack of responses here says otherwise which surprises me a bit.

BTW, I haven't tried the EZ-Display on my tablet yet, but I will do later today and let you know how it goes (although I can't see any ussurs using it in portrait mode as the window can be resized).

PRO
Synthiam
#42  

The colors are which are words, ie red, green, etc are universal computer colors. You can find that by searching c# color names. Or html color names. Those are the two most popular resources I use.

As for using RGB values (red, green, blue). The colors are numbers between 0 and 255. You will find programming languages and html use the same. The alpha is transparency. It is also a number between 0 and 255. I'm on my phone, but I'm certain there was an example of using rgb in the sketch pad example project. If not, I'll add it.

As for your ez display program... Awesome. May I suggest converting it into a ARC skill plugin? Take a look at the plugin howto and there is also source code for the new plugins that I had created in the list.

United Kingdom
#43  

@DJ.

WBS done great didn't he? I agree with you that EZ-Dislay should be available as an ARC plug-in, as it's deffintaly worthy of being one.

I don't know if you read my last post (#42), but would you concider adding a vairable to extract the response text from the Pandorabot control to display on WBS's EZ-Display?

#44  

@DJ Sures Thanks for your reply. At least that confirms what I wrote back in post #35. I also listed the other colors back there.

There are 2 examples of using Sketch Pad but they only address a couple of points concerning it's use.

I would like to convert it to a plug-in but I still have a steep learning path to working with C# and the MS programming environment. Not to mention the fact I am averse to working with anything which has the letter "C" in it. :D It's difficult for me to work up the enthusiasm to do it at the moment. Thus my penchant for using a text file interface and going with what I am familiar. The only problem is that the text file interface is slow. Considerably slower than I expected even.

What I would like to see in ARC is a way to load a script from a text file. That would open up all sorts of possibilities by itself.

@SteveG I can put in a background "color picker" option, no problem. Putting in a background image would be a bit more difficult however. I'll look into it.

Also, no problem on making the display stay on top of other windows. There is a setting especially for that in windows. I'll put in an option for that as well.

Making it do that from a script should also be possible by allowing for some control characters to be recognized. For example, you could send the character sequence "<StayOnTopOn>" to the display either by concatenating it to the beginning of a string you want to send, or simply by itself like this:


$DisplayText =&quot;&lt;StayOnTopOn&gt;&quot;+&quot;Sentence to send to display&quot;
  #Or
$DisplayText =&quot;&lt;StayOnTopOn&gt;&quot;
Sleep(2000)
$DisplayText =&quot;Sentence to send to display&quot;

  #Using a varible:
$StayOnTopOn =&quot;&lt;StayOnTopOn&gt;&quot; #Declared somewhere like in an Init script
$StayOnTopOff =&quot;&lt;StayOnTopOff&gt;&quot; 

$DisplayText =$StayOnTopOn+&quot;Sentence to send to display&quot;
  #Or:
$DisplayText =$StayOnTopOn
Sleep(2000)
$DisplayText =&quot;Sentence to send to display&quot;

To turn that off you could send: "<StayOnTopOff>"

Concatenating the On-Off code to a string you wish to send anyway is the best method since it eliminates that extra Sleep(2000).

In a similar way the other options could be changed in value via text sent from the ARC making the Display program fully remote controlled. Doing that was the next phase of this project I had planned anyway.

Then there is the two-way communication as suggested by Justin. Also quite feasible.

PRO
Synthiam
#45  

I'll add the pandorabot response to a variable.

United Kingdom
#46  

@DJ.

Thanks for getting back to me and thanks so much for adding that. Having that vairable will be very useful indeed, and I for one would be very grateful to you for that. :)

@WBS.

Okay, that would be great. In regards to having the display in the foreground,I figured there would be a command for keeping there, but for the life of me I couldn't figure out how to do it. Having it as an option would be a great addition.

Again, great work with EZ-Display (I like the name too). :D

Steve.

United Kingdom
#47  

@WBS.

I just tried EZ-Display on my Acer W3... It worked great, better than I expected in portrait mode actually. Half the screen is ARC, and resizing the display fills the other half of the screen with good visibility when using the font size option you added.

Thanks again for this. I'm really pleased you are part of the community. :)

Steve.

United Kingdom
#48  

@WBS.

One last little suggestion I wanted to add, and that is having a line space inbetween each of the robots responses in the display.

PRO
Synthiam
#49  

Get the latest ARC and install this plugin: https://synthiam.com/redirect/legacy?table=plugin&id=17

That's an example of how easy it is to create a plugin.

@WBS source code is available as well

United Kingdom
#50  

Okay DJ. I'll check it out. Thanks.

#51  

@SteveG Thank you for the kind words and you are more than welcome. Glad to help. Thanks for the video as well. It was great to see it working on the tablet. I was happy to see it automatically change width appropriately when being turned from Landscape to Portrait modes. I wasn't sure about that. Now that you have a Pandorabot variable, (even a response script) the last piece of the puzzle falls into place. That's excellent!

I have gotten the control code for having the window go to stay on top mode (and setting it back to normal) to work ok. Also, it is an option via the options menu. I plan to have similar control codes for the other functions as well. I'll make up a list of variables set to the appropriate values that will be initialized when the SendToDisplay script is first run. You will be able to refer to that script for the names as needed. Also, I'll have an option for placing a blank line between the text lines.

Additionally the color picker for the background is installed. You can choose from a set of predefined colors and create up to 16 custom ones. As with the other options, all that will be saved upon program termination and reloaded upon restarting the program.

Finally I am going to go ahead and enable the hide buttons feature so as to hide the two front buttons and the bar upon which they are mounted. I should be able to initiate that by double tapping in the display area. It would toggle from visible to invisible, giving you only a nice clean display area. That means the Clear button will be gone too, but I can make it clear the screen by a single tap. Unless you can simulate a right mouse click with your fingers somehow.

Once I have those things going well, I'll upload the new updated executable file, as well as, a revised SendToDisplay script so you can use those features soon.

United Kingdom
#52  

@WBS.

That's great. I posted the video as I guessed that you might like to have seen a quick demo on the tablet. Yes, I'm so happy that DJ added the Pandorabot vairable (and so quickly too), and the added bonus of the script editor as well which comes in useful for the text displays (your's and the plug-in DJ added to ARC too). The other features you mentioned will be a great addition to EZ-Display. I look forward to the update.

#53  

There are some files attached which are explained at the end of this post.

I finally have the newer version of the EZdisplay program done. The changes are:

  1. Alphabetical arrangement of the selections in the popup menu.
  2. A debug option added just in case.
  3. An "Extra Linefeed" option for blank line between text inputs.
  4. A background color select option.
  5. The "Show Buttons" option has been enabled.
  6. A "Stay on Top" option has been added.
  7. Clearing the display area by clicking/tapping it once.
  8. Showing/hiding the Set and Clear buttons with double click/tap
  9. A start on the remote control commands functionality.
  10. A remote control option to make the Display visible or invisible.

Taking these one at a time:

Debug option. This, when selected, generates a Debug.txt file in the same place as where the EZDisplay.exe file is located. It will allow me to analyze problems should they arise.

An Extra Linefeed option. This is simply for the blank line between text lines from the SendToDisplay script.

The Background Color option for selecting the desired color of the background of the display.

The Show Buttons option. This makes the "Set" and "Clear" buttons, and the strip they are on, visible or invisible. This option can also be set and reset with a mouse doubleclick or touch doubletap.

Usually the clearing of the display can be done manually by clicking/tapping the Clear button on the display. But, since the Set and Clear buttons can now be hidden as desired, a way was needed to bring them back. Usually this would be done by a mouse Right Click. No mouse on a touch screen, so I added a doubletap option to bring the buttons back. The doubletap will toggle the visibility of the buttons as desired. Additionally, the screen can be cleared by a single tap to the display area. When you doubletap to hide the buttons, the display area will also clear, however. I don't think that will be a problem. If it is, there is a way around that but will make the code more complicated (and possibly prone to errors) so I didn't employ it. If there comes a need to keep the text on the screen when doubletapping let me know and I will do that.

The stay on Top function. When selected, the display will stay on top of all other windows regardless of what they do. This function can also be set/reset via sending a remote control sequence. When the $StayOnTopOff is sent via remote control, the Display will probably immediately go behind another window.

Remote Controls These are sent via the SendToDisplay script as desired. Currently, the Remote Controls are:


$StayOnTopOn   =&quot;&lt;StayOnTopOn&gt;&quot;
$StayOnTopOff  =&quot;&lt;StayOnTopOff&gt;&quot;
$ClearScreen   =&quot;&lt;ClearScreen&gt;&quot;
$WinNormal     =&quot;&lt;WinNormal&gt;&quot;
$WinMaximized  =&quot;&lt;WinMaximized&gt;&quot;
$WinMinimized  =&quot;&lt;WinMinimized&gt;&quot;
$WinVisibleOn  =&quot;&lt;WinVisibleOn&gt;&quot;
$WinVisibleOff =&quot;&lt;WinVisibleOff&gt;&quot;

They are sent either by themselves or in conjunction with a line of text:


$DisplayText =$StayOnTopOn

    #Or
$DisplayText =$StayOnTopOn+&quot;Sentence to send to EZDisplay&quot;

More Detail:


$WinNormal     =&quot;&lt;WinNormal&gt;&quot;
$WinMaximized  =&quot;&lt;WinMaximized&gt;&quot;
$WinMinimized  =&quot;&lt;WinMinimized&gt;&quot;

These are associated with the usual functions accessed via the negative sign (-), the square, and the X found at the top right of most windows.

$WinNormal: This is the usual state for the window depending on what size it was last set to.

$WinMaximized: Fills the screen with the Display program.

$WinMinimized: Reduces the window to an icon at the bottom of the screen.

$WinVisibleOff Makes the Display invisible. Unlike $WinMinimized, it is still there so when it it is made visible again, It just appears. This, instead of flying out from it's minimized state.

$WinVisibleOn: Makes the Display visible again. NOTE: $WinNormal, $WinMinimized, and $WinMaximized will also make the Display visible again.

Generally, the $WinNormal, $WinMinimized, $WinMaximized, $WinVisibleOff, $WinVisibleOn will be sent to the Display by themselves. They will still need a sleep period between them when they are.

The attached files are: A new executable (EZDisplay.exe) and new SendToDisplay script containing the remote control variables. Simply overwrite the old executable wherever you put the original one in the install process, and copy and paste the contents of the SendToDisplay file into the SendToDisplay script in place of that is there now. NOTE: Leave the first line as it is and overwrite the rest.

Let me know how it goes.

EZDisplayFiles.zip

United Kingdom
#54  

@WBS.

Many thanks for the update and for adding the upgrades. I haven't had a chance to download and try the update yet, but once I do, I'll let you know how it performs. Over the weekend, I had a great idea for using EZ-Display on my current project which should be fun. :)