Hey Everyone...
I wanted to start a discussion on my goal to have a smart house with a Jarvis (Ironman) / Sarah (Eureka) like AI and my little ARC robot servants roaming around.
So feel free to jump in and contribute...
I have been using a combination of open source programs with my ARC software and EZBv3 boards. ARC is the brains of my system and provides me with the intelligent front end that responds to voice commands and communicates to my four EZB-3 boards. Lawrence, as I like to call it, replies to my voice commands using the Ivona Brian voice file. Sound very like Jarvis.
After recognising the voice command Lawrence selects what to say back to me from a pool of scripted phrases. This addresses repetitiveness in responses and adds a nice human-ish touch.
At the same time Lawrence is replying several scripts are firing commands to other applications or devices through an MQTT broker that I have installed. This Broker, from the Mosquitto open source project, acts like my AI's nervous system or neural net. I store values that are stored in the broker, similar to database that get reloaded to my various devices when they wake up.... I find this easier than an a database.
Various other devices are subscribed to the broker topics and act appropriately but you must have your own broker installed. I run mine on a separate Raspberry Pi elsewhere on my network. I keep it as a stand alone device as everything connects to it so if I need to reboot my PC it does not cause problems. It is fast, works well on the newer Pi3 and supposedly can handle 100,000 posts per second... I have only rebooted it a couple of times since installation and that was only because I wanted to move it.
So I post directly from ARC to my MQTT neural net with the EXEC command like this:
exec("C:\Program Files\mosquitto\mosquitto_pub.exe", "-h 10.1.1.120 -p 1883 -t /HOME/EZ-BUILDER/COMMAND -m Command:Lounge_Lights_On' ")
I also use a second program called EventGhost that watches for those MQTT messages and fires commands to other devices on my network that ARC can't talk to. I complete this action by triggering events in EventGhost with the same exec command in ARC:
exec("Eventghost", "-event Guest.Welcome.Video.Fullscreen")
exec("Eventghost", "-event It's_Spa_Time")
Another big player in my system is Rainmeter which adds great visuals to my screen. It lets me change the entire look and feel of the PC. While not tied directly to ARC I can launch ARC from a fancy picture on my desktop. This simple script in Rainmeter simply launches ARC when I click it. (I know I could use a shortcut but this looks way cooler)
[Rainmeter]
Author=Wayne
MiddleMouseDownAction=!DeactivateConfig
Update=86400000
MouseActionCursor=0
[Metadata]
Version=1.0
Information=Phone_Wayne TextItem
License=Creative Commons Attribution-NonCommercial-NoDerivs 3.0
[Variables]
height=60
[noimage]
Meter=Image
ImageName=#CURRENTPATH#\Tablet.jpg
W=(#Height#)
LeftMouseUpAction=["C:\Program Files (x86)\EZ-Robot Inc\EZ-Builder\EZ-Builder.exe" "C:\Users\Wayne\Dropbox\My Robotics\Launcher.EZB" "Autostart"]
[Title]
Meter=STRING
MeterStyle=sTitle
Text=WAYNE
FontSize=9
X=10
Y=55
FontColor=255,255,133,255
And of course OpenHAB is the most recent player to come to my table. As it can communicate with the same MQTT broker I am now extending my domain out from my workshop to the rest of the house. OpenHAB is giving me a nice web front end with switches and buttons, status updates, etc. This means ARC can now turn on my lights through MQTT from a simple voice command.
So friends .... My goal of Home automation with a responsive AI and little robot servants roaming the property is slowly becoming a reality thanks to ARC.
What have you been doing?
Other robots from Synthiam community
Xuven's Megabyte
Ezang's My New Adventruebot Robot
Just got my ez dev kit today and just started messing with it. Not used to using gui based environments for this kind of thing but thought this would be the "ez" way to get started with bots and would be good for my 7 year old also.
I used eventghost some years back but openHAB can handle anything that you would need it for with the exec binding, so it may be a good avenue for you to consolidate your software based automation.
If you don't have an amazon echo yet, you should definitely consider picking one up as the voice recognition is really superb and ties in nicely with openHAB or anything else using http or POST by using the "hue" emulation bridge. Great for lighting, tv, alarm, doors, etc. I am looking forward to checking out ez's as well for the bot control.
I too envision tying the ez bot(s) into the home "nueral" net as you say, and MQTT seems to be a leading model for connecting nodes. I'm already using to connect arduino nodes to my openHAB server for things such as relay triggers, sensor states, RGB commands for strip lighting, etc. The reading I have done on ROS seems to show this as the form of communication also between all of the nodes in the bot. I already have mosquitto running on my openhab server so I thought it would be cool if ez would ever integrate an mqtt client. But as that probably doesn't exist I plan to figure out how to use http instead.
Also, speaking of a front end for openhab, there is a pretty cool project in alpha currently that you can register for called "rotini", runs on android and is widget based and is very visually appealing, especially for cheap wall controllers.
I also have a HAI home auto panel that handles my security, fire, UPB lighting, but I leave the zwave to a vera controller, again all is integrated in openHAB too.
I'm waiting on my omnibot(s) to arrive so I can get to the teardown and rebuild but will be looking for ideas when I get to putting some projects together with ARC
@Tameion, you are doing some fantastic work here! I would love to see a video of your system in operation as it sounds like it is amazing!
Tony
I'll work on that. It's been a couple of years have passed since I started in this direction. Others had been using MySQL as a database. I had used text files I loaded and unloaded. but I wasn't happy with either.
I solved my dilemma by using an MQTT Broker. I monitor this stream of data and use a Python script that @rich gave me to post data from EventGhost to ARC using the Terminal Port.
See Here synthiam.com/Community/Questions/5128&page=1
I also have ARC posting back to EventGhost with a single line of code. EG is of course in the system path on the PC. The command looks something like:
An actual example would be:
I could add the payload but don't usually bother as the event itself launches a macro. I also echo the event to my MQTT Broker feed just incase I have a device later on that may need to subscribe to this information.
So I now have a constant MQTT feed monitored by EventGhost. When it finds a relevant event it fires a message off to ARC for actioning. ARC in turn can fire events off to the MQTT stream for anyone else to action or ignore.
This way devices that don't play nicely together and refuse to talk to ARC can just post their data to the MQTT feed and EventGhost can happily watch the events until something triggers a macro that fires an update to ARC.
@Tameion thanks for the thread link! I am checking it out and looking how I can implement for my case and I will get back. I really appreciate it.
I still think your use of mqtt is more what I am looking for but Rich's python example is helpful for some other possibilities. I'm trying to figure out how to handle my mosquitto instance since I do not host it on the same machine as ARC.
Edit: Ok I installed a windows based mqtt client tool called mqtt.fx which I got connected to my mosquitto broker in about 30sec, so once I figure out how to pass the pub command to this I should be able to use your publish examples to get everything talking.
mqtt.fx does not seem to have cli available so I installed a second mosquitto install to use as a client pub/sub on the ARC pc. Working good for pub with your code
I am wondering if ez-robot might consider an mqtt client "plugin?" for easier integration in the future. I think MQTT has become the preferred cross platform protocol in the M2M and IoT space from everything I have seen in the last couple of years. It could bring additional value to the product out of the box.
Now looking for the best way to receive commands/topics since I am not using eventghost. I am looking at setting up some openHAB rules to handle and send http commands instead.
Thanks again, I will get back with any further progress.
@DJSures
I know that you get hundreds of requests for additions to ARC but would you consider having another think about adding an MQTT client to ARC?
@Tameion thanks for putting this forward to @DJ, this truly would be an excellent method for "connecting it all together". I can imagine that they are flooded with feature requests all the time, but I can see this one also making sense for their target demographic since mqtt can be easily handled in the educational space with a simple subscribe/publish of easy to understand "topics" and the ability to connect one project to another.
The great thing about the ARC skill plugin environment is that we don't need to wait for DJ to be available to add this kind of functionality. Any C# programmer (not me yet, but maybe one day) can write a plugin. There is an MQTT C# library (maybe more than one) that should make the integration quite easy: https://code.msdn.microsoft.com/windowsdesktop/M2Mqtt-MQTT-client-library-ac6d3858 or http://www.hivemq.com/blog/mqtt-client-library-encyclopedia-m2mqtt
Alan
Alan, you bring up a great point. I think all kinds of great mods come from active community development, I hope to find alot of cool plugins once I get going here...
In this instance I suppose I was looking at MQTT support as a request to support an actual protocol, just like HTTP, etc. I thought that protocols tend to be implemented and maintained with the product to ensure continued and reliable functionality through the product life cycle.
In IoT and node to node communication, II think MQTT is tending to be the preferred protocol now due to its simplicity and lightweight, that's why I thought it would be great in general for the product, not just very specific users.
I am not a C# programmer either and most of my scripting has been linux based but maybe I could try to see what's involved (very limited JS, C, python), although I am not sure I am capable. I think most users here are initially attracted to the user friendly approach to get started in robotics here...I really wanted to jump into ROS but after reading much of the wiki I can see a "bit" of a curve, hence EZ-robot.
What is cool is that MQTT could also tie in (I believe) future ROS nodes to the EZ-robot universe, supporting additional progressions in learning and education, but I could be wrong here ...
Anyway thanks for the suggestion, maybe someone else here in the community might jump in if it is of interest and value to them to write a plugin, it would certainly be awesome in my case to "connect it all together"!