I wanted to start a discussion on my goal to have a smart house with a Jarvis (Ironman) / Sarah (Eureka) like AI and my little EZ-Builder robot servants roaming around.
So feel free to jump in and contribute...
I have been using a combination of open source programs with my EZ-Builder software and EZBv3 boards. EZ-Builder is the brains of my system and provides me with the intelligent front end that responds to voice commands and communicates to my four EZB-3 boards. Lawrence, as I like to call it, replies to my voice commands using the Ivona Brian voice file. Sound very like Jarvis.
After recognising the voice command Lawrence selects what to say back to me from a pool of scripted phrases. This addresses repetitiveness in responses and adds a nice human-ish touch.
At the same time Lawrence is replying several scripts are firing commands to other applications or devices through an MQTT broker that I have installed. This Broker, from the Mosquitto open source project, acts like my AI's nervous system or neural net. I store values that are stored in the broker, similar to database that get reloaded to my various devices when they wake up.... I find this easier than an a database.
Various other devices are subscribed to the broker topics and act appropriately but you must have your own broker installed. I run mine on a separate Raspberry Pi elsewhere on my network. I keep it as a stand alone device as everything connects to it so if I need to reboot my PC it does not cause problems. It is fast, works well on the newer Pi3 and supposedly can handle 100,000 posts per second... I have only rebooted it a couple of times since installation and that was only because I wanted to move it.
So I post directly from EZ-Builder to my MQTT neural net with the EXEC command like this:
exec("C:\Program Files\mosquitto\mosquitto_pub.exe", "-h 10.1.1.120 -p 1883 -t /HOME/EZ-BUILDER/COMMAND -m Command:Lounge_Lights_On' ")
I also use a second program called EventGhost that watches for those MQTT messages and fires commands to other devices on my network that EZ-Builder can't talk to. I complete this action by triggering events in EventGhost with the same exec command in EZ-Builder:
exec("Eventghost", "-event Guest.Welcome.Video.Fullscreen")
exec("Eventghost", "-event It's_Spa_Time")
Another big player in my system is Rainmeter which adds great visuals to my screen. It lets me change the entire look and feel of the PC. While not tied directly to EZ-Builder I can launch EZ-Builder from a fancy picture on my desktop. This simple script in Rainmeter simply launches EZ-Builder when I click it. (I know I could use a shortcut but this looks way cooler)
License=Creative Commons Attribution-NonCommercial-NoDerivs 3.0
LeftMouseUpAction=["C:\Program Files (x86)\EZ-Robot Inc\EZ-Builder\EZ-Builder.exe" "C:\Users\Wayne\Dropbox\My Robotics\Launcher.EZB" "Autostart"]
And of course OpenHAB is the most recent player to come to my table. As it can communicate with the same MQTT broker I am now extending my domain out from my workshop to the rest of the house. OpenHAB is giving me a nice web front end with switches and buttons, status updates, etc. This means EZ-Builder can now turn on my lights through MQTT from a simple voice command.
So friends .... My goal of Home automation with a responsive AI and little robot servants roaming the property is slowly becoming a reality thanks to EZ-Builder.
What have you been doing?
Just got my ez dev kit today and just started messing with it. Not used to using gui based environments for this kind of thing but thought this would be the "ez" way to get started with bots and would be good for my 7 year old also.
I used eventghost some years back but openHAB can handle anything that you would need it for with the exec binding, so it may be a good avenue for you to consolidate your software based automation.
If you don't have an amazon echo yet, you should definitely consider picking one up as the voice recognition is really superb and ties in nicely with openHAB or anything else using http or POST by using the "hue" emulation bridge. Great for lighting, tv, alarm, doors, etc. I am looking forward to checking out ez's as well for the bot control.
I too envision tying the ez bot(s) into the home "nueral" net as you say, and MQTT seems to be a leading model for connecting nodes. I'm already using to connect arduino nodes to my openHAB server for things such as relay triggers, sensor states, RGB commands for strip lighting, etc. The reading I have done on ROS seems to show this as the form of communication also between all of the nodes in the bot. I already have mosquitto running on my openhab server so I thought it would be cool if ez would ever integrate an mqtt client. But as that probably doesn't exist I plan to figure out how to use http instead.
Also, speaking of a front end for openhab, there is a pretty cool project in alpha currently that you can register for called "rotini", runs on android and is widget based and is very visually appealing, especially for cheap wall controllers.
I also have a HAI home auto panel that handles my security, fire, UPB lighting, but I leave the zwave to a vera controller, again all is integrated in openHAB too.
I'm waiting on my omnibot(s) to arrive so I can get to the teardown and rebuild but will be looking for ideas when I get to putting some projects together with ez builder
@Tameion, you are doing some fantastic work here! I would love to see a video of your system in operation as it sounds like it is amazing!
I'll work on that. It's been a couple of years have passed since I started in this direction. Others had been using MySQL as a database. I had used text files I loaded and unloaded. but I wasn't happy with either.
I solved my dilemma by using an MQTT Broker. I monitor this stream of data and use a Python script that @rich gave me to post data from EventGhost to EZ-Builder using the Terminal Port.
I also have EZ-Builder posting back to EventGhost with a single line of code. EG is of course in the system path on the PC. The command looks something like:
exec("Eventghost", "-event eventname payload1 payload2 payload3")
An actual example would be:
exec("Eventghost", "-event EZ-Builder_Terminal.Connected")
I could add the payload but don't usually bother as the event itself launches a macro. I also echo the event to my MQTT Broker feed just incase I have a device later on that may need to subscribe to this information.
So I now have a constant MQTT feed monitored by EventGhost. When it finds a relevant event it fires a message off to EZ-Builder for actioning. EZ-Builder in turn can fire events off to the MQTT stream for anyone else to action or ignore.
This way devices that don't play nicely together and refuse to talk to EZ-Builder can just post their data to the MQTT feed and EventGhost can happily watch the events until something triggers a macro that fires an update to EZ-Builder.
@Tameion thanks for the thread link! I am checking it out and looking how I can implement for my case and I will get back. I really appreciate it.
I still think your use of mqtt is more what I am looking for but Rich's python example is helpful for some other possibilities. I'm trying to figure out how to handle my mosquitto instance since I do not host it on the same machine as EZ-Builder.
Edit: Ok I installed a windows based mqtt client tool called mqtt.fx which I got connected to my mosquitto broker in about 30sec, so once I figure out how to pass the pub command to this I should be able to use your publish examples to get everything talking.
mqtt.fx does not seem to have cli available so I installed a second mosquitto install to use as a client pub/sub on the EZ-Builder pc. Working good for pub with your code
exec("C:\Program Files\mosquitto\mosquitto_pub.exe", "-h x.x.x.x -p 1883 -t /SAMPLE_TOPIC/EZ-BUILDER/COMMAND -m Command:Sample_Command' ")
I am wondering if ez-robot might consider an mqtt client "plugin?" for easier integration in the future. I think MQTT has become the preferred cross platform protocol in the M2M and IoT space from everything I have seen in the last couple of years. It could bring additional value to the product out of the box.
Now looking for the best way to receive commands/topics since I am not using eventghost. I am looking at setting up some openHAB rules to handle and send http commands instead.
Thanks again, I will get back with any further progress.
I know that you get hundreds of requests for additions to EZ-Builder but would you consider having another think about adding an MQTT client to EZ-Builder?
@Tameion thanks for putting this forward to @DJ, this truly would be an excellent method for "connecting it all together". I can imagine that they are flooded with feature requests all the time, but I can see this one also making sense for their target demographic since mqtt can be easily handled in the educational space with a simple subscribe/publish of easy to understand "topics" and the ability to connect one project to another.
Alan, you bring up a great point. I think all kinds of great mods come from active community development, I hope to find alot of cool plugins once I get going here...
In this instance I suppose I was looking at MQTT support as a request to support an actual protocol, just like HTTP, etc. I thought that protocols tend to be implemented and maintained with the product to ensure continued and reliable functionality through the product life cycle.
In IoT and node to node communication, II think MQTT is tending to be the preferred protocol now due to its simplicity and lightweight, that's why I thought it would be great in general for the product, not just very specific users.
I am not a C# programmer either and most of my scripting has been linux based but maybe I could try to see what's involved (very limited JS, C, python), although I am not sure I am capable. I think most users here are initially attracted to the user friendly approach to get started in robotics here...I really wanted to jump into ROS but after reading much of the wiki I can see a "bit" of a curve, hence EZ-robot.
What is cool is that MQTT could also tie in (I believe) future ROS nodes to the EZ-robot universe, supporting additional progressions in learning and education, but I could be wrong here ...
Anyway thanks for the suggestion, maybe someone else here in the community might jump in if it is of interest and value to them to write a plugin, it would certainly be awesome in my case to "connect it all together"!
I agree that built in has advantages, but this is exactly the kind of function that the plugin architecture is perfect for. Populating variables from external sources, and sending variables or commands from scripts. DJ's todo list, and bug fix list is very long, and he is the only one doing primary programming on EZ-Builder at the same time he is trying to run and expand the company from a few friends in a garage to a real business in the STEM space.
If it peaks his interest, he'll add it ahead of other things on the list, and I have often seen conversations like this drive new features (usually just as I figure out a workaround, he adds a feature....), but I know he is really trying to encourage the user community to learn to program and write plugins where they are a good fit.
I am hoping this fall/winter (once it is too cold to mountain bike on the weekends) to sit down with Visual Studio and get back up to speed on programming. I used to be pretty good with VB6, but I have not been able to wrap my head around dotnet yet, and I also want to learn Java for some work related stuff. I have a lot of plugin ideas I want to create, and this one looks like it would be a snap to complete since the APIs are already in place and well documented.