Anyone interested in discussing building DIY Home Automation software and hardware?

[tattema]

Here you go:
 
Firmware Drivers V2.1.0 from Semtech.
 
These drivers are written in ANSI C so should run on an Arduino without much modification. If you get stuck, just let me know and i'll be glad to help.
 
Listen mode isn't covered in the library but it can be added. The big difference between the manufacturer reference drivers and the standard RFM69 drivers floating around is quality, style and approach. They use a finite state machine engine to track the different states the silicone is in so it's quite robust and consistent. As expected, the drivers are functionally low level.
 
I'm surprised that HopeRF don't supply these libraries :-(
 
Implementation tips:
- In addition to the MISO, MOSI, CS, CLK pins for SPI, you'll need to map 5 INT pins to IO pins and a reset pin to the device. The implementation is such that it polls the pins rather than waits for interrupts so the INT pins can be wired to any available GPIO.
- It's important that the reset pin be mapped as the driver overcome startup issues found with other implementations by resetting the chip in a controlled manner.
- The Radio structure is key and can be expanded to expose additional driver functions. I found the functions exposed through this interface to be a bit limiting.
 
You'll notice a huge difference in performance. Firstly, you'll be able to sent huge packets without the 63/4 byte limitation (encryption engine limits still apply) imposed by the current JeeLAB and Motino implementations. Secondly, the driver doesn't try to handle receive from within an interrupt.... once again poor design by Motino/JeeLABs. The detection of a preamble means you can reduce packet collisions. 
 
Enjoy

 

[tattema]

Thanks once again tattema for the good info.  FYI, from what I've been reading recently,  RFM69x is most similar to SX1231H. Also, I can confirm that the RFM69x library has incomplete coverage of the chip features, with the workaround being that you can set registers on your own.  For instance, the "Listen-mode" has no library coverage and fits that scenario.  So, if the library that you're referring to really is that much better, I'm at the point where now would be a good time to switchover to it.  Do you have a link to the one that you like, and will it easily work on the Arduino platform?

any updates?

 

[NeverDie]

any updates?

 
Hi Tattema,
 
It turns out that over the course of just the last month or so a few members on the lowpowerlab forum have uncovered very efficient ways to exploit the listen-mode of the radio, yielding on the order of 10,000x reduction in power consumed versus staying in Rx alone.  My conclusion is that the ATMEGA328P plus RFM69x combination is turning out to be good enough for my simple needs.     I think 10 years battery life (maybe even longer) is here-and-now achievable while still doing useful work, and I don't know that an MCU that's even more power efficient would extend that by much, given that the self discharge rate of the battery itself starts to become a governing limitation.       

 

[deandob]

Haven't been posting here for a while, been busy with my Raspberry Pi Windows 10 IoT wall mounted 5" touchscreen (the hardware for the touchscreen ended up harder than expected) and now waiting on parts. As part of the investigation into the RPi touchscreen I worked out that I needed a small USB micro to interface between the touchscreen controller IC and Windows to emulate a HID mouse (single touch only but sufficient for my needs initially) and will be using the PIC16F1455 which is a cheap & small USB PIC (self powered from the USB port, no need for a crystal, simple USB library available).
 
As I also need to redo my old IO board for the front gates automation plus a couple of other things I though I'd reuse the PIC16F1455 to build an IO board and use it to learn about USB on the PIC and Windows as a weekend project. Folks here may be interested about this as it has the following specifications:
- 8 relay outputs
- 8 optocoupler inputs (senses open or closed circuit) with ESD protection
- isolated 5v power supply that can be switched in as the power source for the optocoupler inputs
- Isolated digital IO for 1 wire sensors
- USB enumerated on the host as a serial port (serial class for Windows, I assume the same for Linux but I'm not a Linux user)
- Serial communication is similar to MQTT topic formal and my home brew automation software (<port#>/<1 or 0>) so easy to integrate into existing systems
 
I'm going to use one of the cheap Chinese board houses (probably ITead) which will get me 10 boards and I will only need a couple, so if this solution is of interest I can post out the spare boards (+ postage costs).
 
Schematic below. I can upload the Kicad schematics, PCB and gerbers to GitHub as well.
https://onedrive.live.com/redir?resid=850A95A2C4EB0195!48294&authkey=!ABUrODrLmjOjqTo&v=3&ithint=photo%2cPNG

 

[pgray007]

I only got through the first couple of pages, but I'm interested in your approach. What I see as missing in HA is applying modern architectural standards, and essentially putting all the devices behind a middleware layer that exposes a standard REST API. Then you can get out of the business of front end designers, which always seem to stink, and just use a standard mobile framework like Ionic or similar that calls your automation APIs, which then hit the device APIs.

If this was open source, JavaScript/Node based, and readily leveraged other tools you might be able to get some interest in the dev community since HA is "hot" again.

 

[deandob]

Hi pgray.
 
Modern architectural standards still have their pros and cons, like the older approaches used by the more established HA players, just different set of compromises. For example, REST / HTTP isn't best for chatty long term bi-directional communication which is why I use webSockets for the front end (although also support REST) and Node.JS isn't great at highly computational tasks which is why I use .NET for the speed and multi-threaded ability for the HA engine (although developing plug-ins with Node.JS is considerably easier than .NET).

 

[freedom]

Hi all
for our framework we expose xml/json Rest API documented with Swagger (backend in Java) and use websockets to listen to events. 
An online example is available on https://github.com/freedomotic/freedomotic/wiki/Freedomotic-APIs
 
The web interface is developed using polymer 
https://github.com/freedomotic/freedomotic/wiki/Demo for a prototype live demo 
 
All source code is available on https://github.com/freedomotic/fd-polymer-webapp
 
With polymer you can create your own web components and reuse them in different projects. For example with Apache Cordova you can develop mobile apps with html, javascript and css. With a single base code you can target different platforms.

 

[deandob]

Still can't access your demos (using Internet explorer).

 

[freedom]

Which IE version?
Chrome, Safari and Firefox are OK. With IE we have some problems because it doesn't support well HTML5.
 
Let me know. We are trying to test on a large number of platforms to identify possible leaks.
 
Thanks

 

[deandob]

IE11 and the Windows 10 edge browser. Both support HTML5 although the edge browser is better at HTML5 compliance. Windows browser compatibility should be a major part of your testing - netmarketstats has IE9 and above as 33% of the browser market.

 

[Dean Roddey]

BTW, it's quite possible to have a persistent HTTP connection where the server just continues to send messages without being queried. So it's basically two way HTTP that supports async reporting. The ISY uses this scheme and it works fine. Your HTTP client side infrastructure has to support it, but we have our own so it was easy enough to support that type of thing.
 
Websockets is nice, but it's very javascript oriented, i.e. it's a completely callback oriented system, so it's very difficult to do a blocking call. It turns into a much more complex stateful scheme, so you lose some of the simplicity of Websockets in any setup where you need to send something and wait for a specific response back (very, very common in device control.) You can't just sit there and wait because you won't get another response until you return back from the previous callback and get another callback. You can fake it, but at the cost of a lot more complexity.
 
Non-callback oriented schemes don't have that issue.

 

[deandob]

Dean, yes, if you want to implement a generic HTML5 / Javascript client as the front end then websockets is the best approach (assumes all browsers are HTML5 compatible which should be a safe bet now days). The techniques you are talking about historically have been hacks to get around the HTTP session limitations (due to its statelessness) but are still useful in the right context (SignalR is a good example).
 
Async does add some complexity but for the client end I never had a problem with that (when needed I use simple state machines) however I have had to deal with extra async complexity with node.JS when writing more sophisticated plugins. I will say coming from a C/C# background I do like the simplicity of javascript as it does make development easier due to its more flexible approach to casting and prototypes (eg. in my .NET HA engine where I have been careful to implement a layered design it is tedious having to cast all the variables being passed between the layer functions) but to write really good Javascript takes experience due to the async nature and features like closures. However dealing with the vagaries of different browser HTML implementations does claw back some of the advantage that Javascript provides.

 

[deandob]

As I have some spare space on the IO board I'm making that I posted about a couple of posts up, I thought I'd add a radio (even though I prefer my sensors wired) as a permanent part of my HA solution as previously I had just experimented with the different wireless option used in HA.
 
So given the interest in this thread about wireless I thought I'd put out here to get your opinions on the 'best' wireless option for HA. Here is my use case:
- Temperature sensors for indoor & outdoor
- Water flow sensors from the water meter outside the house
- Ambient light sensor
- proximity sensor
- Low power (replace battery not more than once per year)
- Physically small
- Use 3rd party automation devices like wireless doorlocks.
- Distances indoors over 25m & outdoors another 10m (large home / land)
- Can use MQTT ideally for interface but not critically (but have the ability to write a driver plugin)
 
So as I look at the various wireless options I am arriving at the following conclusion:
- Bluetooth LE is fast becoming a favourite for HA but doesn't have great range (through multiple walls) or meshing ability. Long battery life and easy use for proximity detection. I can use a Bluetooth dongle on the PC HA hub with a Windows driver so no need to add it to the IO board.
- Wifi is ubiquitous and I have a couple of wifi routers around my house, so can use it for rough proximity detection but hasn't got the range for outdoors and isn't low power. Anything I use on wifi would talk MQTT over sockets directly to the HA hub so no need for a WiFi radio on the IO board.
- Z-Wave has support for a lot 3rd party HA devices, low power, mesh, doesn't seem to be easy DIY options for devices (ZM5202 modules look the easiest), there are open source drivers on Windows supporting cheap dongles - so no need to add to the IO board.
- ZigBee is also mesh, not as good 3rd party support, devices tend to be more expensive than z-wave, better device DIY and low power use. But like the ones above would be easier to handle in Windows / Linux rather than on a PIC microcontroller that then interfaces with a hub.
 
So basically I'm looking for less sophisticated radio solutions (ie. without the 'full stack' that the wireless solutions above offer) for small low powered sensors that I would DIY (eg. a small standalone temperature sensor). So that leaves the 2.4Ghz and sub Ghz solutions from TI (eg. CC110L for 915Mhz), Nordic Semi (I like these as they are easy to work with), Si Labs (Si4468 posted earlier in this thread) and a couple of others. I'd prefer the sub Ghz solutions due to the longer range, no need for high transmission rates and no wifi interference. I have 3 IO pins left free on the PIC so at this point I'm thinking the Nordic NRF905 fits the bill as I can squeeze the SPI control lines into the rest of the design (shared clock).
 
NeverDie, Tattema, both of you are much deeper into the radio technologies than I am - any advice on which chip / radio transceiver (ie. not SoC as I have the PIC already) technology is best suitable for my requirements given the above and before I hack away at my Kicad PCB design?
 
Also for those of you interested in the ESP8266 there is a new version coming out soon with lower power use, support Bluetooth LE and more CPU power. See here for more info: http://www.cnx-software.com/2015/08/03/espressif-is-working-on-a-new-wi-fi-and-bluetooth-le-chip-with-hardware-ssl-support/

 

[deandob]

Regarding my post above, did a bit more research on suitable sub-Ghz radio solutions for small sensors:
 
Nordic Semi NRF905 - quite common, cheap, well documented on the net, many modules on eBay / Aliexpress (although the module makers market these as compatible with 433Mhz as well as 905Mhz with no RF component changes - sheesh...).
Rx sensitivity: -100dBm, 12.5mA
Tx sensitivity: 10dBm, 30mA (-10dBm 9.5mA)
Standby: 135uA
 
TI have a good range including a number of high performance chips (-127dBm) that are a bit more expensive and fairly rare, or the more common CC1101 which still has good specs and is common like the NRF905 with good support
Rx sensitivity: -116dBm, 16mA
Tx sensitivity: 12dBm, 34mA
Sleep: 200nA
 
Silicon Labs Si4468 is the champ for receive & tx sensitivity at -133dBm but isn't a popular chip however is documented enough for DIY
Rx sensitivity: -133dBm, 13mA
Tx sensitivity: +20dBm, 70mA (+10dBm 18mA)
Standby: 40nA
 
Clearly the Si4486 is the winner here as well as the cheapest at my latest supplier of choice RS-Components (their prices are more competitive than mouser / digikey and quite often better than aliexpress for most things, massive range, excellent web site, 2 day delivery for free & 10 minute drive if I want to pickup but why bother).

 

[NeverDie]

NeverDie, Tattema, both of you are much deeper into the radio technologies than I am - any advice on which chip / radio transceiver (ie. not SoC as I have the PIC already) technology is best suitable for my requirements given the above and before I hack away at my Kicad PCB design?

 
I've been through this exercise already, and for me the answer turned out to be the RFM69(H)W.  I explain why here:  http://cocoontech.com/forums/topic/28256-anyone-interested-in-discussing-building-diy-home-automation-software-and-hardware/?p=237172  The chip on the module is reputedly genuine Semtech.
 
For a low cost yet easy solution, you can buy an RFM69  module for $4 and hook it up to a $2 Arduino 8Mhz 3.3V Pro Mini with just 5 wires.  It really is that simple.  As to the software, that trail has already been blazed, so you aren't starting from square 1.
 
Also, if you have a radio with great range, you needn't bother with mesh networking, so that also helps with keeping things simple.  When it comes to HA, simple is good.  
 
As long as you're considering the entire landscape of options, you might want to consider some of the LoRa solutions as well.