Anyone interested in discussing building DIY Home Automation software and hardware?

[pete_c]

Welcome to Cocoontech Trev!
 
Thank-you for sharing your automation experiences with us.
 
Lots of friendly folks here and many doing automation now for many years. 
 
How do you like living in Argentina?
 
Many years ago did some stuff there with Impsat (well they did the doo of a dog and pony show for me which I was impressed with and I did get to test their communication wares which was fun for me).
 
That said thinking Impsat did initially put some communications stuff relating to the beginning of a project (gas) in Peru a few years back.

 

[NeverDie]

LoRA is insane!!! Amazing. It boasts a link budget of 157db which is just unbelievable!!! .... Very cool (and expensive) relative to other radio technologies.

Yes!  Exciting stuff.  Recently noticed this (http://www.ebay.com/itm/Semtech-LoRa-SX1276-SX1278-UART-Interface-RF-wireless-module-DRF1278DM-/191273642230?pt=LH_DefaultDomain_0&hash=item2c88cc28f6 ), which might be simple to use in a basic way because of its UART interface.  I imagine the range should be pretty good, but then again, with one-off China modules, how can you ever really know what you're getting?  It might have the right components but turn out to be junk.
 
In terms of saving energy, recently noticed the Si4463 has a mode where it can listen at 50bps at a current drain of just 10uA, which is kinda interesting.  Alternatively, running very short bursts at much higher data rates (and currents) seems, in general, to be the approach that's getting traction.  The NRF24L01+ may transmit 1mbps at 20ma, but it's typically on and then off (to less than 1uA drain) in under a millisecond.

 

[tattema]

Welcome to Cocoontech Trev!
 
Thank-you for sharing your automation experiences with us.
 
Lots of friendly folks here and many doing automation now for many years. 
 
How do you like living in Argentina?
 
Many years ago did some stuff there with Impsat (well they did the doo of a dog and pony show for me which I was impressed with and I did get to test their communication wares which was fun for me).
 
That said thinking Impsat did initially put some communications stuff relating to the beginning of a project (gas) in Peru a few years back.

 
Thanks for the welcome Pete. Re Argentina, I hate it and love it at the same time. For high-tech projects where you need easy access to components and PCB's the country is in the dark ages and I hate it because import controls mean that things are blocked for import and I can't get what I need. For the family and socially it's great so I love it from that perspective. One thing for sure, it does force one to do more with less!

 

[tattema]

Yes!  Exciting stuff.  Recently noticed this (http://www.ebay.com/itm/Semtech-LoRa-SX1276-SX1278-UART-Interface-RF-wireless-module-DRF1278DM-/191273642230?pt=LH_DefaultDomain_0&hash=item2c88cc28f6 ), which might be simple to use in a basic way because of its UART interface.  I imagine the range should be pretty good, but then again, with one-off China modules, how can you ever really know what you're getting?  It might have the right components but turn out to be junk.
 
In terms of saving energy, recently noticed the Si4463 has a mode where it can listen at 50bps at a current drain of just 10uA, which is kinda interesting.  Alternatively, running very short bursts at much higher data rates (and currents) seems, in general, to be the approach that's getting traction.  The NRF24L01+ may transmit 1mbps at 20ma, but it's typically on and then off (to less than 1uA drain) in under a millisecond.

 
Hi NeverDie
 
That wireless module sure does sound interesting so let us know the result you get if you end up buying it. I'm interested hear all about it, especially the UART interface so I can't help but wonder if they supply the schematics for the board and source code for the interface chip. That alone would be a compelling reason to buy it I think.
 
A properly matched LoRa output stage should give you between 7-14kms, line of sight, transmission range at 1200 bps. That's awesome!!!
 
I've been looking for the info on the Si4463 but cant seem to find anything on the 10uA listen mode. Would you mind posting a link?
 
Regards
Trevor

 

[NeverDie]

Hi NeverDie
 
That wireless module sure does sound interesting so let us know the result you get if you end up buying it. I'm interested hear all about it, especially the UART interface so I can't help but wonder if they supply the schematics for the board and source code for the interface chip. That alone would be a compelling reason to buy it I think.
 
A properly matched LoRa output stage should give you between 7-14kms, line of sight, transmission range at 1200 bps. That's awesome!!!
 
I've been looking for the info on the Si4463 but cant seem to find anything on the 10uA listen mode. Would you mind posting a link?
 
Regards
Trevor

Sure:  http://www.silabs.com/Support%20Documents/TechnicalDocs/Si4463-61-60-C.pdf
 
Turns out it's 50kbps, not 50bps.  The front summary page gives the headline:  "10 µA average RX current at 50 kbps and 1 sec sleep interval."  Not sure, but a quick read suggests it's for detecting the preamble and then waking up the radio.  Some call that "wake on radio" or similar.
 
Of perhaps greater interest to you isn't that, but here (http://www.silabs.com/products/wireless/EZRadioPRO/Pages/si446x.aspx?tab=info), where it says "The Si4468/3 offers exceptional output power of up to +20 dBm with outstanding efficiency. The high output power and sensitivity results in an industry leading link budget of 153 dB allowing extended ranges and highly robust communication links."  It's not 157, but still impressive, no?  Perhaps more importantly, available on a wider range of less expensive modules.
 
Then there's the HopeRF modules, which seem to be gaining in popularity, including at JeeLabs.  I bought some a year ago, but it turned out there wasn't much example code on how to use them, so I decided to shelf them and wait.  I think maybe by now it will be easier to try them out without pondering long spec sheets and setting their registers directly rather than, as by now might be possible, through an easier to use abstraction layer.

 

[tattema]

Hi NeverDie
 
Thanks for showing me exactly where to find the info and it is quite impressive. I understand how they get "10uA average RX current" and it's through clever slight of hand. It works when the transmitter preamble length is longer than 1 second. The receiver radio wakes up every n ms intervals (1000ms) and listens for the preamble (1-10ms) and then goes back to sleep. Is this ideal because it means that a 1 second preamble is required when transmitting. That to me seems like a long time. When detected, an interrupt can wake up the CPU to download the packet.  Semtech follow the same approach for reasons that might not be immediately apparent.
 
Here's what I mean. All these chips appear to be designed by a the same company (German I think but I can't find the reference to this right now) and the designs are then licensed to companies like Semtech, Hope electronics, SiLabs and others. For example, the Hope chips are re-badged Semtech chips and the data-sheets are almost identical except for the header and footer!! The drivers available on the Semtech website work with the RFM69 hope boards and their LoRa equivalents (horrible manufacturing quality though).
 
Looking at the Si4468 data sheet I can see remarkable similarities in design between it and the Semtech silicone so it makes sense that a good design should be licensed and expanded upon by different manufacturers. Even the RF front end is identical in design and so are the LNA's.
 
With regards the Si4468.... it's truly impressive so thanks for sharing. I'm even inclined to buy a few modules for experimentation purposes. It's FSK so I presume that I can be made to work with my Semtech based boards with little hassle.
 
Regarding an abstraction layer, download and look at the Semtech drivers. You'll see that it's all plug and play with a single 'Radio' interface for Sx1232 and the LoRa series. All you need to implement is the HAL SPI interface and that's not difficult for anybody semi-experienced with microcontrollers.
 
Regarding JeeLabs and the drivers he wrote... i'd tread carefully because they're incomplete and not state machine driven which means that large packets over the internal buffer size (64 bytes) are not supported. In addition, buffers are emptied from within an interrupt which isn't ideal. The JeeLab drivers were then copied by Low Power Labs as part of their AVR to RFM69C implementation so they too are flawed.
 
The Semtech drivers are beautifully written and support variable length packet sizes up to 255 bytes so they should be used. I bring this up because if we really want low power from the Semtech radio silicon, similar to the Si4468 preamble sniff mode, then the Semtech drivers should be used as they support this out of the box too.
 
Regards
Trev

 

[deandob]

Great info on low power high range radio options. I'm sure you already know but when you get to such high levels of sensitivity (153 vs 157dBm) than your antenna design and efficiency has a major role to play. Trev, you are doing some hard core stuff with your radio design and PCB impedance matching (also as you say important for higher speed RAM traces) and correct that a typical board house won't be sufficient. I haven't dabbled in RF for a while and stick with RF modules and CPU modules and concentrate on the automation hardware and software - you have perked my interest in doing something on the RF side for sensors, to date I have only played with the BLE and 433/918Mhz generic boards from TI and Nordic Semi.
 
Regarding front end HTML5 interfaces, the graph widget I have built is the main visualization interface for time series graphs. There are a number of HTML5 charting widgets around, FLOT being the most popular and built on top of jQuery (www.flotcharts.org) and the easiest to get started on. I prefer the D3 javascript library as it is lower level and a lot more flexible, you can do some amazing visualizations with D3, check out d3js.org although the learning curve is higher, I can share the chart widget I wrote which has zoom and pan, auto axis scaling and a few more features. I meant to post more details about it but haven't got around with it. Maybe describe your use case for the front end a little more and I can help you select the right technologies (or even use one of the open source front end HA solutions like the links posted in this thread one or two pages back). I'm considering making my solution open source and you are welcome to try it but it needs the back end automation engine so would need to work to retool it for something else. If I get time I'll setup a cloud based demo instance of the back end and HTML5 front end for folks to play with.
 
Can you describe your gateway device a little more - sounds like you built the CPU module from scratch if you had to mess with trace impedances (also critical to have the right test equipment to debug, which I don't have).

 

[tattema]

Hi Deandob
 
Thanks for the information on D3. I'll definitely take a good look, especially if it supports HTML5. Ideally I want my front end to support tablets, phones and larger screen sizes without needing to write a separate app. I thing the term is responsive design.
 
The gateway device runs an embedded scripting language (LUA) for gateway triggered scenes. The nodes also run an embedded scripting language that's compiled on the gateway. Byte code for any number of scenes is then uploaded to the nodes and executed in situ. This makes the nodes highly configurable and autonomous.
 
The gateway hardware runs an LPC1788 CPU at 120 mhz. It has 64mb or external SDRAM and 128mb of NOR flash. It's LINUX capable  but why bother with all the overhead when when  it can be efficient, small and fast down to the metal.
 
I use single wire debugging instead of JTAG... it's newer, faster and better suited to the new range of M3 and M4 processors.
 
Here's a pic. Sorry about the bodge wires... A couple of address lines went to the wrong pins :-(
 

 
In the picture you can see the RF module soldered to the board along with the MCU.
 
Regards
Trevor

 

[tattema]

Great info on low power high range radio options. I'm sure you already know but when you get to such high levels of sensitivity (153 vs 157dBm) than your antenna design and efficiency has a major role to play. Trev, you are doing some hard core stuff with your radio design and PCB impedance matching (also as you say important for higher speed RAM traces) and correct that a typical board house won't be sufficient. I haven't dabbled in RF for a while and stick with RF modules and CPU modules and concentrate on the automation hardware and software - you have perked my interest in doing something on the RF side for sensors, to date I have only played with the BLE and 433/918Mhz generic boards from TI and Nordic Semi.
 
Regarding front end HTML5 interfaces, the graph widget I have built is the main visualization interface for time series graphs. There are a number of HTML5 charting widgets around, FLOT being the most popular and built on top of jQuery (www.flotcharts.org) and the easiest to get started on. I prefer the D3 javascript library as it is lower level and a lot more flexible, you can do some amazing visualizations with D3, check out d3js.org although the learning curve is higher, I can share the chart widget I wrote which has zoom and pan, auto axis scaling and a few more features. I meant to post more details about it but haven't got around with it. Maybe describe your use case for the front end a little more and I can help you select the right technologies (or even use one of the open source front end HA solutions like the links posted in this thread one or two pages back). I'm considering making my solution open source and you are welcome to try it but it needs the back end automation engine so would need to work to retool it for something else. If I get time I'll setup a cloud based demo instance of the back end and HTML5 front end for folks to play with.
 
Can you describe your gateway device a little more - sounds like you built the CPU module from scratch if you had to mess with trace impedances (also critical to have the right test equipment to debug, which I don't have).

 
I just checked out D3 and it's really nice... so polished!!!
 
If you get time to setup your cloud based demo instance I'd love to have a play.
 
Regards
Trevor

 

[NeverDie]

This is a great discussion.  Thanks, guys, for sharing your insights.  

 

[deandob]

Trevor, great stuff for the micro board you have prototyped. Cut traces and bodge wires are all part of the experience! We could also start a new thread about soldering small pitch SMD devices, I don't go lower than 0603 and took a while to perfect a technique for soldering large QFN chips (flux is your friend....), that monster in your pic is a LQFP144 package? Did you choose this particular micro because it has leads? A number of the newer micros are ball grid array packages which are harder to DIY. Personally I prefer Microchip for the smaller micro devices and TI for the larger & for peripherals, mostly because I have the toolchains setup and familiar with the software and hardware architectures used. If you are not using LINUX what RTOS are you using instead?
 
I'm pretty busy at work but will send you a PM in a week or so to access my HTML5 front end from a cloud service that I use for remote access to the automation system after I fix a couple of things that broke with a recent change. It's not secure (yet) and has full access to the house automation so not willing to post it here, but it is exactly the same version that I use on the house LAN, and as it uses a responsive design (thanks to twitter bootstrap & the flexibility of the dashboard) it works on mobile just fine as I posted about earlier in this thread.

 

[tattema]

Trevor, great stuff for the micro board you have prototyped. Cut traces and bodge wires are all part of the experience! We could also start a new thread about soldering small pitch SMD devices, I don't go lower than 0603 and took a while to perfect a technique for soldering large QFN chips (flux is your friend....), that monster in your pic is a LQFP144 package? Did you choose this particular micro because it has leads? A number of the newer micros are ball grid array packages which are harder to DIY. Personally I prefer Microchip for the smaller micro devices and TI for the larger & for peripherals, mostly because I have the toolchains setup and familiar with the software and hardware architectures used. If you are not using LINUX what RTOS are you using instead?

 
HI Deandob
 
I have to laugh now at those bodge wires but at the time I spent quite a few hours trying to work out why the SDRAM wouldn't initialise it it was a bit frustrating!!
 
If you think there would be interest starting a thread on DIY soldering small pitch SMDs then I'd support it and participate. A simple rule I discovered while soldering  small pitch SMD components... and I learned the hard way... no wine, spirits or beer while doing it!! Not a drop! I jest but it's true... I've destroyed a couple of boards this way.
 
I think 0603 is probably good enough for most situations and that's generally what I like to use but 0402 was needed in a few places and they're really hard to position and solder. A USB microscope with a flexible arm is key
 
Interesting that you should suggest BGA because I've been thinking about using them on the node now that the platform is stable. The LQFP package (LPC1788 device) was chosen because I needed to debug the board and knew that BGA would make that almost impossible. You saw in the photo how I had to bodge wire the SDRAM well that fix would have been terribly difficult with BGA. Now that it all works, BGA could be a good choice to reduce overall size and to make the design more compact. Having never soldered BGA myself, do you have any tips or suggestions to make the process bulletproof?
 
Operating system (if it can be called that) - the board runs a bare metal RTOS. The standard interface is ARM CMSIS with the open source Keil RTX in the engine room. YAFFS was ported to the NOR device as a file system and LWIP runs as the gateway communications stack. Boot time - .85 seconds
 
I think that the investment we put into learning a toolchain and micro controller family often means we stick with that choice for a long time because changing or moving to different platforms reduces productivity and is generally an uncomfortable experience. I started with 8bit AVR's but Atmega128 (arduinos) but quickly moved over to ARM M3/M4 devices like EFM32's and LPC17xx LPC40xx devices.
 
Regards Trev

 

[NeverDie]

I started with 8bit AVR's but Atmega128 (arduinos) but quickly moved over to ARM M3/M4 devices like EFM32's and LPC17xx LPC40xx devices.
 
Regards Trev

I'm not disagreeing, but I'm wondering what it was about those other devices that you found so compelling that you quickly moved over?

 

[tattema]

At the lowest level I think my decision was influenced by time constraints so If I had more time I'd probably have experimented with other brands and technologies too.
 
Other factors were architecture, energy consumption, peripherals, price, simplicity and community support.
 
My micro-controller HA journey began with the ATMega series and the 128. The Jee Node and the work done on that site inspired me to learn more so I tried different architectures and finally settled on the EFM32 series for low energy and the NXP CPU range for the level of community support it has received through the mbed LPC1768 platform. I'm certain that other brands (ST, TI) have compelling devices too but once you're in the M3/M4 space with cmsis os then the architectures are quite similar and many of the concepts apply across the board.
 
A few years ago I experimented with microchip devices and thought they were fantastic but I never did anything serious with them.
 
Next steps will be to experiment with more powerful/capable/complex ARM devices for experimentation purposes. Perhaps an A9 or something like it. So much to learn and enjoy in this space so it's certainly very nice to meet and discuss these things with other like minded persons on this forum.

 

[deandob]

Yes, time is a big constraint, so much to explore with limited hobby time.
 
Although I'm interested in learning ARM at this point I'd rather spend the time on the device & software aspects, moving away some aspects of DIY to standard protocols like TCP/IP and MQTT as well as standard CPU modules like Raspberry Pi and Beaglebone. Windows 10 on Raspberry Pi is interesting as there is so much functionality available under Windows that is now accessible with small devices like the Pi that it opens up new opportunities (similar can be done on Linux but its usually stuff of variable quality cobbled together).
 
When I'm putting together a design one of the challenges I like to give myself is to design & build the solution for as cheaply as possible (so $250 for a 4 layer board is out), makes it more interesting and creative (in addition to criteria like low power use).