Why Are All Automation Engines These Days Trigger Based?

[upstatemike]

Back in the days when Stargate was my main automation platform life was simple. If I needed to trigger something when x is true and y is true (or on or secure or 1 or open or whatever state terminology you like) then all I had to do was build an event that said if x is true and y is true then do z. Stargate would evaluate that statement many times per second to see if all the conditions became true and if they did then z would be executed. Simple.
 
Nowadays you can't simply expect an automation system to evaluate a set of criteria and respond accordingly. You have to pick one item to act as a trigger and only if that one trigger changes will it evaluate the rest of the conditions to see if something should be executed. So the the example above becomes more complicated because you need to watch for changes in x and y separately: If x becomes true then if y is true do z PLUS if y becomes true and x is true then do z. What is the point of this? The Stargate way is simpler and the logic easier to follow so why dumb things down to require more statements to accomplish the same thing?
 
I also notice that Stargate handles nested If-Then logic using any combination of AND and OR much more intuitively than any other system available today (that I have seen anyway). Why is this so kludgey in most modern HA platforms? 
 
I am not a person who can't deal with change but it seems to me that changes should be meant to improve things. The way logic statements are built up in modern automation platforms seems to be one area where things have gone backwards and gotten worse instead of better.

 

[123]

In Home Assistant you can specify multiple triggers and the default behavior is to logically OR them.
 
Pseudo-code (implicit logical OR):
 
trigger:
 - kitchen light is on
 - bedroom light is on
 - hallway motion is on
 
If you want to logically AND them then you must create an expression.
 
Pseudo-code (explicit logical AND):
 
trigger:
 - kitchen light is on AND bedroom light is on AND hallway motion is on
 
When Home Assistant first examines the expression it creates a 'listener' for each item found (three listeners for the given example: kitchen light, bedroom light, hallway motion). When a listener detects a state-change, the expression is evaluated. So when hallway motion turns from off to on, the expression is evaluated and if bedroom light and kitchen light are on then the trigger's expression evaluates to 'true' and the automation proceeds to the next step (either to evaluate a separate condition, serving as an additional filter, or directly to executing the action). You can also use expressions to create complex combinations of AND and OR.
 
 
NOTE:
The actual expression would look something like:
 
{{ is_state('light.kitchen', 'on') and is_state('light.bedroom', 'on') and is_state('binary_sensor.hallway_motion', 'on') }}
 
 

 

[upstatemike]

So that works a little better than most systems but I still don't understand why a listener needs to trigger the evaluation of the expression. It seems like the listener mechanism should update a register where the state of the device is held and the expression should be evaluated every so may milliseconds regardless of listener activity. This removes any chance that the expression misses a state change and fails to trigger. The measures to ensure the register update happens and device changes are not missed should be handled within the protocol for the device that the register is representing.
 
The logic certainly is back to what I would like to see though as a user I don't want to view the syntax of the expression. I would rather that syntax was behind scenes and the programming done using pseudo code input. (Or maybe HA does that... I haven't looked into Home Assistant yet.) How does it do with nesting additional layers of conditions under each device? As in:
 
- kitchen light is on
     OR garage door is open
AND bedroom light is on
     OR bathroom fan is on
AND hallway motion is on
     OR garage door is open AND time is after 6AM
     AND TV is off

 

[123]

Event-based
I assign a task for you to complete and when it's done you report back to me.
 
Polling
I assign a task for you to complete and then repeatedly ask you if it's done.
 
Clearly the event-based technique involves far less communication (polling: is it done? is it done? is it done?). In addition, polling at millisecond intervals would present performance problems for common mesh wireless protocols like ZigBee and Z-Wave.
 
Event-based triggering is more efficient not only from the standpoint of communications but from the polled device's viewpoint. It doesn't have to spend any time responding to consecutive requests asking for its status (polling: not done yet, not done yet, not done yet). Plus, many wireless devices nowadays are battery-operated (coin cells) and polling them even at 1 second intervals would greatly reduce their battery life.
 
Yes, the example you provided can be done in Home Assistant. You can use parentheses to group the statements to make it explicitly clear how you wish to have it evaluated.
 
Pseudo-code:
((x and y) or (w and z)) and (a or b or c)


NOTE:

There are situations where polling is employed in Home Assistant. For example, let's say you create a sensor to monitor outdoor temperature that gets its data from an external web-site. The web-site is unable to tell you when the temperature changes so you must poll it. When the sensor is defined, you indicate you are using the REST platform to acquire the data and then indicate the desired scan_interval (default is every 30 seconds).

 

[LarrylLix]

Having an ISY994 you should know that all the options are available with it's logic system.
 
If
    switch X is switched ON
  OR
   switch Y is switched ON
Then
…..gives an either or logic
 
If
   switch X is switched ON
  AND
   switch Y is switched ON
Then
….will never be true but run Else with either action
 
If
    switch X status is ON
   AND
  switch Y status is ON
Then
…..gives true only when both are ON
 
Insteon is all trigger based to give instantaneous responses and not wait for a cycling logic engine to come around the bend, delaying responses. Many Zwave devices used this concept and were dumped as soon as people could. In addition it makes a lot of traffic making system even slower with bogged down ports.
Cyclic polling systems become very slow when the number of devices exceed large amounts as found in a mature HA system.
 
I worked on one system that used a real time engine, on a cyclic evaluation engine. It was extremely easy to duplicate older relay logic systems in software. I just took a relay logic system schematic and defined a logic statement for each relay with a formula that matched all the contacts activating that relay. Turned it on and it ran just like the relay based system. It was quite easy for systems using over a few hundred relays and incorporating timers etc..  But like all cyclic evaluation engines the timing got slower based on a 100 cycles per second, and the faster stuff had to be handled with hardware signal handlers on the S100 bus. Electrical grid protection cannot wait for a cyclic engine clock  without burning down transmission lines and electrical fires. LOL
 
I worked on SCADA systems most of my life and the best protocol I found was reporting by exception (changes) ...no polling. but the thing that needs to be done is occasional full data reports. These can be used as heartbeats to know devices are alive or not. One protocol started to report an additional few devices every report so that every device was reported over time in a round robin style and the all data poll was not hardly needed. That worked very well but one poll was always required....a complete data poll for when the host crashes or gets powered up knowing nothing. This must be controlled from the master end to control  data storm flooding. When a city's electrical grid first powers back up, the amount of data arriving can hang the most powerful of data collection systems.

 

[ano]

Old systems like Stargate were programmatically driven. Your program was just a big loop that repeated again and again. It wasn't so much evaluating conditions, as much as the change of conditions. That works, but as you know, as your program gets bigger and bigger, the loop runs slower and slower.
 
Most more modern hardware runs on interrupts, where if hardware (memory locations) change, this causes an interrupt, and programming starts from there. Much faster, in reality. Unfortunately, you can only watch for one change, and any second or third tests need to be conditions. The program can be very long, and it doesn't matter, because only the interrupt takes effect. Many software systems hide this architecture, but underneath, that is how things will always work.

 

[upstatemike]

Some good points from everybody. Thanks!
 
Note that my question was about how the logic engines in a controller work. I was not suggesting that polling should be applied to Z-Wave or any other external protocol. Only to the internal registers that track the state of those sensors within the controller. The logic engine should not know or care what kind of communications and validation a protocol is using to set the register values.
 
Without pretending to have technical skills that in reality I do not. I will just note some observations about interrupt vs polling. Polling in Stargate seems to be more resilient than interrupts in other systems because if a register update is missed it will be caught the next time around (30 milliseconds in the case of my Stargate) while a missed interrupt is just lost. Polling lets you directly interrogate physical inputs instead of relying solely on a device table that could get out of sync with reality. There is no issue with resource contention with polling because the resources needed are constant and do not vary between all devices being idle all the way to all devices change state at the same time.
 
I did forget about the "status is" option in the ISY. I haven't added any new programs to it since I updated it to version 4.6.2 a couple of years back. As I recall ISY has some other things about the programming that didn't seem intuitive. I recall an example that went something like:
 
If X is on
AND 
If X is not off
THEN....
 
I don't remember what that was about. There was also something about THEN statements not completing if the trigger became false before all the THEN actions were done. I need to gt back into it once the Polisys comes. Probably after the Holidays the way things are going.
 
As far as a loop running slower and slower as you add devices, I just don't see that. There are a finite number of memory locations to track hardware state and if clock and processor as designed to to cycle through all of them at the required rate then it would not matter if you max out your system or run it with only one sensor since the cycle would poll every location whether it was in use or not. The cycle time should never change.

 

[LarrylLix]

Some good points from everybody. Thanks!
 
Note that my question was about how the logic engines in a controller work. I was not suggesting that polling should be applied to Z-Wave or any other external protocol. Only to the internal registers that track the state of those sensors within the controller. The logic engine should not know or care what kind of communications and validation a protocol is using to set the register values.
 
Without pretending to have technical skills that in reality I do not. I will just note some observations about interrupt vs polling. Polling in Stargate seems to be more resilient than interrupts in other systems because if a register update is missed it will be caught the next time around (30 milliseconds in the case of my Stargate) while a missed interrupt is just lost. Polling lets you directly interrogate physical inputs instead of relying solely on a device table that could get out of sync with reality. There is no issue with resource contention with polling because the resources needed are constant and do not vary between all devices being idle all the way to all devices change state at the same time.
 
I did forget about the "status is" option in the ISY. I haven't added any new programs to it since I updated it to version 4.6.2 a couple of years back. As I recall ISY has some other things about the programming that didn't seem intuitive. I recall an example that went something like:
 
If X is on
AND 
If X is not off
THEN....
 
I don't remember what that was about. There was also something about THEN statements not completing if the trigger became false before all the THEN actions were done. I need to gt back into it once the Polisys comes. Probably after the Holidays the way things are going.
 
As far as a loop running slower and slower as you add devices, I just don't see that. There are a finite number of memory locations to track hardware state and if clock and processor as designed to to cycle through all of them at the required rate then it would not matter if you max out your system or run it with only one sensor since the cycle would poll every location whether it was in use or not. The cycle time should never change.

I vaguely remember that discussion, I think. It has happened over and over throughout the years.
 
If
        switch is switched On  <-----------looks at the signal from the paddle being tapped
      AND
        switch status is ON  <---------- looks at the existing state of the dimmer electronics
Then
 
At first, it seemed confusing and wasted  to me,  and I, for one opposed the logic sensibility of it. Once we understand the switched construct comes from the paddle and the status comes from the dimmer electronics, and an Insteon SwitchLinc really is three independent devices, it makes more sense, and is actually a workable consideration.
 
OTOH: Checking the status of a switch we want on,before turning it on, only saves a redundant ON command going out to an already On switch, and just complicates the readability of the program with a useless line of code.
 
OTOH:OTOH:  Using the status test in a program with a bad status in ISY (problem with push technology style), could avoid turning a light off for hours/days until a forced query was done and the real status was known. 
 
Afterthought.
The "Not Off" was to circumvent dimmers that may be on at 50%. They are NOT on and they are Not off. Testing ON fails.

 

[123]

I was not suggesting that polling should be applied to Z-Wave or any other external protocol. Only to the internal registers that track the state of those sensors within the controller. The logic engine should not know or care what kind of communications and validation a protocol is using to set the register values.

 
In the big picture, there are external devices being monitored/controlled (either via some wireless or wired protocol). The logic engine's "internal registers" get populated through some kind of process, most efficiently by an event-based one.
 
Once the "internal registers" are populated, another event-based process can be employed to handle the register's state-changes. Basically we're talking about using a state-machine for this purpose. For anyone unfamiliar with state machines, trying to find a simple introduction to them is … challenging. Even worse is finding a simple introduction to the software architecture of a state-machine implementation. To keep the discussion simple, they don't employ a simple looping/polling mechanism.
 
From Home Assistant's Developer Docs:
 
State Machine: keeps track of the states of things and fires a state_changed event when a state has been changed.

 

[upstatemike]

In the big picture, there are external devices being monitored/controlled (either via some wireless or wired protocol). The logic engine's "internal registers" get populated through some kind of process, most efficiently by an event-based one.
 
Once the "internal registers" are populated, another event-based process can be employed to handle the register's state-changes. Basically we're talking about using a state-machine for this purpose. For anyone unfamiliar with state machines, trying to find a simple introduction to them is … challenging. Even worse is finding a simple introduction to the software architecture of a state-machine implementation. To keep the discussion simple, they don't employ a simple looping/polling mechanism.
 
From Home Assistant's Developer Docs:
 
State Machine: keeps track of the states of things and fires a state_changed event when a state has been changed.

Thanks for the link. A little bit much to absorb. I also went to the HA home page to get more of a general introduction to the system but the site doesn't really do a good feature overview. There is a list of integrations and a list of examples but nothing that really summarizes the system in a way that lets you picture how it all might work for you.

 

[123]

 nothing that really summarizes the system in a way that lets you picture how it all might work for you.

 
http://drzzs.com/homeassistant-101/

 

[upstatemike]

That's better. Thanks for the link!

 

[Desert_AIP]

Omni programming would stack those triggers



IF X SECURE
IF Y SECURE
AND IF X SECURE
AND IF Y SECURE
THEN Z


If either is true evaluate if both are true, then perform the action if they are.
It's essentially monitoring the state of both X and Y every pass through the loop.
A change in either one triggers evaluation of the other, but both are always initially evaluated independently.

Max resolution if I recall correctly is 50ms.