Help with "roll your own" scripting for rain forecasting and ethernet based irrigation controller (I

[uh2]

Hello,
 
I've been lurking here for about 2 years now ever since I bought my new house. I plan to implement automation in steps, with my priority being security and irrigation at the moment.
 
There was an existing very old school rain bird controller that I recently replaced with an Irrigation Caddy wireless controller and a Toro wireless rain sensor. That takes care of the "shutting down the sprinklers" of there is rain happening at the moment, but I'd like to shut the system down if rain is forecasted, and then back on again once it passes. There is a limited API for the controller that allows me to send commands to it that will start/stop the whole system. My thought, is that I would like to query something like Weather Underground for the next day's forecast, and if the precipitation is above a certain %, I would send a command to the controller to shut the system down until the next check. I would be doing this from my PC for the time being (always on, redundancy within reason). In the future I would move this function to a hardware based controller, but I'm not at that point yet.
 
I've searched and had some promising leads, but ultimately I'm tapping out to see if someone else has crossed this road so I don't have to reinvent the wheel. I'm running Windows and Linux currently. I have a MacBook Pro, but that's for work and not an option right now. (Indigo seemed to fit the bill).
 
Any thoughts?  Thanks in advance.

 

[pete_c]

Many years ago here after installing the Rainbird ESP controller I bugged Rainbird relating to being able to modify the controller.  I had no luck.  I then put in the Rainbird wireless module thinking that I could just tap into that for my automation stuff.  I ended up just going to Rain8Net serially connected modules.  Note this is the early 2000's and there were not many choices.  Oh yeah I also wanted to see it on my touchscreens back then (~2003 or so).
 
I was privy to using software (mothership) which I could create a number of events (if then statements) to include initially weather services stuff creating on and off variables type stuff.  The more I added the more baggage I had to watch (in a virtual sense).  I went from using on line weather stuff to mixing in locally installed weather station stuff.  Initially it was mostly using 1-wire; then added a Davis Weather station to the mix and a digital moisture device (like the one built into your automobiles windsheild).  Purchased a 1-wire Dallas instruments rain tipping bucket.  It worked except if it got really windy.  I sort of got tired of adding more and more stuff  / variables to watch and just went to using irrigation software which included all of my stuff and then more calculating (ET variables).   The manifold with the solenoids, zones and the controller controlling the solenoids methodologies and hardware is really basic and hasn't changed much over the years.  Just the provisioning of means of control or tranport adds have been neato stuff.  (serial, X10, UPB, automated switches, Z-Wave, network is all neato stuff).  I do lean more on the analog hardware connected stuff than internet weather stuff mostly because this stuff is "connected" to my lawn rather than a few miles away.  I am in the midwest and the Great Lakes here create very unpredictable weather.  In Florida it weather never changes year after year (well last 30 years).
 
Go slow; baby steps; one at a time creating variables and mixing those variables with whatever you want (weather stuff from the internet, weather station stuff, whatever is your fancy).  Personally it was a learning experience that I have enjoyed over the years.
 
Today the software is running fine.  I don't pay attention to it unless I see an issue in the yard.  Most of the variables are analog direct connects for measuring stuff.  (water flow, water shut off, rain, now moisture, UV, Wind, temperature, humidity so forth and so on.)  I also played with soil moisture sensors for a time (thinking I still have a few in place).  Over the years never really worried about the water use costs.  Today I do.  That said today the irrigation setup is both efficient and effective only watering when the grass needs to be watered (still using a schedule of sorts but with the added variables of time, amount, when type of stuff) such that I get the most bang for my buck.  I mean its really about effective use of the software and it's way better than putting a rain sensor on the Rainbird old ESP timer device.

 

[az1324]

If you're not wanting to write something in perl, python, vbscript etc... you can check eventghost or autoit.  I'm sure there are many snippets out there that will get you started.  The question is what if there is a high chance of rain for several days but it never does rain.  At what point do you say ok go ahead and water (and then immediately following the watering cycle it starts to rain).

 

[uh2]

Thanks for the replies and suggestions. I'm anxious to get this project started and I'm sure I'll be spending a lot of time tweaking it.

As far as bad forecasts goes, that's a good question. There's a lot of logic that could go into making an automated decision like that. Hmmmmmm.

 

[uh2]

I looked at EventGhost, and there used to be a weather plugin that called Google's API, but I guess Google yanked the API. There's another plugin called SunTracker, but it's not as verbose as I need. I checking out AutoIt right now.

 

[neillt]

If you ever add more automation, an HAI panel paired with HaikuHelper will store the forecasted rain amount in mm for the next 5 days as flag values that you can use in programming.
 
It's not as crazy as ET formulas, but if you add an outdoor temp and humidity sensor, you may have most of the variables needed to do the rather complex math, and HaikuHelper can do that if you take the time to code it in Java.

 

[JimS]

I found this NWS site that gives forecast AMOUNT of rain (not just a chance of rain). 
 
http://www.nws.noaa.gov/mdl/synop/avnmosmap.php
 
It links to state by state text forecasts for cities that could be parsed.  Some time ago I had figured out the URLs for the individual city forecasts.  I figured I could take the three closest stations and average them.  Then do something like NeverDie suggested - if it is going to rain then minimize irrigation to reach the rain event.  If the rain event doesn't happen they you can simply irrigate at that point.
 
There is a perl misterhouse module for MOS forecast that parses this data for chance of rain, etc.  But I don't think it provides any information about amount of rain.
 
Have been interested in this for some time but haven't gotten very far on it.

 

[uh2]

If you ever add more automation, an HAI panel paired with HaikuHelper will store the forecasted rain amount in mm for the next 5 days as flag values that you can use in programming.
 
It's not as crazy as ET formulas, but if you add an outdoor temp and humidity sensor, you may have most of the variables needed to do the rather complex math, and HaikuHelper can do that if you take the time to code it in Java.

 
 

At the moment I'm using soil moisture sensors rather than ET (which I will probably add at a later time), but I think the logic may be the same: either it's time to irrigate (you're beginning to approach a plant stress point in soil moisture that you want to avoid), or it isn't.  If rain is expected and the quantified precipitation forecast is non-trivial, I reduce the irrigation amount so as to guarantee the plants can last at least another day without reaching the "to be avoided" stress point..  That covers the case where the forecast is wrong and it doesn't rain at all.
 
Are there better ways to do it?

 
I'm going to keep this in mind when I get it running. It will be a 'Phase 2' implementation.
 

I found this NWS site that gives forecast AMOUNT of rain (not just a chance of rain). 
 
http://www.nws.noaa.gov/mdl/synop/avnmosmap.php
 
It links to state by state text forecasts for cities that could be parsed.  Some time ago I had figured out the URLs for the individual city forecasts.  I figured I could take the three closest stations and average them.  Then do something like NeverDie suggested - if it is going to rain then minimize irrigation to reach the rain event.  If the rain event doesn't happen they you can simply irrigate at that point.
 
There is a perl misterhouse module for MOS forecast that parses this data for chance of rain, etc.  But I don't think it provides any information about amount of rain.
 
Have been interested in this for some time but haven't gotten very far on it.

 
I stumbled on Misterhouse today and found some scripts that dealt with sprinkler control and one that dealt with control AND weather querying. I didn't get very far with any of them. I did find the get_weather script and batch file and that gives me what I need (not sure how accurate it is but it's giving me a lot of data). Here's a sample of the output below. The problem is that MY city doesn't have any data when the service get queried. I can use any other city and state, and it works. 

Getting the current weather for Lorain, OH
Retrieving http://www.weather.gov/view/prodsByState.php?state=oh&prodtype=hourly
Getting the forecast for Lorain, OH
Retrieving http://www.weather.gov/view/prodsByState.php?state=oh&prodtype=zone
Retrieving http://www.weather.gov/view/prodsByState.php?state=oh&prodtype=zone
 
Current conditions: No data available
 
The forecast is As of 6:54pm Tue Apr 29, 2014:
Tonight: Showers likely with a chance of thunderstorms. Lows in the mid
    50s. Southeast winds 15 to 20 mph. Chance of rain 70 percent.
Wednesday: Mostly cloudy. Showers likely in the morning...Then a chance of
    showers and thunderstorms in the afternoon. Highs in the upper 60s.
    South winds 10 to 15 mph, becoming southwest 5 to 10 mph in the
    afternoon. Chance of rain 70 percent.
Wednesday Night: Mostly cloudy. A chance of showers and thunderstorms in
    the evening. Lows in the upper 40s. Southwest winds 5 to 10 mph,
    becoming west around 15 mph with gusts up to 25 mph after midnight.
    Chance of rain 40 percent.
Thursday: Mostly cloudy with a 40 percent chance of showers. Highs in the
    upper 50s. Southwest winds 10 to 15 mph.
Thursday Night: Mostly cloudy with a 40 percent chance of showers. Lows in
    the mid 40s. Southwest winds 10 to 15 mph.
Friday: Mostly cloudy with a 40 percent chance of showers. Highs in the mid
    50s.
Friday Night: Cloudy with a 40 percent chance of showers. Lows in the lower
    40s.
Saturday: Mostly cloudy with a 50 percent chance of showers. Highs in the
    upper 50s.
Saturday Night: Mostly cloudy with a 50 percent chance of showers. Lows in
    the lower 40s.
Sunday: Partly sunny with a 30 percent chance of showers. Highs in the
    upper 50s.
Sunday Night: Mostly cloudy. Lows in the upper 30s.
Monday: Partly sunny. Highs in the upper 50s.
Monday Night: Mostly cloudy with a 30 percent chance of showers. Lows in
    the mid 40s.
Tuesday: Mostly cloudy with a 40 percent chance of showers. Highs around
    60.

My biggest problem is not being a coder, and having to come up with the logic that parses this info and determines what the next days forecast is going to be. The second problem is going to be the lack of entirely accurate data since I'll have to pull the info from a bordering city. I'm still doing research and digging into more pre-made perl scripts.

 

 

[JimS]

Since this is an automation forum, are there better sources for programmatic access to quantified precipitation forecasts?

How about the source I gave above in post #8?l.  They give the forecast for several days out.  See the list on the left, links to descriptions of the formats are farther down on the left.  GFS and NAM are both 3 day forecasts, based on different modeling methods I think.  GFS extended is 7 day.  It is a short text format that is easy to parse.  The precip fields are coded:
 
0 no precipitation
1 0.01 to 0.09 inches
2 0.10 to 0.24 inches
3 0.25 to 0.49 inches
4 0.50 to 0.99 inches
5 1.00 to 1.99 inches
6 2.00 inches or greater
 
I suppose you could want more resolution on rainfall amounts but it IS just a forecast.  You may be kidding yourself if you think it can be forecast more closely.  I think an on site rain gauge would be needed for actual rainfall.

 

[uh2]

I've been working on and off for a few days on this. I've made a little progress, but not much. Jim, the NOAA would work for someone else, but not for me as my city isn't listed. I've run into that on another site. Weather Underground has a few weather stations near me, so it's probably a tad more accurate. But like you said...this IS a forecast and will often be inaccurate.
 
So, I've found some sourceforge stuff that people have done in various languages..Python, Perl, php, etc. I just stumbled on one that supports the EtherRain controller called jirrigate. I figured I might be able to get it running and pulling the weather data at a minimum, and worry about the interaction with my IrrigationCaddy later.
 
This is the output it gives me:

Station: Avon
Record: Last Updated on May 6, 9:50 PM EDT
Current Temp (C): 11.2
Today's Max Temp (C): 11.2
Today's Min Temp (C): 11.2
Current Temp (F): 52.2
Today's Max Temp (F): 52.2
Today's Min Temp (F): 52.2
Avg 7-day Temp (C): 11.2
Avg 7-day Temp (F): 52.2
Today's Rainfall (mm): 0.0
Today's Rainfall (in): 0.0
Last 7-day Rainfall (mm): 0.0
Last 7-day Rainfall (in): 0.0
Humidity (%): 60.0
Current Wind (kph): 3.2
Current Wind (mph): 2.0
PoP 1-day (%): 10
PoP 3-day (%): 60
PoP 7-day (%): 90

 
jirrigator looks like you can base the watering on a set threshold of X amount of days in the past and/or in the future, and the precipitation received or forecast.
 
Unfortunately, I'm no coder so I won't be modifying this to work with IrrigationCaddy anytime soon. I've read that the Vera has a weather plugin AND someone has written an IrrigationCaddy plugin as well. I might just go that route.
 
FYI, I ended up buying a soil moisture sensor, so I'll be playing with that as well.

 

[uh2]

What kind of soil moisture sensor did you get?

 
The Toro XTRA SMART. We'll see how smart it is....the reviews are mixed. It appeared to be discontinued, so I decided to snatch one up before they disappeared.
 
One thing I didn't think of was using a RasberryPi for botht he controller and weather delays. I wish I had thought of that before I bought the IrrigationCaddy. There's already a product that does what I want, although it's not wireless which was the big selling point for me with the IC.
 
So, right now, I've got a Python script that was designed to work with an RPi and control the actual valves. I don't need valve control, I need the ability to parse output and execute commands/scripts based on that output to tell the IC to turn off and on. Here's the output of a script I found and modified to suppress the RPi pin commands. Right now, it's checking the weather and making decisions, but not actually executing anything:

Config file found, loading previous values...
Checking forecast for zip code: 44011
System will be disabled for 1 days prior to and after rain
System will wait 3600 seconds between checks
     or 60.0 minute(s) between checks
     or 1.0 hour(s) between checks
 
Current Forecast for current day, plus next 9 is:
7:00 PM EDT on May 07, 2014, 1399503600, Thunderstorm
7:00 PM EDT on May 08, 2014, 1399590000, Partly Cloudy
7:00 PM EDT on May 09, 2014, 1399676400, Chance of Rain
7:00 PM EDT on May 10, 2014, 1399762800, Chance of Rain
7:00 PM EDT on May 11, 2014, 1399849200, Chance of a Thunderstorm
7:00 PM EDT on May 12, 2014, 1399935600, Thunderstorm
7:00 PM EDT on May 13, 2014, 1400022000, Chance of Rain
7:00 PM EDT on May 14, 2014, 1400108400, Chance of Rain
7:00 PM EDT on May 15, 2014, 1400194800, Partly Cloudy
7:00 PM EDT on May 16, 2014, 1400281200, Chance of Rain
 
### START Checking if raining TODAY ###
Rain condition found
It will rain today. Storing current epoch as 'last rain': 1399503600.0
### END Checking if raining now ###
 
### START Checking for rain in forecast range ###
Checking 7:00 PM EDT on May 08, 2014 for rain conditions:
No rain found for current day. Watering may commence
### END Checking if rain in forecast ###
 
 
Last rain from forecast timestamp: 1399503600.0
Current Time: 1399509443.76
Days since last rain: 0.067636142558
Seconds since last rain: 5843.76274991
Days disabled in seconds: 86400
Has NOT rained within daysDisabled range: False
Rain not in forecast, but it has rained recently. Watering Disabled
Checking forecast again in 60 minute(s)

 
So now I've got something in Java that will give me precipitation and humidity amounts, and something in Python that will give me the current and future forecast, plus check to see if it's rained OR WILL rain in the last/next X days.
 
I need something to parse both of these scripts and the logic to make a decision based on the output. 

 

[uh2]

Sounds like a plan. Are you using multiple controllers or are you connecting these sensors to a single controller?
I plan on installing mine tomorrow evening.

 

[JimS]

I've been working on and off for a few days on this. I've made a little progress, but not much. Jim, the NOAA would work for someone else, but not for me as my city isn't listed. I've run into that on another site. Weather Underground has a few weather stations near me, so it's probably a tad more accurate. But like you said...this IS a forecast and will often be inaccurate.
 
So, I've found some sourceforge stuff that people have done in various languages..Python, Perl, php, etc. I just stumbled on one that supports the EtherRain controller called jirrigate. I figured I might be able to get it running and pulling the weather data at a minimum, and worry about the interaction with my IrrigationCaddy later.
 
This is the output it gives me:
 
jirrigator looks like you can base the watering on a set threshold of X amount of days in the past and/or in the future, and the precipitation received or forecast.
 
Unfortunately, I'm no coder so I won't be modifying this to work with IrrigationCaddy anytime soon. I've read that the Vera has a weather plugin AND someone has written an IrrigationCaddy plugin as well. I might just go that route.
 
FYI, I ended up buying a soil moisture sensor, so I'll be playing with that as well.

 
Could you share the links to what similar things you found? 
 
I am not much of a coder either.  I mostly hack up other code that is close to what I want. 
 
Since you already have the Irrigation Caddy you may want to keep going with it.  For starting from scratch the OpenSprinkler on the pi version looks interesting.  Should be able to do other things on the pi at the same time too.

 

[uh2]

Could you share the links to what similar things you found? 
 
I am not much of a coder either.  I mostly hack up other code that is close to what I want. 
 
Since you already have the Irrigation Caddy you may want to keep going with it.  For starting from scratch the OpenSprinkler on the pi version looks interesting.  Should be able to do other things on the pi at the same time too.

 
Jim, I plan on collecting everything and sharing it tonight. My problem is as soon as I found anything remotely interesting I bookmarked it, so now I have a ton of bookmarks to sift through and decide whether it was worth keeping.
I'm like you, I can hack up other people's code but I don't have a lot of time to learn the languages and code myself. Having said that, and after looking at Ruby, Python, Java..etc...I may end up with something like a bash script mixed with python scripts/code.
 
One thing I've found is that the Irrigation Caddy API , at least what they have published, isn't very feature rich. I've read you can use Chrome's dev tools and figure out what commands need to be sent though. For my purposes, I think being able to send a simple on/off command will work. Having said that, I can also see in the not too distant future where I would want more control of the IC. Another thing I didn't think about when I first posted was the need for status notifications. At some point, I'll want to be able to view the logs (output of the batch files) and get notifications via smtp or sms that a certain action was initiated , or the current status.
 
If anyone is reading this that has coding experience and could help me parse the outputs of these two scripts, or merge them into one, the help is much appreciated. I did find a tool that would convert java to python...not sure how well that would work or if I need to pursue that at all.

 

[uh2]

Well, I haven't taken the Toro out of the box because after doing some more research, I'm wondering if it's worth it or if I need to change my approach. I have a Toro wireless rain sensor that occupies my rain sensor port in my controller. Initially I thought I could just piggyback the xtra smart into the same terminal but that's not gonna work..... Unless.... Maybe... If both sensors can be changed to 'always open'? I can't see it working with both sensors always closed. Second issue I see is that if I don't use thw rain sensor terminal and wire it directly to the common valve wire, I'll have no indication whether this thing is suppressing the valves or not. The controller won't know... It will just run through it's schedule and won't have any idea what the sensor is doing. That's not something I think I can live with. I want some sort of status that I can query or at a MINIMUM eyeball from a script or webpage. I might have to return this and go another route if I can't figure an easy way around this. Notice I said 'easy'. I'm surr I could throw some more hardware (arduino or RPi) at it, but I'm trying to avoid adding to the points of failure if possible.