OMNI II network ports locks-up

I have an Omni II that once a week will totally lock-up the network port on the board. The only way to recover is to do a hard reset on the board and then it will work fine for about a week and happen again.
 
I had talked to someone at HAI and they said they’ve seen the network port lock-up with a lot of traffic on the network since it’s only 10mbps
 
I’ve done the following:
 
replaced the network cable
set the smart switch to only 10mbps (full duplex)
switched to a secondary switch
checked the event log (nothing)
have the latest firmware loaded
 
My guess is it probably does have something to do with becoming overwhelmed by the network traffic (Netflix, etc). I hate to put a router in-front of the Omni just to filter traffic going to it. And then having another point of failure. Does anyone have any recommendations?

I had a similar problem and tried many solutions (including putting the OPII on a different network).  The final solution was to replace the OPII board.  I have had no problems since I swapped it out.  I could have also sent in the board for repair, but was concerned that an intermittent problem might not get repaired on the first try.  So I bought an OPII (this time with upgradeable FW) on eBay.  
 
I would think HAI/Leviton could make a serial to ethernet converter that uses a more up to date ethernet chipset, but that doesn't seem to be in their plans...

Curious pct88; did you also isolate the HAI OPII panel from other devices on the network physically on a separate switch?  This is wierd then because your replacement just works without any changes to your network?  So I am wondering if there was some design changes on the NIC then as my OPII Pro panel is only maybe 3 years old or there is actually something wrong with the network interface on my panel.   I put the old one in the Florida house where its been working fine (now its like 10 years old).
 
Here I also had the same issue and just isolated the HAI OPII pro panel and it fixed my issues.  I have not had any to date.
 
First signs though of the network issue were related to my thermostat serial communications with communications errors on the serial bus, then losing time (which would go off by almost an hour in a 12 hour time span), then intermittent disconnects on the network; finally a total disconnect on the network interface.
 
Resetting the panel would work for a period of time which could have been 1-2 days or months. 
 
The test that always seemed to work was to disconnect the network interface and just utilize the legacy Omnitouch screens to get to the panel or PCA via a serial interface.  After doing this time would remain just fine and serial comm would also be good.  Historically too leaving it disconnected for a couple of days then connecting would sometimes work for many months until the next time it would occur. 
 
The initial (pending doom) "tell tale" sign though was initially always the Omnistat2 to panel communications errors and legacy Omnitouch thermostat controls window show zero degrees settings; then the time would go off concurrently; then intermittent network issues slowly progressing until the network interface no longer worked.
 
Matt, try isolating the connection on a managable switch.  If that is not available then just stick the HAI OPII Pro behind a cheap o not utilized AP combo switch firewall and configure a rules set to put the HAI OPII in a DMZ or open up the firewall to allow everthing.  My first test was doing this and it worked for me.
 

Does anyone have any recommendations?

Click to expand...

 
Yeah; you can "test" first by physical isolation of the HAI OPII board network connection. 
 
Just test it for as long as you want; then change over to managed switch.
 
1 - I just changed the IP of the OPII Pro to some other IP on a tiny subnet.
2 - created a new tiny subnet on the inside interface of a cheap old wireless AP/router/switch
3 - bridge or put the new HAI IP into a DMZ on the inside. 
4 - Use the old HAI OPII IP for a static IP configuration on the outside of the firewall.
5 - I did this leaving the IP cameras configurations on the PCA OPII set up the same.
6 - reason #2 was that I left the Omnitouch 5.7e's IP connectivity the same pointing towards the same HAI OPII IP.  I did flip flop these connections from same said subnet to new tiny subnet back to same old subnet.
 
Direction here anyways though is to create a separate network for just the HAI OPII and IP consoles to it.  Here kind of starting to separate my networks physically.  That said I have been adding interfaces to my PFSense firewall.  Waiting now on a quad NIC such that I have 8 network interfaces on the PFSense firewall box.
 
Then put it on a managed switch with the rest of your stuff upgrading your network switch (or one maybe?)
 
Recently here I have upgraded my 24 port Gb unmanaged switches to managed switches.  Cheap these days to do.
 
I purchased a managed by web GUI only TP-Link switches. 
 
Working well these days and priced reasonable. (really most reasonally priced managed switch I have ever played with).

Thanks for your messages.. I do have a level 2 switch. The issue is I think my only option is putting a router in front of the Omni II since on the
smart switch I can't find a feasible way to have a vlan that can filter certain traffic to the Omni II but also allow all workstations to have access to it.
 
My sense with my brief conversation with HAI is this isn't the first time this issue has come up. The network port must be overwhelmed
by other traffic on the network since it's only 10mbps. The strange thing is the system runs fine after the network drops. The Omni
is almost disabling the port but not reporting anything in the logs.
 
I'm going to try to reset the RAM and I'll try putting a router in front of it and see what happens.

Yup; that was the same conversation that I had with HAI technical folks after some 2 years (?) of this problem occurring intermittently.  I did post my issues on a thread here somewhere.  Its kind of long read though.  HAI techs did state if I wanted to; to send it over for a checkup; but also stated that most likely there was nothing wrong with the OPII Pro hardware.
 
The issue did get progressive worse as I updated to mostly Gb and added more networking devices. Though when I unplugged the network interface on the HAI OPII and warm reset the OPII all of my problems went away; except that I couldn't get to using the Omnitouch 5.7e consoles, CCTV cameras connected to these et al.  BUT I was fine with the serially connected Omnistat2 and the legacy serially connected Omnitouch 5.7's and their video coming from the HAI Omntouch video hub (legacy).
 
I do not have the OPII Pro in a VLAN.  I am just filtering out all multicast, broadcast stuff from the port.  Everything that needs to get to it today gets to it the same as when it was behind the firewall. 
 
To make the switch of IP's fast I used the keyboard console the first time I tested and the legacy Omnitouch console the second time I played.  That said though I set up a serial port configuration in addition to the IP configuration for PCA.  Works fine this way.  I never really used the PCA application in serial mode before.  I didn't really notice much of a difference though.
 
What mfg of managed switch do you have?  MFG and model number?
 
My TP-Link managed switch offers only a gui for "storm-control" and a thresold (but I don't really know if its a thresold).
The more expensive managed TP-Link switch does offer both a cli and gui. 
 
On a Cisco Layer 2 switch you can do the same via the CLI.  (it will probably / most likely be similiar for other managed switches).
 
1 - "configure terminal"
2 - "interface interface-id"
3 - "storm-control {broadcast | multicast | unicast} level {level [level-low] bps bps [bps-low] pps pps [pps-low]}" 
4 - "storm-control action {shutdown} | trap}"
5 - "end"
6 - "show storm-control {interface-id} {broadcast | multicast | unicast}
7 - "copy running-config startup-config"
 
Here is the gui configuration for the TP-Link  TL-SG1024DE (getting used to using it now).
 

"forget about the storm-control setting".
 
I started to play some more with it switching back and forth between the managed switch and the firewall.  I went to setting thresold to 64Kbits/sec setting and immediately say comm issues with thermostat, time changes, IP disconnects.
 
I will play some more because it "was" working.

I'm using a netgear GS724T-300NAS Smart Switch. I ordered a non wifi router (Treadnet TW100-BRV214) and I'll put
the Omni behind it and just allow the ports the Omni needs to communicate onto the open network.
 
It's a strange problem because it seems the Omni's were never designed for the amount of traffic that newer
networks use (streaming, etc). At the very least if the network port locks-up it should be reported in the event log.
The one common thread from everyone who has the problem is a hard reset always fixes the issue for a period of time.

I'm going to give it another try in the next couple of days.

Once I swapped out my board I have not had any problems since.  The old board sits in the basement awaiting further experimenting-  though I find it unlikely that I will put much effort into that given that the new one is happy to deal with the same network conditions.  I did isolate my assorted Arecont megapixel cameras from the OPII early in the troubleshooting:  they provide many tens of MB of continuous traffic on the network.
 
I was actually planning to put a USB ethernet adapter on my Mac Mini running Haiku Helper so it would be an entirely isolated network (except when updating the OPII configuration.  Never tried that experiment though.

Thank you pct88.
 
This morning changed the IP on the HAI OPII again and moved the connection over to the new switch.
 
Did a quickie cable test with the built in diagnostics on the TP-Link switch.  It is showing open and a cable fault and seeing error packets.
 
I had left the run alone originally using a couple of patch panels to bring the connection to one switch.  That said I do have another new switch adjacent to it but its a pita to run another cable. 
 
I will use my new cable testing tool first and fix the patch over; concurrently will run a temporary cable over to the HAI panel to see if it makes a difference.
 
Turned on IGMP snooping.
 
Turned storm-control and tweaked rate down to 1024 Kbps.
 

 
I also set the port to 1/2 duplex at 10 Mbps.
 

 
Initial test showing the TP-Link Cable diagnostics
 

 
I haven't done anything with cable yet.  HAI PCA is showing the Omnistat comm issue right now.
 

 
Omnitouch 5.7's showing zero (0) for ambient temperature and HAI OPII panel has lost 2 minutes in about 10 just now.
 
Temperature and combo temperature/humidity sensors are all fine.  Not touching Omnitouch 5.7e's for the time being.
 
Disconnected the panel and checked my patch panel to patch panel connections.
 
Patch cables were fine.  Wire mapping is incorrect patch panel to patch panel.  PITA - repunching down ends right now.
 
This is what I get for patching cables to patch panels in the dark....and doing it fast and not having the proper tools to check my stuff.
 
I checked patch panel to patch panel cabling and all is OK.  Wire mapping is fine now.  I had to repunch down a couple of cables.
 
That said though when I plug the HAI OPII I still see an open connection on the switch.
 
Testing the cable I plugged a generic switch instead of the HAI panel and the connection is normal.
 
Next making up a new cable to test the switch to HAI panel connection.
 
Tested with two TP-Link managed switches, GB unmanaged switch, 100/10 unmanaged switches, a couple of really old unmanaged switches and new patch cables.
 
The two managed switches  directly connected to the HAI OPII show an open cable fault; no matter what I tried.  I can venture a guess here but will not. I don't think though its a hardware issue; but in a way it is relatively speaking.
 
The older gb unmanaged switch doesn't show a link even though one is there.
 
This kind all goes back to an old style network connection that you really can't do anything about relating to this problem.  This is a best guess right now.  Note that I have more than 50 devices on my network and lately its closer to about 100 devices which is more than I had 10 years ago.
 
I am sort of rearranging my network such that I have more than two internal networks anyways using the PFSense firewall. 
 
Moved the HAI OPII connection back to the separate LAN behind the internal firewall that I am using.
 
All the above testing and playing is helping me a bit with thoughts relating to a new linux based plugin for the HAI OPII panel.
 
Initially I wanted the plugin to run on some little boxolinux small footprint arm based CPU (like a pogoplus or similier).
 
That said maybe the plugin / device should also have two network interfaces; one in and one out and still fit inside of the HAI OPII can.  Over the years have played with those small pocket sized mini do-what routers and maybe a bit more tweaking could turn one of these into a combo plugin appliance thing for the OPII panel. 
 
I can't really modify the board but this way I can do a bit more with the network interface on the board.  The plugin would be sort of appliance like and would provide a new ethernet interface to the HAI OPII board.
 
I have a few of these around such that will give it a go and install one inside of the HAI panel.  Its a bit cluttered these days but it should not be an issue.
 
On a side note ...
 
Been playing a bit with LinuxMCE and do like the methodology that the software/hardware is a router / firewall to its consoles on a separate network
 
Update - 18th of September, 2013
 
For kicks created a separate VLAN and put the HAI OPII panel on said VLAN.  I could see the comm issues crop up as soon as I moved the panel to the specified port / VLAN.
 
Looking for a micro router today.  I have a little Asus but it only has one port.  I also have a Sapida but its a bit bigger than the Asus.
 
I have one but it only has one port on it.  That said found a few out there in internetlandia. 

Playing with this stuff convinced me to set up a management VLAN as there are many devices now that need to be managed and it would be better to kept them on a separate VLAN. 
 
Thinking I will call this VLAN - # 51 - Area 51....

Arecont are notorious bandwith suckers....

pct88 said:

Once I swapped out my board I have not had any problems since.  The old board sits in the basement awaiting further experimenting-  though I find it unlikely that I will put much effort into that given that the new one is happy to deal with the same network conditions.  I did isolate my assorted Arecont megapixel cameras from the OPII early in the troubleshooting:  they provide many tens of MB of continuous traffic on the network.
 
I was actually planning to put a USB ethernet adapter on my Mac Mini running Haiku Helper so it would be an entirely isolated network (except when updating the OPII configuration.  Never tried that experiment though.

Click to expand...

I cleared the RAM on my board and so far I haven't had a lock-up.. But the week isn't up yet.. HAI has told me they've seen issues with the port when someone has a loop in their automation
programming but I had them check out the programming and no loops. If it happens again I plan to put a dedicated router in front of the Omni. I don't entirely trust a VLAN over a router because
your just can't filter to the same level as a router. And because most equipment now a days is at least 100mbps compared to the Omni's 10mbps  I'm not sure how well a VLAN is going
to be able to segregate the traffic on that port.

Yup here in have increased the number of devices close to the max of the /25 bit subnet.  It was a gradual increase.
 
That said the issue started a while ago but was very sporatic; sometimes I would run for months and other times for days.  My personal solution a while ago would be to just disconnect the HAI network connection for a day or so and plug it back in; it would always work for months afterwards just fine after this.
 
The "problem" / "issue"  though always the same and the same order; initially seeing a zero ambient temperature number on the legacy Omnitouch 5.7's, time going off sync and network connection becoming irratic.  I have a spare Omnitouch 5.7 on the Omnitouch hub and can watch the "zero" effect by just plugging in the network connection from the panel to the main network switch and watch it go away by plugging in the network cable back to the router internal network with no devices on it.
 
My programming is simple though mostly utilized for the lighting stuff.
 
The management vlan I wrote about will just be for stuff like switches; the HAI will probably remain behind a router. 
 
I did move all of the Omnitouch 5.7e's back to the main subnet and they can see the panel fine via the router.
 
 I have the second HAI panel on a way smaller network; IE: less than 10 devices and have not ever seen a similiar issue.
 
Personally the solution here is what will work for you; three seen here:
 
1 - get another panel and move IP cameras off the main HAI subnet (as PCT88 did with his stuff)
2 - move the HAI panel to a "place" behind a router (taking it to a layer 3 connection)
3 - send the board back to HAI for repairs - though there may not be anything to fix on it  - BTW the 10Mb HAI NIC connection only talks at half duplex - I could see that but not change that.
 
In my testing I did put the HAI logger program on the main subnet and connected to the HAI panel behind the router; and it worked just fine; such that a HAIKU server should work in the same fashion.  I am here sort changing things by the addition of more than two internal lans with direct connects to the firewall, multiple WAN connections for load balancing plus the addition of VLANs internally (which is really a bit much for a residential network of yesteryear; but may be not for today). 
 
Off on a tangent here again....
 
The LinuxMCE software sort of does this with two network interfaces; one in and one out sitting on your network; with all of the "consoles" / "Orbiters" only connecting to the internal network configured by one NIC on the Linux MCE box.
 
Personally this is optimal because it protects your consoles a bit from talking to the internet directly; or being dependant on the internet to function. 
 
Today for example the HAI devices connected serially to the HAI OPII panel do not directly speak to the internet as they are all serial connections (IE: keypad consoles, Omnitouch legacy 5.7's, Omnistat2 that I have plugged into the panel).  The legacy HAI video hub / Omnitouch hub has no network connections; albeit only a serial connection subpanel to HAI OPII panel wire on it.  BUT the technology of a serial connection is a bit antiquated and related to the speed of the serial port; but what the heck; you are not looking at that much back and forth data anyways.  Same goes for the UPB PIM, X10 PIM, Z-Wave PIM and Russound connections.  Even a fast Gb network interface on the HAI OPII board to said serial devices wouldn't really change the speed of your talking to these serial devices; unless maybe you changed the legacy serial port to USB3.0 ports on the panel.
 
I write about this; but then there is the "other side" of the HAI OPII console connections related to the Omnitouch IP connected consoles and software running on the network along with the IP HD cameras; thus here I am trying achieve that isolation in a virtual sense of sorts; but still playing much with it.

I am pretty much having this exact problem with my Omni Pro II v3.11 I bought in fall of 2013.  It wasn't really an issue until we started doing a lot of Netflix.
It works nearly perfect when direct connected with only a CAT5 cable to my laptop, When going through a switch (with only OPII and laptop, it drops about 15% of the packets.   when plugged into the rest of my network it drops about 70% of the packets.
 
Rather than start a new thread, I thought I'd check if anyone has come up with any new solutions or workarounds.   I see that there is a v3.13 firmware available.  Any chance of that solving the problem?
 
Thanks,
Warren