Enumerating Z-wave nodes with VRC0P

[toro]

I just connected a VRC0P to my PC with the intention of using it to programmatically toggle a few HM-AM001 appliance modules scattered throughout the home.

If I know the node number for the AM I want, it is easy enough to turn the module on/off: ">N5ON" ">N5OFF"

But I am having some issues obtaining a list of node numbers from the VRC0P:

- ">?N" lists on/off/dimmer status for somewhere between 2-5 of the nodes, but not all of them

- LumenLink takes several hours to probe the network, and AFAICT they are doing it incorrectly anyway. Their strategy is to send 4,176 total commands in the form of: ">?FIx,y,0,z", with z ranging from 1-232. This usually times out after 1-2 seconds. On my network this procedure does not work, as all of the nodes respond to instance 1 only.

- I suppose I could hard-code the node numbers on the PC, but it seems like they change around when I muck with my primary remote. I would rather have a way to uniquely identify each appliance module so I don't ever have to change the associations on the PC side.

A few additional questions:

- Is it possible to query each node for a unique serial number? How about manufacturer, class codes, etc.? The only data I seem to get from ">?FI" is the number of matches.

- Are there any interesting "SE" commands for appliance modules?

- When I program a primary/secondary remote, it has the ability to detect when I have pushed the button on one of the appliance modules. Is there a way to do this on ZRC0P?

- Has anybody else noticed that with old VRC0P firmwares ($Leviton(C) 2008 V2.2S/Z-Wave 3.11) there is an off-by-one bug regarding the node numbers?

- All of this seemed so much easier with X10, because X10 let me set the numbers myself. How do Z-Wave users cope with the fact that the network auto-assigns the node numbers and it isn't under your control?

My VRC0P is running the latest FW from leviton.com: $Leviton(C) 2008 V2.31S/Z-Wave 3.11

 

[intozwave]

Welcome to the journey this has been a great place to gather information and share experiences. What Handheld remote are you using as the Primary in your network? The primary assigns a node # to the device when it is added to the network. If you are adding then deleteing devices or resetting the primary controller node #'s will change. One of the nice features of Leviton devices is 2-way communication or instant status - the HM-AM001 appliance module does not have this capability and needs to be polled to know what state it is in.

 

[Frunple]

Did you do the rs-232 setup of the vrc0p? Older versions will say HAI setup.

 

[etc6849]

There's an opensource module for the VRC0P and VRC0Pv3 found here:
http://www.cocoontech.com/forums/index.php?showtopic=19134&view=findpost&p=159044

I would read the first three z-wave threads found in this forum too:
http://www.cocoontech.com/forums/index.php?showforum=51

The short answer to your question is use the find command, have your driver create objects for each device found. When you want to turn off all dimmers, just iterate through the created objects.

However, I would definitely study the module linked above in detail, but you'll need to install Premise which is free to study it... There's even a built in port spy program to study things. There's no need to do 4700 commands to find all nodes, so I encourage you to study the Premise module's structure.

PS: as others have stated ensure the VRC0P is associated and that it has been updated.

 

[electron]

Sounds like he has the old non-upgradeable VRC0P unit tho.

 

[toro]

Thanks all for the quick feedback. Replies inline.

What Handheld remote are you using as the Primary in your network? The primary assigns a node # to the device when it is added to the network. If you are adding then deleteing devices or resetting the primary controller node #'s will change.

GE 45601

Are there other primary controllers on the market that would give me more control over the node numbering?

One of the nice features of Leviton devices is 2-way communication or instant status - the HM-AM001 appliance module does not have this capability and needs to be polled to know what state it is in.

It would be nice to have instant status reporting, but I need a bunch of these AMs. It was the low cost of the AMs and of the GE 45600 (secondary) remotes which finally drove me to upgrade from X10 to Z-Wave.

Did you do the rs-232 setup of the vrc0p? Older versions will say HAI setup.

The only setup I performed was to add VRC0P to the network as a secondary controller. i.e. hold down the VRC0P button until the LED flashed amber, then TRANSFER -> SECONDARY -> NTWK ONLY on the 45601.

Based on my reading of the VRC0P API document, I got the impression that doing an "ALL" transfer would not teach the VRC0P the keypad->node mappings from the 45601.

The short answer to your question is use the find command, have your driver create objects for each device found.

I will check out the referenced threads... but could you please provide a quick example of how to use the find command to list my devices? I just want to see the raw data.

Sounds like he has the old non-upgradeable VRC0P unit tho.

It is the +3 unit. I have upgraded it to the latest FW from Leviton's site, and verified that the version banner changed to V2.31S.

BTW: has anybody managed to disassemble the .hex files and map out interesting parts of the FW? Looks like Thumb assembly for the Cortex M3 on the STMicro controller.

 

[etc6849]

Unfortunately, I'm not at home to paste the FI results. After finding all devices, the driver begins fetching node names. The protocol used is very similar to what's in the VRC0P documentation. Look for switches -> keep augmenting nodeID until <F000; look for dimmers->keep augmenting nodeID until <F000; look for thermostats; look for sensors; etc... You must have the basic type for whatever device type to use this method. That's why I suggested you examine the Premise module as it already has this information... You could also examine the source to the openz-wave project for these special integer values.

It really is easy to install Premise, the VRC0P module, and use portspy to view what's going on. The module is also event based and is very easy to understand as it uses a vbscript like language.

The GE 45601 is a poor primary controller in my opinion. I would recommend one of the PC based USB primary controllers such as the one by Leviton (it's less than $50 I think) along with their free "RF Installer Tool" software. Once you set up the network, you can always unhook the primary controller and everything will work fine. The Leviton USB stick also supports beaming, I'm not sure that the GE controller does... The Leviton software provides a means to insert a "Future" nodeID, and can remove/replace a given node. If you want to try it, you could add x future nodes, then add each device to whatever future node you want. This gives more control over node numbering... No solution that I know of lets you assign a given nodeID however you want as this could potentially break things if you accidently picked a nodeID already in use.

If you can examine the firmware's source, you would have found a great resource for the z-wave community as the source code would show a complete implementation of how to use the Zensys Serial API. Leviton's protocol is much simpler and the details of the Zensys Serial API are apparently kept secret unless you spend thousands of dollars. If you can research the firmware and understand it, please let us know If we can replicate what the firmware does, we could use any serial controller (as the other serial controllers use the Zensys Serial API directly) with Premise or other HA programs.

 

[toro]

Unfortunately, I'm not at home to paste the FI results. After finding all devices, the driver begins fetching node names. The protocol used is very similar to what's in the VRC0P documentation. Look for switches -> keep augmenting nodeID until <F000; look for dimmers->keep augmenting nodeID until <F000; look for thermostats; look for sensors; etc... You must have the basic type for whatever device type to use this method. That's why I suggested you examine the Premise module as it already has this information... You could also examine the source to the openz-wave project for these special integer values.

For future reference, many of them are on the Linux MCE wiki:

wiki.linuxmce.org/index.php/ZWave_API

Basic classes:

Controller         1
Static controller  2
Slave              3
Routing Slave      4

Device types:

CONTROLLER       0x01 (like the ZTH200,    Classes: )
SWITCH/APPLIANCE 0x10 (ZRP200,             Classes: 25, 27, 75)
DIMMER           0x11 (ZDP200, ZDW230, .., Classes: 26, 27, 75, ...)
TRANSMITTER      0x12
THERMOSTAT       0x08
SHUTTER          0x09

LumenLink also probes for a few others which I do not know:

Class 1, device type 0x00 (isn't controller supposed to be type 0x01?)
Class 2, device type 0x02
Class 3, device type 0x07
Class 3, device type 0x13
Class 3, device type 0x20
Class 3, device type 0x21
Class 4, device type 0x07
Class 4, device type 0x13
Class 4, device type 0x20
Class 4, device type 0x21

But since it takes a second or two to time out on each FI operation, this is a very slow way of taking inventory.

It really is easy to install Premise, the VRC0P module, and use portspy to view what's going on. The module is also event based and is very easy to understand as it uses a vbscript like language.

This is basically what I did with LumenLink. I'll try Premise next and see what can be gleaned from it.

The GE 45601 is a poor primary controller in my opinion. I would recommend one of the PC based USB primary controllers such as the one by Leviton (it's less than $50 I think) along with their free "RF Installer Tool" software. Once you set up the network, you can always unhook the primary controller and everything will work fine.

Any chance that the communication protocol for this device is documented?

I chose VRC0P because it looked like it had the most open API.

If you can examine the firmware's source, you would have found a great resource for the z-wave community as the source code would show a complete implementation of how to use the Zensys Serial API.

I'm sure we will never see the source code without an NDA, but we do have hex files (binary object code + load addresses) for both components.

We do not have symbols, and the number of ascii strings is limited. But the ST file is only ~25KB, and the Zensys file is ~64KB. It might not be too hard to trace backward and figure out how a couple well-known commands are implemented under the hood.

 

[etc6849]

If you examine the xml files for the open z-wave project, you might find the other integer values.

The Prmeise module works very well. It was built by two smart folks (123 and Markh) and me. It sounds like the LumenLink implementation wasn't as well thought out as the Premise module. The Premise module uses a FIFO buffer for commands; it waits for commands such as <X000, <E000, <F00x, etc before sending the next job. In this manner, it doesn't rely on timers, but it does have a job timeout feature just in case. It even has the capability to retry failed jobs (<X002 or jobs that timeout) automatically and will even reset the serial port following x consecutive failed jobs.

Reverse engineering the firmware is not a simple task, especially given the complexity of beaming. Beaming is a necessity in my mind as I must be able to use locks with my setup. If you figure out it though, you would be the hero of the z-wave community!

 

[etc6849]

PS: The only z-wave controllers that I know of that are open are:

1. Cooper AspireRF RS232 controller (no beaming support)
2. VRC0Pv3 (beaming support)
3. ControlThink USB stick + SDK (I don't think the SDK supported beaming, and it's been discontinued since Leviton bought ControlThink). The ControlThink stick itself does support beaming.

 

[electron]

Ah sorry, my bad, the 2008 copyright threw me off.

 

[toro]

If you examine the xml files for the open z-wave project, you might find the other integer values.

Beaming is a necessity in my mind as I must be able to use locks with my setup.

I should forewarn you that my use case is rather narrow, and I don't think I'd ever want to give hackers a way to unlock my front door and walk right inside (however unlikely the scenario may be).

BTW I see the SS command for beaming, but where the heck is the key management and authentication handled?

The Prmeise module works very well. It was built by two smart folks (123 and Markh) and me. It sounds like the LumenLink implementation wasn't as well thought out as the Premise module. The Premise module uses a FIFO buffer for commands; it waits for commands such as <X000, <E000, <F00x, etc before sending the next job. In this manner, it doesn't rely on timers, but it does have a job timeout feature just in case. It even has the capability to retry failed jobs (<X002 or jobs that timeout) automatically and will even reset the serial port following x consecutive failed jobs.

That was helpful. After sniffing the Premise traffic I am starting to get a better feel for how device discovery works:

>?FI0,16,0,1
<E000
<F002
>?FI0,16,0,2
<E000
<F004
>?FI0,16,0,3
<E000
<F007
>?FI0,16,0,4
<E000
<F008
>?FI0,16,0,5
<E000
<F009
>?FI0,16,0,6
<E000
<F000

So you iterate through the routing table entries for basic class bbb, generic class ggg. If you get Fxxx (nonzero), xxx is a node number of a matching device. If you get F000, you know that there are no more matches in the routing table. This takes advantage of the fact (assumption?) that there are no gaps in the instance number space.

I suspect this is what LumenLink was "supposed" to be doing, but for whatever reason it kept going even after it hit F000.

Reverse engineering the firmware is not a simple task, especially given the complexity of beaming.

Agreed, it's not trivial at all. But consider:

- You can start with things you do understand (maybe simple ON/OFF commands) and start using that knowledge to map out the FW internals

- You (or somebody else) can build on that knowledge and start putting more pieces of the puzzle together over time

- Maybe you'll find debug features in the FW, or be able to add them

- 25KB of Thumb2 code is pretty dense, but it's not nearly as bad as trying to reverse engineer e.g. the latest NVIDIA graphics driver (easily 1MB+ of binary code)

- App notes and datasheets for the microcontroller in question (STM32F103C8) are freely available

Do you happen to know anything about the routing commands: R, RO, DR?

 

[etc6849]

I should forewarn you that my use case is rather narrow, and I don't think I'd ever want to give hackers a way to unlock my front door and walk right inside (however unlikely the scenario may be).

Premise uses encryption if you enable it. Z-Wave Beaming also uses encryption. For anyone to hack your Z-Wave network, they'd also need extensive proprietary knowledge and that's assuming it's possible without direct access to your controller and lock's programming buttons.

BTW I see the SS command for beaming, but where the heck is the key management and authentication handled?

It's handled by the VRC0Pv3.

So you iterate through the routing table entries for basic class bbb, generic class ggg. If you get Fxxx (nonzero), xxx is a node number of a matching device. If you get F000, you know that there are no more matches in the routing table. This takes advantage of the fact (assumption?) that there are no gaps in the instance number space.

Unless your routing table is corrupt, I believe this should always be the case.

It probably wasn't designed for the VRC0Pv3 since they changed the protocol a little since the VRC0P.

I suspect this is what LumenLink was "supposed" to be doing, but for whatever reason it kept going even after it hit F000.

Agreed, it's not trivial at all. But consider:

- You can start with things you do understand (maybe simple ON/OFF commands) and start using that knowledge to map out the FW internals

- You (or somebody else) can build on that knowledge and start putting more pieces of the puzzle together over time

- Maybe you'll find debug features in the FW, or be able to add them

- 25KB of Thumb2 code is pretty dense, but it's not nearly as bad as trying to reverse engineer e.g. the latest NVIDIA graphics driver (easily 1MB+ of binary code)

- App notes and datasheets for the microcontroller in question (STM32F103C8) are freely available

I hope you can figure this out and that you'd be willing to share I don't have embedded systems experience; it sounds like you do so I hope you're successful!

Do you happen to know anything about the routing commands: R, RO, DR?

RO is used to set routes. It's implemented in the Premise module, but you have to manually click set routes. Where did you here about DR and R? I don't remember those...

 

[toro]

Z-Wave Beaming also uses encryption.

So does 802.11 WEP...

I hope the Z-Wave designers did a better job than the WEP designers.

For anyone to hack your Z-Wave network, they'd also need extensive proprietary knowledge and that's assuming it's possible without direct access to your controller and lock's programming buttons.

It is highly unlikely for a random burglar to hack their own way into my house with a 0-day exploit, but if there are holes lurking in the protocol, eventually somebody will write a program that makes it a one-click procedure that anybody can use.

I would be particularly concerned about people who have been inside your house, know what kinds of valuables are inside, and are aware that you have remote controlled locks on the doors. e.g. your teenage son's punk friends.

RO is used to set routes. It's implemented in the Premise module, but you have to manually click set routes. Where did you here about DR and R? I don't remember those...

There is some help text in the firmware (">H"):

-UP     - Update
-ON     - Turn On
-OF     - Turn Off
-Lxxx   - Set level
-GSxxx  - Store Group
-DE     - Default
-IN     - Include
-AB     - Abort
-Nxxx  - Node ID
-APxxx - Add node to list
-SE(SS)x,..- Send raw mesage
-Rn1,[n2..n4]- Route
-PRd,n1,[n2..n4]- Store route
-DRd - Remove route
-DI     - Dim
-BR     - Bright
-ST     - Stop
-GRxxx  - Group Recall
-FIb,g,s,i-Find node
-ROs[,d]-  Assign/Delete route
-Ss[,f] - Call scene
-PSs[,f]- Program scene 
-NN1ab..- Node name
-NL1ab..- Node location

Your friend at Leviton might want to fix the spelling of "mesage".

Are the routing changes local to the VRC0P, or do they propagate out to the nodes and affect the way non-VRC0P traffic is passed?

Have you ever gotten an "x026" return code (lower case 'x')?

 

[toro]

The GE 45601 is a poor primary controller in my opinion. I would recommend one of the PC based USB primary controllers such as the one by Leviton (it's less than $50 I think) along with their free "RF Installer Tool" software.

I bought a Leviton CTZUS-1US USB ThinkStick. When I try to run the Vizia RF + Installer Tool, it says "The selected installer stick is not supported"

I guess I should have bought a VRUSB-1US instead?

Same problem was reported here:

communities.leviton.com/thread/1605

But this may be a generic error that means "I don't see a compatible USB stick attached to the system."