Here's what I've gleaned from nebulous's capture so far.
The primary devices are 5001 (AC) ,2001 (Stat), 4201 (Furnace). 1F01 does a lot of interrogation up front, including capturing the furnace model number, but after startup remains silent for the capture (at least for the startup log). It seemed to interrogate a lot of other addresses that I don't recall on mine, but I'll have to verify that (new ones for me are: 5101, 4202, 6001, 6101, 0E01, 2401, 2501, 2601, 2701, 2801 - none of the devices respond in his capture). 2001 still manages the time of the system using broadcasts, like mine. 2001 also captures the model number like mine. I don't remember 1F01 on my system, but I'll need to do a startup capture myself again.
I also forgot how crappy it is there is no coordination among devices - They all just seem to talk whenever they want and rely on retries and repeating the same thing alot to ensure communication. This makes it really hard to parse at times, and very difficult to determine when to inject packets into the live comm. I have yet to see a rhyme or reason to this - I keep dwelling on it because it seems like really poor design, but I've yet to come up with a better theory.
As for the tools for this effort here's what I've thought of, but I don't have a ton of working code.
I've been working towards a tool, that can:
1) capture and optionally store raw data
2) interpret data in real time
3) inject packets into the stream
4) watch each register/table for changes, and only show those
5) allow filters to hone on on specific data
My hope is that the tool would be able to capture the changes in raw data and store it in a database with a timestamp. Then, as the mappings are determined you could make trends of data you have been capturing but didn't necessarily know the meaning of at the time. If it is in a database over time, you could also look back at specific bytes and only focus on ones that change to try to interpret them. Part of what I have struggled with is how to store the interpretations of data in a way that is useful and forward looking. Right now my file looks like this:
2001,SYSTIME ,02,01,13,01,1,19
2001,SYSTIME ,02,02,03,03,1,1,Hours,>B
2001,SYSTIME ,02,02,03,03,2,1,Minutes,>B
2001,SYSTIME ,02,02,03,03,3,1
2001,SYSTIME ,02,03,03,03,1,1,Day,>B
2001,SYSTIME ,02,03,03,03,2,1,Month,>B
2001,SYSTIME ,02,03,03,03,3,1,Year,>B
2001 is the device from which the request comes
SYSTIME is any text, but it refers to the table name
02 is the table address
the next column is the row in the table
the next column is the length of the row (number of bytes in hex)
the next column is a value from the table structure register (generally at a pattern of 0x??01) I think it relates to the number of values in a row, but it isn't consistent.
Then you have the byte to start, and how many bytes to take, a name to give it, and how to interpret the data (a string that feeds into a python function)
The problem is I am expecting values to be within a byte or bytes, where it is completely possible for a byte to contain multiple pieces of data. I'm considering adding a bitmask and bitwise shift value to deal with this.
I realize I'm getting ahead of myself with my nomenclature and have made alot of assumptions in this post so far. I'll digress into that for a minute.
The data in the system is somewhat self describing. The register as I call it is generally 2 bytes - a table and a row. The first row defines the table, and the following rows have data. This table identical in device 5001 (my AC unit), but it is where a broadcast from 2001 (my stat) write the data every minute or so.
0x0201 00 21 53 59 53 54 49 4d 45 20 00 19 03 13 01 03 03 03 03
0x0202 0d 36 06
0x0203 05 0c 0a
So 0x02 is the table, and the following byte is the row. I'll walk across the first row and show how it defines the table. You can only get this row by asking for it from the device.
I don't recall what 0021 is.
The next 8 bytes are the name of the table = 53595354494d4520 is SYSTIME with a trailing space (ascii)
00 19 relates to the size of the table (25 bytes)
03 is the total number of rows in the table
13 is the bytes of this row (in hex) and 01 is unknown
03 is the bytes in the second row, and 03 is unknown
03 is the bytes in the third row, and 03 is unknown.
The second byte of the last three lines of my description might relate to the number of pieces of data in this row, although I haven't confirmed this. In this case row 2 each byte represents hour, minute, and day of week, and in row 3 each byte is day, month, year. This was discovered by watching data, and intrepreting the table name. This one is a good example to start with and everything is pretty clear, but other tables are more difficult.
