Very Cheap TED 1001 House Power Monitors (and possible hacking)

Changed the script a bit more:
 
 

#!/usr/bin/perl -W -s

system("stty -F /dev/ttyUSB0 1200 cs8 raw");

open INFILE, "/dev/ttyUSB0"
   or die "\nCannot open /dev/ttyUSB0!\n";


@data = ();
$buf = "";
# $cnt = 0;
$started = 0;
sub processPacket($);

while (read(INFILE, $buf, 1)) {
  $d = ord($buf) ^ 0xff;
  if ($d == 0x55 && $started == 0) {
    $started = 1;
    $a = 0;
  }
  if ($started == 1) {
    $data[$a] = $d;
    $sum += $data[$a];
    # printf("0x%02X ", $data[$a]);
    $a++;
    if ($a >= 11) {
      $sum -= $data[9];
      $sum &= 0xff;
      if ($sum == 0) { processPacket($data); }
      $started=1;
      $a = 0;
    }
  }
}

  # Working sub process packet stuff
  sub processPacket($) {
  local($data) = @_;
  #  If the power reading is way off, uncomment the other power line
  $power = (($data[5]^0xff)<<8) + ($data[4]^0xff);
  # $power = ($data[5]<<8) + $data[4];
  $voltage = ($data[8] << 8) | $data[7];
  $voltage = sprintf("%.3f",123.6 + ($voltage - 27620) / 85 * 0.4);
  $power = sprintf("%.3f",(1.19 + 0.84 * (($power - 288.0) / 204.0)));
  @tda = localtime(time);
  $td = ($tda[4]+1)."/$tda[3]/".(1900+$tda[5])." $tda[2]:$tda[1]:$tda[0]";
  print "$td, $power, $voltage\n";
  }

11/22/2013 23:46:43, 266.395, 122.447
11/22/2013 23:46:44, 266.408, 122.447
11/22/2013 23:46:45, 266.383, 122.447
11/22/2013 23:46:46, 266.379, 122.447
11/22/2013 23:46:47, 266.366, 122.447
11/22/2013 23:46:48, 266.404, 122.447
11/22/2013 23:46:49, 266.441, 122.447
11/22/2013 23:46:50, 266.420, 122.551
11/22/2013 23:46:51, 266.387, 122.447
11/22/2013 23:46:52, 266.362, 122.447
11/22/2013 23:46:53, 266.350, 122.447
11/22/2013 23:46:54, 266.387, 122.447
11/22/2013 23:46:55, 266.395, 122.447
11/22/2013 23:46:56, 266.375, 122.447
11/22/2013 23:46:57, 266.296, 122.447
11/22/2013 23:46:58, 266.309, 122.447
11/22/2013 23:46:59, 266.334, 122.447
11/22/2013 23:47:0, 266.354, 122.447
11/22/2013 23:47:1, 266.301, 122.339
11/22/2013 23:47:2, 266.325, 122.551
11/22/2013 23:47:3, 266.329, 122.551
11/22/2013 23:47:4, 266.338, 122.551
11/22/2013 23:47:5, 266.358, 122.447
11/22/2013 23:47:6, 266.317, 122.447
11/22/2013 23:47:7, 266.292, 122.447
11/22/2013 23:47:8, 266.313, 122.447
11/22/2013 23:47:9, 266.375, 122.447
11/22/2013 23:47:10, 266.395, 122.447
^C
 
I still don't understand why my data stream is so different - I guess I first need to try it on a Linux PC just to see if it's something kookie with the Mac. Pete, I tried your script as listed except for changing the tty port to match and changing the -F to a -f in the stty line (again to match the mac version of the command) I get no output what-so-ever, which isn't surprising considering the raw data that I get from the device (I inserted the colons for readability):
 
aa:df:81:13:cf:fc:d7:47:94:3:5c
aa:df:80:97:ce:fc:d7:47:94:2:da
aa:df:7f:b:d0:fc:d7:47:94:3:65
aa:df:7e:49:f3:fd:7e:31:94:3:73
aa:df:7d:cf:eb:fe:25:1b:94:3:64
aa:df:7c:c7:ec:fe:25:1b:94:3:6c
aa:df:7b:cf:eb:fe:25:1b:94:3:66
aa:df:7a:4b:ec:fe:25:1b:94:3:ea
aa:df:79:c7:ec:fe:25:1b:94:3:6f
aa:df:78:c7:ec:fe:7e:31:94:3:1
aa:df:77:d7:ea:fe:7e:31:94:2:f4
aa:df:76:cf:eb:fe:7e:31:94:2:fc
aa:df:75:4b:ec:fe:7e:31:94:3:80
aa:df:74:c7:ec:fe:7e:31:94:3:5
aa:df:73:4b:ec:fe:7e:31:94:3:82
aa:df:72:d9:7c:fd:d7:47:94:2:f7
aa:df:71:71:d3:fc:30:5e:94:3:9a
aa:df:70:69:d4:fc:d7:47:94:3:12
aa:df:6f:69:d4:fc:d7:47:94:3:13
 
I tried flipping the bits (1's compliment) as az1324 suggested but that didn't work either. If trying it on a Linux PC doesn't get me anywhere, I guess I'll pull the TED apart and make sure that I attached the jumper to the correct places...  :unsure:
 
BTW, here is the python code I'm using to generate the above output...
 
import serial
 
ser = serial.Serial('/dev/tty.usbserial-A4003wHw', 1200)
mask = 0xFF
 
while 1:
        tedPacket = ":".join("{0:x}".format(ord(c)) for c in ser.read(11))
        if tedPacket.split(':')[0] == "aa":
                print tedPacket
ser.close()
 
Terry
 
I guess I'll pull the TED apart and make sure that I attached the jumper to the correct places... 
 
Guessing if you are getting output then the jumper is wired correctly. 
 
Here I cut the trace then sanded it a bit right in front of the IC and soldered the jumper wire right to the trace.
 
I tried the earlier python scripts posted and am not getting anything but decoder errors.
 
Changed your script a bit but getting an error:

 
#!/usr/bin/env python

import serial

# Special bytes

# PKT_REQUEST = "\xAA"
# ESCAPE      = "\x10"
# PKT_BEGIN   = "\x04"
# PKT_END     = "\x03"


class TED(object):
    def __init__(self, device):
        self.port = serial.Serial(device, 1200, timeout=0)
        self.escape_flag = False


# ser=serial.Serial('/dev/ttyUSB0',1200)

mask = 0xFF

while 1:

        tedPacket = (":".join("{0:x}".format(c) for c in ser.read(11))
        # tedPacket = (":".join(format(ord(c),"0:x") for c in ser.read(11))

        if tedPacket.split(':')[0] == "aa":

                print tedPacket
 
root@ICS-ZM2:/tmp# ./test.py
  File "./test.py", line 28
    if tedPacket.split(':')[0] == "aa":
                                      ^
SyntaxError: invalid syntax
root@ICS-ZM2:/tmp#
 
 
I am looking in this direction.  This though was configured for the unlocked Ted 1001 device.
 
http://www.energycircle.com/blog/2009/05/27/ted-open-source-how-we-went-live-with-our-electricity-use
 
First piece to get working is this one:
 
http://www.bananabend.net/energy_detective/python/ted_orig.py
 
run it like this:
 
python ted_orig.py /dev/ttyUSB0
 
I think that the perl script above will print out the values you need for the below script.
 
Code:
#!/usr/bin/env python
#
# This is a Python module for The Energy Detective: A low-cost whole
# house energy monitoring system. For more information on TED, see
# http://theenergydetective.com
#
# This module was not created by Energy, Inc. nor is it supported by
# them in any way. It was created using information from two sources:
# David Satterfield's TED module for Misterhouse, and my own reverse
# engineering from examining the serial traffic between TED Footprints
# and my RDU.
#
# I have only tested this module with the model 1001 RDU, with
# firmware version 9.01U. The USB port is uses the very common FTDI
# USB-to-serial chip, so the RDU will show up as a serial device on
# Windows, Mac OS, or Linux.
#
# The most recent version of this module can be obtained at:
#   http://svn.navi.cx/misc/trunk/python/ted.py
#
# Copyright (c) 2008 Micah Dowty <[email protected]>
#
# Permission is hereby granted, free of charge, to any person obtaining a copy
# of this software and associated documentation files (the "Software"), to deal
# in the Software without restriction, including without limitation the rights
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
# copies of the Software, and to permit persons to whom the Software is
# furnished to do so, subject to the following conditions:
#
# The above copyright notice and this permission notice shall be included in
# all copies or substantial portions of the Software.
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
# THE SOFTWARE.
#

import serial
import time
import binascii
import sys
import struct


# Special bytes

PKT_REQUEST = "\xAA"
ESCAPE      = "\x10"
PKT_BEGIN   = "\x04"
PKT_END     = "\x03"

class ProtocolError(Exception):
    pass


class TED(object):
    def __init__(self, device):
        self.port = serial.Serial(device, 1200, timeout=0)
        self.escape_flag = False

        # None indicates that the packet buffer is invalid:
        # we are not receiving any valid packet at the moment.
        self.packet_buffer = None

    def poll(self):
        """Request a packet from the RDU, and flush the operating
           system's receive buffer. Any complete packets we've
           received will be decoded. Returns a list of Packet
           instances.

           Raises ProtocolError if we see anything from the RDU that
           we don't expect.
           """

        # Request a packet. The RDU will ignore this request if no
        # data is available, and there doesn't seem to be any harm in
        # sending the request more frequently than once per second.
        self.port.write(PKT_REQUEST)

        return self.decode(self.port.read(4096))

    def decode(self, raw):
        """Decode some raw data from the RDU. Updates internal
           state, and returns a list of any valid Packet() instances
           we were able to extract from the raw data stream.
           """

        packets = []

        # The raw data from the RDU is framed and escaped. The byte
        # 0x10 is the escape byte: It takes on different meanings,
        # depending on the byte that follows it. These are the
        # escape sequence I know about:
        #
        #    10 10: Encodes a literal 0x10 byte.
        #    10 04: Beginning of packet
        #    10 03: End of packet
        #
        # This code illustrates the most straightforward way to
        # decode the packets. It's best in a low-level language like C
        # or Assembly. In Python we'd get better performance by using
        # string operations like split() or replace()- but that would
        # make this code much harder to understand.

        for byte in raw:
            if self.escape_flag:
                self.escape_flag = False
                if byte == ESCAPE:
                    if self.packet_buffer is not None:
                        self.packet_buffer += ESCAPE
                elif byte == PKT_BEGIN:
                    self.packet_buffer = ''
                elif byte == PKT_END:
                    if self.packet_buffer is not None:
                        packets.append(Packet(self.packet_buffer))
                        self.packet_buffer = None
                else:
                    raise ProtocolError("Unknown escape byte %r" % byte)

            elif byte == ESCAPE:
                self.escape_flag = True
            elif self.packet_buffer is not None:
                self.packet_buffer += byte

        return packets


class Packet(object):
    """Decoder for TED packets. We use a lookup table to find individual
       fields in the packet, convert them using the 'struct' module,
       and scale them. The results are available in the 'fields'
       dictionary, or as attributes of this object.
       """
    
    # We only support one packet length. Any other is a protocol error.
    _protocol_len = 278

    _protocol_table = (
        # TODO: Fill in the rest of this table.
        #
        # It needs verification on my firmware version, but so far the
        # offsets in David Satterfield's code match mine. Since his
        # code does not handle packet framing, his offsets are 2 bytes
        # higher than mine. These offsets start counting at the
        # beginning of the packet body. Packet start and packet end
        # codes are omitted.

        # Offset,  name,             fmt,     scale
        (82,       'kw_rate',        "<H",    0.0001),
        (108,      'house_code',     "<B",    1),
        (247,      'kw',             "<H",    0.01),
        (251,      'volts',          "<H",    0.1),
        )

    def __init__(self, data):
        self.data = data
        self.fields = {}
        if len(data) != self._protocol_len:
            raise ProtocolError("Unsupported packet length %r" % len(data))

        for offset, name, fmt, scale in self._protocol_table:
            size = struct.calcsize(fmt)
            field = data[offset:offset+size]
            value = struct.unpack(fmt, field)[0] * scale

            setattr(self, name, value)
            self.fields[name] = value


def main():
    t = TED(sys.argv[1])
    while True:
        for packet in t.poll():
            print
            print "%d byte packet: %r" % (
                len(packet.data), binascii.b2a_hex(packet.data))
            print
            for name, value in packet.fields.items():
                print "%s = %s" % (name, value)

        time.sleep(1.0)

if __name__ == "__main__":
    main()
 
roussell said:
aa:df:81:13:cf:fc:d7:47:94:3:5c

 

I tried flipping the bits
If you invert the bytes you get:
 

55207EEC300328B86BFCA3



for which the checksum is correct, so looks right.
 
Here is what I see just using a serial port monitor:
 
Terminal log file
Date: 11/24/2013 - 10:39:09 AM
-----------------------------------------------
AA F3 C5 1F DE FC C7 06 96 03 38 C4 89 FC 1B 60
F7 AA C3 C7 FE B9 FF AA C2 E1 FF 60 A3 AA C1 D6
FF FF E0 AA C0 F0 FF FF 81 FF D7 FF FF 02 FF F3
F7 FF 02 F3 57 FC DB 02 FF F3 B7 CF DB 95 BB AA
F3 D7 FE FB 95 C8 AA F3 BA 85 E1 FC 16 DA 95 02
B8 FF AA F3 B9 67 E5 FC 16 DA 95 02 D3 AA F3 B8
6F E4 FC 6F F0 95 02 5E FF AA F3 B7 01 E2 FC 6F
F0 95 02 CF AA F3 B6 67 E5 FC 6F F0 95 02 67 AA
F3 B5 85 E1 FC 6F F0 95 02 4E AA F3 B4 67 E5 FC
C7 06 96 02 FA AA F3 B3 E5 A9 FB 84 76 96 02 8D
AA F3 B2 C9 74 FC 79 33 96 03 2C FE 7E E7 FC E9
FC 7E C8 45 D2 6E FF FE FE A1 FC 06 EE FE 4B B2
EB F3 C4 F3 8D FC 86 B9 F6 FF FF F3 C6 FB F3 60
AA F3 AB 51 31 FC 2B 60 96 04 0F F3 51 F1 19 24
F0 AA A9 C6 B9 E0 AA 57 19 FF B9 E0 AA E7 57 65
B1 F3 E6 C6 65 B0 98 F3 E5 99 FE 96 FC AA F3 A4
99 38 FC 2B 60 96 03 C7 FF 37 F3 67 FE FF 60 83
EA D7 65 90 D0 37 97 CE D2 49 C3 EA F8 74 96 C3
FA EE C7 96 C2 FF AA F3 9E C9 A6 FC 79 33 96 03
0E AA F3 9D D1 A5 FC 20 1D 96 03 77 AA F3 9C 3D
A8 FC 79 33 96 03 9A AA F3 9B 4D A6 FC 79 33 96
03 8D AA F3 9A B9 A8 FC 79 33 96 03 20 AA F3 99
45 A7 FC 79 33 96 03 96 75 6E 8D 18 79 33 96 03
84 AA FF A1 FC 9B 6E FF AA F3 96 F5 A0 FC 79 33
96 02 F0 AA F3 95 4D A6 FC 79 33 96 03 93 AA F3
94 D7 A4 FC 79 33 96 03 0C AA F3 93 D7 A4 FC 79
33 96 03 0D AA F3 92 D7 A4 FC 79 33 96 03 0E AA
F3 91 5B A4 FC 79 33 96 03 8B AA F3 90 EF A1 FC
79 33 96 02 FB AA F3 8F C9 A6 FC 79 33 96 03 1D

-----------------------------------------------
Date: 11/24/2013 - 10:40:03 AM
End log file
 
Initially I mentioned that I saw no X-10 noise.  I am taking that back.  I see a lot of noise.   Makes the device useless to me.
 
I am glad though that I only spent some $20 for it.
 
Another thing I noticed is that so far only one outlet works with the device and that outlet is adjacent to the fuse panel.
 
I don't really "need" to see my power use and would prefer my functioning X-10 (not a whole bunch these days) than to utilize this device.
 
@pete_C  I also don't like the idea of putting all that noise on the power line.  I'm going to open the MTU and see if a serial to USB converter can be connected inside, bypassing the power line link.
 
The constant signals caused my JV Engineering XTB-IIR X10 Repeater/Coupler to constantly flash its LED. Indicating high noise.
My XTBM X10 meter also shows things like high noise or corrupt messages.
 
When watching the raw output on mine. Sending an Insteon message on the power lines. Actually showed an output from the raw data on the USB port.
 
That's why I was thinking of eliminating the powerline modem and tapping into the signal at the measurement unit, then relaying it over cat5 or wireless. Actually maybe just cut out the measurement unit and interface directly to the CTs. That way it would be possible to implement some of the power signature detection as promised by that kickstarter project. Probably hard to get a better split core CT for 10 bucks.
 
Yup; here too payed a bit more attention to the JV Engineering XTB-IIR X10 Repeater/Coupler and the older XTB booster; both flashing like a Christmas tree.
 
The HS 2 box logs are chock-full of x10 noise entries.
 
11/25/2013 8:52:43 AM     X10 Received     J Hail Request
11/25/2013 8:52:43 AM     X10 Received     J Hail Request
11/25/2013 8:52:43 AM     X10 Received     J Hail Request
11/25/2013 8:52:44 AM     X10 Received     J Hail Request
11/25/2013 8:53:01 AM     X10 Received     P2 (?) P2 (?) P2 (?) P2 (?) P2 (?) P2 (?) P2 (?) P2 (?) P2 (?) P2 (?) P2 (?) P2 (?) P5 (?) P5 (?) P5 (?) P5 (?) P5 (?) P5 (?) P5 (?) P5 (?) P Off
11/25/2013 8:53:02 AM     X10 Received     P Off
11/25/2013 8:53:03 AM     X10 Received     O All Units Off
11/25/2013 8:53:03 AM     X10 Received     O All Units Off
11/25/2013 8:53:06 AM     X10 Received     I Hail Ack
11/25/2013 8:53:06 AM     X10 Received     I Hail Ack
11/25/2013 8:53:07 AM     X10 Received     I Hail Ack
11/25/2013 8:53:07 AM     X10 Received     I Hail Ack
11/25/2013 8:53:08 AM     X10 Received     I Hail Ack
11/25/2013 8:53:08 AM     X10 Received     I Hail Ack
 
Yup something like a Digi XBee connected right to the MTU serial output would probably work better than blasting the powerline with noise.
 
 

az1324 said:
That's why I was thinking of eliminating the powerline modem and tapping into the signal at the measurement unit, then relaying it over cat5 or wireless. Actually maybe just cut out the measurement unit and interface directly to the CTs. That way it would be possible to implement some of the power signature detection as promised by that kickstarter project. Probably hard to get a better split core CT for 10 bucks.
 
 
I think getting the signal after the "measurement unit"  would be best.  If you follow http://openenergymonitor.org/emon/ you see that getting an accurate high speed totalized voltage * current => watts is not a trivial problem. 
 
having problems with pete_c script under linux opensuse and ubuntu   always get the same error
 
./ted.pl
./ted.pl: line 4: syntax error near unexpected token `"stty -F /dev/ttyUSB0 1200 cs8 raw"'                                                                                                                                                                                     
./ted.pl: line 4: `system("stty -F /dev/ttyUSB0 1200 cs8 raw");'
 
would someone know the error ( copy and pasted exactly as posted by pete_c)
 
 
#!/usr/bin/perl -w -s

system("stty -F /dev/ttyUSB0 1200 cs8 raw");

open INFILE, "/dev/ttyUSB0"
   or die "\nCannot open /dev/ttyUSB0!\n";

@data = ();
$buf = "";
# $cnt = 0;
$started = 0;
sub processPacket($);
 
 
What happens if you type the stty command in at the bash prompt?

stty -F /dev/ttyUSB0 1200 cs8 raw

Also, most importantly, are you sure your /dev/USB0 is the correct device?
 
hi roussell
 thank you for your reply
 if I use moserial or cutecom I see the data come from the ted on /dev/ttyUSB0
example data

00000000: aa 31 80 bd 37 02 d9 ae 92 04 8c aa 31 7f df 36
00000010: 02 85 be 92 04 b0 aa 31 7e 50 37 02 05 cd 92 05
00000020: b0 aa 31 7d 40 36 02 57 c8 92 05 75 aa 31 7c 4e
00000030: 35 02 85 cc 92 05 37 aa 31 7b 31 38 02 59 c2 92
00000040: 05 88 aa 31 7a 50 35 02 37 b6 92 05 9b aa 31 79
00000050: 48 37 02 ef c3 92 04 dd aa 31 78 ec 36 02 47 c4
00000060: 92 04 e2 aa 31 77 82 36 02 35 bd 92 05 66 aa 31
00000070: 76 0d 36 02 75 c9 92 05 90 aa 31 75 49 36 02 49
00000080: c6 92 05 84 aa 31 74 98 36 02 05 d1 92 05 6f aa
00000090: 31 73 d5 35 02 b5 dc 92 04 79 aa 31 72 b1 36 02
000000a0: ff e6 92 04 49

is run in terminal  stty -F /dev/ttyUSB0 1200 cs8 raw  it finishes with out error and displays nothing  and if I use stty -a -F /dev/ttyUSB0 it shows the ubs0  has being changed to 1200 baud  from the default  9600

speed 1200 baud; rows 0; columns 0; line = 0;
intr = <undef>; quit = <undef>; erase = <undef>; kill = <undef>; eof = <undef>; eol = <undef>; eol2 = <undef>; swtch = <undef>; start = <undef>; stop = <undef>; susp = <undef>; rprnt = <undef>; werase = <undef>; lnext = <undef>; flush = <undef>; min = 1; time = 0;
-parenb -parodd cs8 -hupcl -cstopb cread clocal -crtscts
-ignbrk -brkint -ignpar -parmrk -inpck -istrip -inlcr -igncr -icrnl -ixon -ixoff -iuclc -ixany -imaxbel -iutf8
-opost -olcuc -ocrnl -onlcr -onocr -onlret -ofill -ofdel nl0 cr0 tab0 bs0 vt0 ff0
-isig -icanon -iexten -echo -echoe -echok -echonl -noflsh -xcase -tostop -echoprt -echoctl -echoke


 
by the way  did you get it to work for you. as I see the  code    does invert the  hex   using  0xff so the response of aa your were getting was correct.. as it was inverted in the script later on..
 
 
Back
Top