Pulling Electrical Data off a TED MTU

Pulling Electrical Data off a TED MTU

Problem

I got a TED (The Energy Detective) MTU (Measuring Transmitting Unit) for cheap and hooked it up a few years ago. If you're not familiar, that consists of putting little inductive clamps around the main lines that feed your electrical panel from the meter. Those clamps are hooked into a little black box called a CT (Current Transmitter) that is powered by one of the circuits in the box. That box measures your home's energy usage (instantaneous current) using induction and also measures your line voltage (since it's wired in anyway). It then transmits this data over a circuit to the MTU. The MTU records it and runs a little webserver where you can access that data. Here's a picture of all this stuff, if you're curious:

Kinda cool... somewhat useful. e.g. if you have a laptop around you can fire up a web browser and look at your current and historical energy use. You can also tell it about your power rates and it will estimate your bill.

The webpage the little box hosts is pretty nice but I found that I never really looked at it. It is also heavyweight javascript that takes a long time to render in an iframe on my raspberry pi. In short, I wanted to make a display that would tell me this same information quickly, on a "dedicated display" (Note: the TED people also sell displays, if you're not interested in rolling your own). This is an article about how to roll your own, though (for you, if you care and for me, in case I ever have to redo it).

Getting the Data

One of the nice things about this TED stuff is they have a nice RESTful XML API you can poll to read raw data off that little webserver box and have documented it. This is really cool -- I love when products make their functionality available to programmers without forcing you into a closed ecosystem. Here's a sample of the XML data for the /api/LiveData.xml result. Because that XML data is available, I was able to write a little python script to poll it every minute (in a cronjob), stuff it into a RRD (Round Robin Database), and build my own display for the data.

Here's the script to fetch the data off the TED server over the REST API:

#!/usr/local/bin/python

import rrdtool
import urllib2
import xml.etree.ElementTree as ET

DATABASE = 'ted_readings.rrd'

f = urllib2.urlopen('http://ted/api/LiveData.xml')
raw = f.read()
cooked = ET.fromstring(raw)
f.close()

voltage = int(cooked.find("Voltage").find("Total").find("VoltageNow").text)
power = int(cooked.find("Power").find("Total").find("PowerNow").text)
cost = int(cooked.find("Cost").find("Total").find("CostNow").text)

if voltage < 0 or voltage > 400:
    voltage = 0
if power < 0 or power > 200000:
    power = 0
if cost < 0 or cost > 1000:
    cost = 0

template = "voltage:power:cost"
update = "N:%d:%d:%d" % (voltage, power, cost)
rrdtool.update(DATABASE, "--template", template, update)

This reads the raw data from that REST API and stuffs it into a RRD file. I'm using a program called rrdtool to store the historical timeseries data and also to generate graphs of the data. This is a tool I've used before, e.g. in the cabin temperature project, and I really like. It's somewhat complicated (read the man pages and look at the examples) but it works well, is flexible, has python and perl library support and basically does a nice job.

For reference, this is how I created the RRD file I'm populating here. It creates three gauges (voltage, power and cost) corresponding to the three data series I care about from the TED device and keeps them for different time windows. It also makes available the last N raw readings (N=10000). The resultant file is something like 15M of disk space.

% rrdtool create ted_readings.rrd \
        --start 1371018212 \
        --step 60 \
        DS:voltage:GAUGE:900:0:500 \
        DS:power:GAUGE:900:0:1000000 \
        DS:cost:GAUGE:900:0:100000 \
        RRA:AVERAGE:0.5:1:525600 \
        RRA:AVERAGE:0.25:60:87600 \
        RRA:LAST:0:1:10000 \
        RRA:LAST:0.1:20:10000

Making Pretty Graphs

The purpose of this project was to make the current electrical usage more visible than hiding it on a webserver that I never really looked at. I wanted to make it into a page on my kitchen kiosk that would pop up now and again to tell me how much energy I'm using right now and also give me a sense of what is "normal baseline" electrical usage. To do this, I generate four graphs out of the data in the RRD using a script:

#!/bin/bash

RRDTOOL=/usr/local/bin/rrdtool
WWW=/path/to/my/webserver/root

$RRDTOOL graph $WWW/ted_current.png \
  --title "Home Electrical Use, ~Live" \
  --vertical-label "watts" \
  --right-axis 0.025:1 --right-axis-label "volts" \
  --lower-limit 500 \
  --start now-20m --end now \
  --width=640 --height=480 \
  --step=60 \
  --slope-mode \
  --force-rules-legend \
  --watermark "`date`" \
  --font DEFAULT:9: \
  DEF:base_voltage=/home/scott/cron/ted_readings.rrd:voltage:AVERAGE \
  DEF:last_voltage=/home/scott/cron/ted_readings.rrd:voltage:LAST \
  CDEF:voltage=base_voltage,40,* \
  DEF:power=/home/scott/cron/ted_readings.rrd:power:AVERAGE \
  DEF:last_power=/home/scott/cron/ted_readings.rrd:power:LAST \
  DEF:cost=/home/scott/cron/ted_readings.rrd:cost:AVERAGE \
  GPRINT:last_voltage:LAST:"current line\: %3.0lf volts" \
  GPRINT:last_power:LAST:"current usage\: %3.0lg%s watts" \
  AREA:1100#DFFFDF \
  HRULE:1100#009900:"baseline" \
  LINE3:power#800000:"power" \
  LINE3:voltage#000099:"voltage"

$RRDTOOL graph $WWW/ted_one_day.png \
  --title "Home Electrical Use, Past Day" \
  --vertical-label "watts" \
  --right-axis 0.025:1 --right-axis-label "volts" \
  --lower-limit 500 \
  --start now-1d --end now \
  --width=640 --height=480 \
  --step=300 \
  --slope-mode \
  --force-rules-legend \
  --watermark "`date`" \
  --font DEFAULT:9: \
  DEF:base_voltage=/home/scott/cron/ted_readings.rrd:voltage:AVERAGE \
  DEF:last_voltage=/home/scott/cron/ted_readings.rrd:voltage:LAST \
  CDEF:voltage=base_voltage,40,* \
  DEF:power=/home/scott/cron/ted_readings.rrd:power:AVERAGE \
  DEF:last_power=/home/scott/cron/ted_readings.rrd:power:LAST \
  DEF:cost=/home/scott/cron/ted_readings.rrd:cost:AVERAGE \
  GPRINT:last_voltage:LAST:"current line\: %3.0lf volts" \
  GPRINT:last_power:LAST:"current usage\: %3.0lg%s watts" \
  AREA:1100#DFFFDF \
  HRULE:1100#009900:"baseline" \
  LINE3:power#800000:"power" \
  LINE3:voltage#000099:"voltage"

$RRDTOOL graph $WWW/ted_one_week.png \
  --title "Home Electrical Use, Past Week" \
  --vertical-label "watts" \
  --right-axis 0.025:1 --right-axis-label "volts" \
  --lower-limit 500 \
  --start now-7d --end now \
  --width=640 --height=480 \
  --step=1850 \
  --slope-mode \
  --force-rules-legend \
  --watermark "`date`" \
  --font DEFAULT:9: \
  DEF:base_voltage=/home/scott/cron/ted_readings.rrd:voltage:AVERAGE \
  DEF:last_voltage=/home/scott/cron/ted_readings.rrd:voltage:LAST \
  CDEF:voltage=base_voltage,40,* \
  DEF:power=/home/scott/cron/ted_readings.rrd:power:AVERAGE \
  DEF:last_power=/home/scott/cron/ted_readings.rrd:power:LAST \
  DEF:cost=/home/scott/cron/ted_readings.rrd:cost:AVERAGE \
  GPRINT:last_voltage:LAST:"current line\: %3.0lf volts" \
  GPRINT:last_power:LAST:"current usage\: %3.0lg%s watts" \
  AREA:1100#DFFFDF \
  HRULE:1100#009900:"baseline" \
  LINE3:power#800000:"power" \
  LINE3:voltage#000099:"voltage"

$RRDTOOL graph $WWW/ted_one_month.png \
  --title "Home Electrical Use, Past Month" \
  --vertical-label "watts" \
  --right-axis 0.025:1 --right-axis-label "volts" \
  --lower-limit 500 \
  --start now-1m --end now \
  --width=640 --height=480 \
  --step=8400 \
  --slope-mode \
  --force-rules-legend \
  --watermark "`date`" \
  --font DEFAULT:9: \
  DEF:base_voltage=/home/scott/cron/ted_readings.rrd:voltage:AVERAGE \
  DEF:last_voltage=/home/scott/cron/ted_readings.rrd:voltage:LAST \
  CDEF:voltage=base_voltage,40,* \
  DEF:power=/home/scott/cron/ted_readings.rrd:power:AVERAGE \
  DEF:last_power=/home/scott/cron/ted_readings.rrd:power:LAST \
  DEF:cost=/home/scott/cron/ted_readings.rrd:cost:AVERAGE \
  GPRINT:last_voltage:LAST:"current line\: %3.0lf volts" \
  GPRINT:last_power:LAST:"current usage\: %3.0lg%s watts" \
  AREA:1100#DFFFDF \
  HRULE:1100#009900:"baseline" \
  LINE3:power#800000:"power" \
  LINE3:voltage#000099:"voltage"

These generate graphs like this one (not live):

You can take a look at the rrdgraph, rrdgraph_data, and rrdgraph_examples man pages to figure most of this out but I want to explain one thing that was a little tricky: the right axis.

It's easy to make a right axis on your graph; just use the --right-axis and --right-axis-label arguments. The former takes a scaling factor that defines the scale of the right axis as a proportion of the main left axis. In my case, I wanted voltage on the right axis. Nominal voltage is 120 volts while nominal instantaneous power usage is something like 1k watts. Therefore I made the scaling factor 0.1 (1/10).

What wasn't easy was to figure out how to get one of the lines on the graph to "use" the right axis. As far as I can tell, there's no simple way to do it. All lines always plot relative to the left (main) axis. Instead, what you do is to scale the data you're graphing to match the scale of the axis you want to use. In my case, voltage had to be multiplied by 10. That way it would plot on the right spot relative to the left axis and "look like" it was using the left axis. That's the purpose of these parts:

  DEF:base_voltage=/home/scott/cron/ted_readings.rrd:voltage:AVERAGE \
  CDEF:voltage=base_voltage,10,* \
  LINE2:voltage#000099:"voltage" \

That's pulling in a data series called "base_voltage" then deriving another series called "voltage" by multiplying base_voltage by 10. The scaled out "voltage" is the line I then plot on the graph. It will plot relative to the left (main) power axis but appear to be correct relative to the right (alternate) voltage axis since that axis is 1/10th the scale of the main one.