Monitoring electricity usage at the meter
Updated on .
I’d like to know how much energy I’m using to ensure that the baseline load for my activities is reasonable. Since there’s already a meter attached to my electrical service for the purposes of billing, it made sense to use that instead of adding monitoring devices to my electrical circuits.
Unfortunately, it’s difficult to correlate instantaneous usage with the inherent delay involved with the data showing up on my utility’s (Pacific Gas & Electric) website. PG&E’s smart meters can use Zigbee to stream data, which seems relatively rare in the US 1 Double-check that your electricity meter and utility support wireless access before taking this approach and buying any products.. I bought a Rainforest Automation EMU-2 and connected it to a Raspberry Pi that sends realtime data to a self-hosted Grafana dashboard. When that device stopped working, I also registered for self-access PG&E’s Share My Data service and wrote a service to send historical data to InfluxDB.
Rainforest Automation EMU-2
The EMU-2 transmits usage data over a USB serial port at a prescribed rate, about once every 10 seconds. The instantaneous demand can be queried with a request packet, and even settings like the data rate changed, but these features don’t work well in practice. The EMU-2 doesn’t queue writes to the serial port and an interrupt on the device to emit a scheduled response will clobber whatever was in the process of being written. All of the serial interfaces I found rejected invalid XML data to account for this and then retried the request. This is most apparent when first connecting to the device.
The company has an abandoned Emu-Serial-API project on GitHub to parse the XML output with Python 2. Despite the shortcomings of the interface, emu2influx uses it to push metrics to InfluxDB. This is what I use in my home network to visualize energy usage. The most promising 3rd-party interface is Emu-Api (known as emu-power on PyPi), written in Python 3. The RAEdataset project also has a very simple EMU2_reader.py script, if full control isn’t necessary.
Luckily, PG&E supports this device, despite not being listed as a HAN product.
PG&E Share My Data
PG&E allows commercial vendors to request access to their customer’s data with a service called Share My Data. The documentation is centered around the corporate use case, but it does include references to self-service usage under Access Your Own Data. The process is a bit complex and might require setting up a server for any bulk asynchronous data requests to be delivered, but I was able to at least register for the service by completing their testing.
The steps outlined by PG&E might make sense to someone well-versed in web technologies, but I had to navigate it by feel and many web searches. My first obstacle was generating a public/private key pair that PG&E’s endpoint expects for Mutual TLS (mTLS). It seems like this is a common approach for business-to-business communications, but I had never run across it before. I followed whatever online tutorial I could find to generate the key pair, which also required setting up a root certificate authority to sign the certificate. I ended up with key pairs for the server itself and the certificate authority, along with a certificate signing request, and some serial file that was generated during the signing process. The only part that’s needed to communicate with PG&E is the server key pair – the rest are artifacts of the process (which you probably should still keep around somewhere safe).
Authentication
When signing up to access the data, PG&E will also grant you an OAuth client ID and client secret that are used to bootstrap each session using HTTP basic authentication. I wrote my data access service in Go due to the built-in HTTP client and general attitude of doing the simplest thing possible. The first step is being able to authenticate and acquire a bearer token, which can be used in subsequent requests. 2 It should be abundantly clear that I don’t understand the constraints or goals of OAuth. I guess the bearer token allows other servers that handle business logic to operate without access to any means of permanently authenticating themselves.
Right off the bat, I ran into trouble getting the standard library to actually send the client certificate.
To rule out my code, I tried the request another client framework and ended up with the following curl
command:
curl --verbose --request POST \
--cert <path-to-crt> --key <path-to-key> \
--basic --user <client-id>:<client-secret> \
'https://api.pge.com/datacustodian/test/oauth/v2/token?grant_type=client_credentials'
That did work, and when I removed the --cert
and --key
options, I saw the same behavior as my Go program.
All the tutorials on mTLS in Go recommend setting a certificates list in the HTTP client object when it’s created, like this:
http.Client{
Transport: &http.Transport{
TLSClientConfig: &tls.Config{
Certificates: []tls.Certificate{cert},
},
},
}
Where cert
is a certificate created from LoadX509KeyPair
or X509KeyPair
from the crypto/tls
library.
This doesn’t work for self-signed certificates because the Go standard library ensures the certificate being provided is acceptable to the server before sending it.
3 Go’s client implementation calls SupportsCertificate
on the Certificates
list in the TLSClientConfig
, which uses the CertificateRequestInfo
’s AcceptableCAs
to ensure at least one of them is in the signing chain.
It makes total sense, though it would be nice if there was a field that could augment the server-provided list of acceptable CAs.
The correct way to do this when a server doesn’t claim to accept a certificate authority in the signing chain of the client certificate is to use the GetClientCertificate
field of the tls.Config
.
Putting all of the authentication together, but eliding any error handling:
import (
"crypto/tls"
"encoding/json"
"log"
"net/http"
"time"
)
type tokenCredentials struct {
ClientAccessToken string `json:"client_access_token"`
ExpiresIn int `json:"expires_in"`
}
func authenticateWithPGE(cert tls.Certificate,
clientID string, clientSecret string) (
token string, expiration time.Time) {
client, _ := &http.Client{
Transport: &http.Transport{
TLSClientConfig: &tls.Config{
GetClientCertificate: func(
info *tls.CertificateRequestInfo) (
*tls.Certificate, error) {
return &cert, nil
},
},
},
}
req, _ := http.NewRequest(http.MethodPost,
"https://api.pge.com/datacustodian/test/oauth/v2/token?grant_type=client_credentials",
nil)
req.SetBasicAuth(clientID, clientSecret)
now := time.Now()
resp, _ := client.Do(req)
defer resp.Body.Close()
if resp.StatusCode != 200 {
errorBody, _ := ioutil.ReadAll(resp.Body)
log.Fatalf("PG&E returned %d %s response, body: %s",
resp.StatusCode, resp.Status, errorBody)
}
dec := json.NewDecoder(resp.Body)
data := tokenCredentials{}
_ = dec.Decode(&data)
token = data.ClientAccessToken
expiration = now.Add(time.Duration(data.ExpiresIn) * time.Second)
}
Any further requests until the expiration
date just use the same client certificate setup for mTLS and an Authorization
header field with a Bearer <token>
value.
Other wireless devices
PGE only lists a few certified “HAN” products but other utilities have more up-to-date lists, like the one from SDGE.
-
The AzTech in-Home Display has a few more on-device features than the EMU-2 but costs $150 and doesn’t output data over serial.
-
The Rainforest Automation Eagle-200 costs $100 and has a Home Assistant plugin. It uses a network to communicate the data and is based around a cloud service by default. The device phone homes constantly and needs to be blocked by firewall egress rules if the data is only needed locally.
-
Emporia’s Vue Utility Connect Energy Monitor is only $40 but communicates with their cloud service.
-
The Universal Devices ISY series has an incoherent website, but seems to cost $126 for just the module portion of their ecosystem.
I haven’t seriously considered building my own hardware interface to the meter. The ecosystem around the Smart Energy Protocol in Zigbee seems locked down tight, with “production” certificates required to communicate. When setting up the reading device with my utility, they asked for a MAC address that presumably is configured with the meter. It seems like this protocol was built around centralized cryptography to prevent third-parties from snooping on energy usage data.
I was able to find a few resources on Zigbee and Smart Energy, though. It’s unlikely a board based on the Nordic nRF52840, despite it supporting Zigbee, would be able to communicate with the smart meter, since it lacks certification. Texas Instruments has a guide on using one of their modules to monitor Smart Energy packets.
Some smart meters can be read using software-defined radios. The most up-to-date software package I found was rtlamr for Itron ERT meters but I have no experience with it. The Home Assistant Community seem full of people willing to tinker with this sort of thing. The RECESSIM wiki’s page on Advanced Metering Infrastructure also seems useful.
Discontinued
-
Digi, known for their XBee modules, obsoleted their Smart Energy products, like the ConnectPort X2 SE, in 2016. Before that, they wrote a guide on setting up their generic XBee modules for Smart Energy.
-
Until 2015, Rainforest Automation sold a USB stick called the RAVEn that was similar to the EMU-2, but lacked a screen.
-
Google PowerMeter and Microsoft Hohm were two short-lived projects from 2009 to monitor home energy usage using smart meters. Hohm would use non-smart meter data sources at first and then transition to using smart meters as more were brought online.
On the meter
I found a couple of devices that need physical access to the meter to monitor it, but I treated these as last resorts if the wireless options didn’t pan out.
-
Some meters emit pulses as energy is used. The Raspberry Pi-based emonPi monitor can sense these pulses to measure energy.
-
There are a few projects that use OCR to read the visual indicators of meters. The AI-on-the-edge-device (via Hackaday) looks the most promising. There’s also a water meter-focused approach at Integrating my Neptune Water Meter with HomeAssistant (via Hackaday).