Hey Thermostat

Alright then, that works.


I promise I’m never blogging about my thermostat ever again.

Not a Nest

Raspberry Pi Wireless Thermostat

As I wrote before, I’ve been hacking around with my wireless thermostat. Over the weekend I spent a couple of hours throwing together some code to act as a thermostat control loop, running from a Raspberry Pi. I’ve shared the code on GitHub.

There’s two bits: a little C executable using Wiring Pi, that sends the commands to the boiler, and a Go executable that watches the temperature from an attached digital thermometer (a DS18B20) and triggers the thing that sends the commands. (The whole thing was Go for a while, but Go is too unreliable for timing the transmission, so it seemed easiest to move that bit to Wiring Pi in C.)

To put all this on a schedule, I’m using cron to write the desired temperature into a file that’s being read every few seconds. If the monitored temperature goes 0.5°C over the desired temperature, the boiler turns off. And if it goes 0.5°C under, it turns on. It’s a simple little system, but it’s been running for a couple of days now, and hasn’t set the house on fire yet.

The thermostat is now marginally less convenient than it was when I started, so the next step is to do what I promised with a shared calendar (not everyone in the family wants to SSH in to turn the heating on).

And it’s not done until you’ve put it in a nice box and it doesn’t look like an improvised explosive device. I’ve always enjoyed watching Tom put things in boxes, so it might be time to get some advice.

For a stretch goal, I’d like to poke around with HomeKit integration: “Hey Siri, turn the heating on”, etc. And actually, thinking about it, that’s something you can’t do with a Nest.

Reverse engineering a wireless thermostat

We moved the boiler in our house a few months ago, and replaced the wired, rotary thermostat in the process, with a 7 day digital wireless one. Or rather, the builders did, and I didn’t really pay attention.

The timers are pretty simple – a morning schedule, an early evening schedule and a late evening schedule – but we’re often in and out at different times, and the UI is so terrible and the buttons so horrible to press, it’s preferable to just sit in the cold and shiver.

Anyway, I wanted to see if I could control the heating from something smarter. I’ve not quite done that yet, but I’ve managed to reverse engineer the signalling, and I thought it might be useful to write down how I did it.

We’ve got a Tower RFWRT thermostat – it’s pretty cheap. There’s two parts: the heating control unit, connected to the boiler, and a wireless thermostat you can place anywhere within range. These communicate over 433MHz, an unlicensed band which is pretty common for household devices.

The thermostat sends one of two command: turn on or turn off. It sends the current command on a 30s cycle, and when the state is changed. If the on command isn’t received for a certain period (the thermostat has gone out of range, run out of battery, etc.) the heating control unit will switch the boiler off as a safety measure.

If we can find out what these signals are we can send them ourselves, and eventually replace the wireless thermostat with something smarter. (Or, y’know, you can just buy a Nest or something. You can stop reading here.)

There’s a few ways to receive 433MHz signals, but the easiest for me was to use an RTL-SDR USB digital TV receiver. These are about £10, and let you listen across a wide range of frequencies (not just TV/radio), up to about 1700MHz.

I used CubicSDR, a Mac SDR application, and pressed the temperature controls up and down watching for the on/off commands. Sure enough, I spotted a short blip whenever the desired temperature is set.


Using Audio Hijack (Soundflower works too), I recorded both commands to an audio file, trimmed the silence, and put the two segments side by side to compare.


It looks like the signal is repeated twice, probably to reduce the likelihood of interference.

You can see a preamble at the beginning – a long high and long low, four times – probably designed to let the receiver calibrate its automatic gain control. These receivers crank up the volume (and the noise) during silence, and then adjust it when they hear a signal. The preamble gives the gain control a chance to respond so the beginning of the data isn’t lost in the noise.

After that we’re into the command data. I spent a long time staring at this, initially thinking it was Manchester encoded, before realising I needed to switch Audacity into ‘Waveform (dB)’ mode to see what’s really going on. This is actually pulse width modulation, with a 0 sent as a short high and a long low, and a 1 sent as a long high and a short low.


I measured each short section as 250μs long, and each long section as 500μs, and each preamble section as 1000μs.

The off command is just a run of 25 zeros, while the on command sends a few 1s too. It doesn’t really matter what it means — I just need to send the same thing.

I took a cheap 433MHz transmitter and soldered a coiled 17cm antenna on. I attached this to the digital out of an Ardiuno, and wrote a little script to perform the transmission.

#define transmitPin 4 // Digital Pin 4

// on
bool data[] = {
  0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
  1, 0, 1, 0, 1, 1, 1, 1, 0, 0

// off
//bool data[] = {
//  0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
//  0, 0, 0, 0, 0, 0, 0, 0, 0, 0

int shortDelay = 250;
int longDelay = 500;
int preambleDelay = 1000;

void setup() {
  pinMode(transmitPin, OUTPUT);

void loop() {

void sendPreamble() {
  for (int i = 0; i < 4; i++) {
    digitalWrite(transmitPin, HIGH);
    digitalWrite(transmitPin, LOW);

void sendData() {
  for (int i = 0; i < sizeof(data); i++) {
    bool b = data[i];
    if (b == 1) {
      digitalWrite(transmitPin, HIGH);
      digitalWrite(transmitPin, LOW);
    } else {
      digitalWrite(transmitPin, HIGH);
      digitalWrite(transmitPin, LOW);

A little click, the boiler goes on, and we’re in business. To start with, I’d like to schedule the timing from a shared family calendar, so now I need to make a thermostat control loop, and drive the whole thing from something web connected, like a Raspberry Pi.

CityCyclist 1.0


tldr; I made an app for bike navigation, you can download it here.

For a few months, in slivers of spare time, I’ve been working on a little app for city bike navigation, called CityCyclist.

I’ve tried to build something clean and accessible, that gets a good bike route on the screen as quickly as possible. That’s glanceable while on a bike, and more useful when off.


There’s a little scrubber on the elevation profile at the bottom to fly quickly along a route without zooming and panning around. My hypothesis was that might make it easier to consign a route to memory. I suspect that’s not true, but I still like it.

The search results use a combination of Foursquare and Apple’s address geocoder, and seem fairly good.

The routing is powered by CycleStreets (backed by OpenStreetMap) with a selection of three options: fast, balanced, quiet. (UK only for now.)


I think version 1.0 is a good base for future experiments. I’d like to try a take on turn-by-turn navigation that works when the phone is in your pocket or bag, and I’d like to find a feedback loop to improve routing/map data. But this’ll do for now.

It’s in the App Store now and it’s free. If you give it a try, I’d love to know what you think. Get it here.

(PS: If you work for the kind of organisation that would benefit from me supporting, improving, and getting this app to as many people as possible, and want to help out, do get in touch.)

What Do People Do All Day?

Everyone is a Worker

One of Sam’s favourite books at bedtime is Richard Scarry’s ‘What Do People Do All Day?’. I was going to write a post all about it, because it’s kind of brilliant (explaining economic and industrial processes for the under fives! lovely drawings!) and kind of terrible (gender roles! a society seen solely through the eyes of capitalism!).

So I started writing, but then I found this great post deconstructing it better than I could:

Which rather begs my usual question: so why is it that we no longer make these kinds of books? Why is it that we have shifted our focus to how things work or how people used to work as opposed to how people work now? Is it that work is too elusive, that new economy jobs are harder to draw? Can we not deal with the fact that Alfalfa has become a derivatives trader? But work of course is far from invisible. It’s not just that we do so many of the occupations lovingly drawn by Scarry, and in more or less the same way. It’s also that people still work in manufacturing, only mostly elsewhere. We could teach our children about that, just like we teach them that everybody poops.

(Obviously, someone has done a ‘funny’ 21st century version, but that doesn’t count.)

So I read it, and I looked at the news, and it became pretty clear to me while I’d like to see more of this sort of book, the kids will be just fine. But we could do with a bit more Richard Scarry, and a bit more How Things Work, and a bit more Secret Life of Machines for everyone else.

New year, new job!

Me at Desk

This is my last fortnight at Newspaper Club, after a brilliant five and a half years. We’ve built a great little company that’s growing fast, but it’s time for me to move on to something new. I’m pleased to be leaving it in very capable hands, and I can’t wait to see all the 2015 plans come to fruition. I’m going to miss all my colleagues there, but a special mention to Anne, who has worked tirelessly to turn our silly idea into, well, a non-silly idea. One that employs people, pays taxes and has happy customers all over the world.

This has been in the works for a little while, and I’d planned on going freelance again, until the next meaty long-term thing came along. So it was a surprise to find that thing before I’d even got started.

I’m joining Moo, as Head of Technology of a new multidisciplinary team working on scalable digital products. We’re starting from scratch, with the Moo engine behind us, and a blank whiteboard in front of us. Exciting! (Also, in an odd, but nice twist, we’ll be occupying the old RIG/BERG/MakieLab space on Scrutton St! Come visit!)

Happy 2015 everyone!

Custom EC2 hostnames and DNS entries

I’ve been doing some work with EC2 recently. I wanted to be able to bring up a server using Ansible, pre-configured with a hostname and valid, working FQDN.

There’s a few complexities to this. Unless you’re using Elastic IP addresses, EC2 instances will change public IP address on reboot, so I needed to ensure that the DNS entry of the FQDN will update if the host changes.

A common way of doing this is to bake out an AMI, pre-configured with a script that runs on boot to talk to the DNS server and create/update the entry. But you still need a way of passing the desired hostname in when you launch the instance for the first time, and you end up with your security keys baked onto a AMI, making it difficult to rotate them. And custom AMIs are fiddly – I’d prefer to use the official ones from Ubuntu or Amazon so I don’t have to bake out a new AMI on every OS point release.

I ended up with an approach that uses a combination of cloud-init, an IAM instance role, and Route 53, to set the hostname, and write a boot script to grab temporary credentials and set the DNS entry.

EC2 supports a thing called IAM Instance Roles, allowing an EC2 instance to grab temporary credentials for a role, letting it access AWS resources without hardcoding the access tokens. It does this by fetching credentials from an internal HTTP server, but if you use awscli or other official libraries, they’ll do this for you, unless you provide credentials explicitly.

In this case, we grant just enough permission to be able to update a specific zone on Route 53. I chose to put all my server DNS entries in their own zone to isolate them, but you don’t have to do that. I made a role called ‘set-internal-dns’ and gave it a policy document like this:


Next, I wrote an Ansible task to boot a machine set to that role, with a user-data string containing cloud-init config.

- name: Launch instance
    keypair: "{{ keypair }}"
    region: "{{ region }}"
    zone: "{{ az }}"
    image: "{{ image }}"
    instance_type: "m3.medium"
    vpc_subnet_id: "{{ vpc_subnet_id }}"
    assign_public_ip: true
    group: ['ssh_external']
    exact_count: 1
      Name: "{{ item.hostname }}"
      Name: "{{ item.hostname }}"
      role: "{{ item.role }}"
      environment: "{{ item.environment }}"
      - device_name: /dev/sda1
        volume_size: 30
        device_type: gp2
        delete_on_termination: true
    wait: true
    instance_profile_name: set-internal-dns
    user_data: "{{ lookup('template', 'templates/user_data_route53_dns.yml.j2') }}"
    - hostname: "computer1", 
      fqdn: "computer1.{{ domain }}"
      role: "computation"
      environment: "production"

Ansible expects the user_data property as a string, so we load a template as a string using lookup.

cloud-init has the lowest documentation quality to software usefulness ratio I think I’ve ever seen. In combination with EC2 (and presumably other cloud services?), it allows you to pass in configuration settings, packages to install, files to upload and much more, all through a handy YAML file. But all the useful documentation about the supported settings is completely hidden away or just placeholder text, except for a huge example config.

Our user_data_route53_dns.yml.j2 template file is below. If you’re not using Ansible, the bits in the curly brackets are templated variables being set by the task above.


# Set the hostname and FQDN
hostname: "{{ item.hostname }}"
fqdn: "{{ item.fqdn }}"
# Set our hostname in /etc/hosts too
manage_etc_hosts: true

# Our script below depends on this:
  - awscli

# Write a script that executes on every boot and sets a DNS entry pointing to
# this instance. This requires the instance having an appropriate IAM role set,
# so it has permission to perform the changes to Route53.
  - content: |
      FQDN=`hostname -f`
      ZONE_ID="{{ zone_id }}"
      PUBLIC_DNS=$(curl ${SELF_META_URL}/public-hostname 2>/dev/null)

      cat << EOT > /tmp/aws_r53_batch.json
        "Comment": "Assign AWS Public DNS as a CNAME of hostname",
        "Changes": [
            "Action": "UPSERT",
            "ResourceRecordSet": {
              "Name": "${FQDN}.",
              "Type": "CNAME",
              "TTL": ${TTL},
              "ResourceRecords": [
                  "Value": "${PUBLIC_DNS}"

      aws route53 change-resource-record-sets --hosted-zone-id ${ZONE_ID} --change-batch file:///tmp/aws_r53_batch.json
      rm -f /tmp/aws_r53_batch.json
    path: /var/lib/cloud/scripts/per-boot/set_route53_dns.sh
    permissions: 0755

We’re installing our script into cloud-init’s per-boot scripts rather than anywhere else because I know cloud-init will run it on first boot, after it has been installed. If we put it in rc.d, for example, we’d still have to tell cloud-init to go and run it on first boot, so this is just one less thing to mess up. I’m already feeling pretty bad about writing JSON in a shell script in a YAML file.

When you boot the instance you should be able to tail /var/log/cloud-init-output.log and see a confirmation from the awscli script that the DNS change is pending. It can take 10-60 seconds to become available.

We’re using a CNAME to the EC2 public DNS entry because I still want to use split horizon DNS – if you look that entry up from inside your EC2/VPC network you’ll get the internal IP address.


Surprises, small and big

OK, look, I know think pieces about Apple belong on Medium, but just watch video of the iPhone launch in 2007. The bit where Steve demos the ‘slide to unlock’ (15:14), and there’s an audible intake of breath from the audience. And then again, later, with the ‘pinch to zoom’ (33:22).

(It’s also a really funny presentation! I forgot how light hearted they used to be.)

The first iPhone astounded me, because it felt like something from 2-3 years in future. At the time, multitouch screens were reserved for table top projection surfaces (Microsoft’s original Surface project, for example), and while it seemed clear that it was going to be an important form of interaction in the future, every other device on the market had a physical keyboard or a stylus.

I didn’t even know multitouch technology was capable of being shrunk to that size, for that price. Apple had managed to not only make it happen, but had orchestrated their supply chain into producing millions of them, leaving the rest of the phone industry dumbfounded. And due to their exclusivity contracts, it would take years before they truly caught up. Incredible.

Satellite View of GCHQ Bude

It’s summer 2013, and the Snowden leaks are in full swing. All the geeks will tell you that they expected it all along, but they’re lying. No-one expected it to be that fierce, that pervasive, that explicit.

The culmination of it, for me, was when we learnt that GCHQ intercepts and stores all transatlantic network communications through their intercept station in Bude, Cornwall.

The story isn’t clear at first. All of it? That can’t possibly be right. But yes, that turns out to be 21 petabytes a day, over a rolling 3 day buffer. Around 60+ petabytes at any one time.

Storing it I can fathom – just about. You need lots of space, lots of disks, lots of money, and I guess they get a volume discount. But searching it in real-time too? Huh.