Hacking 433Mhz support into a cheap Carbon Monoxide detector

Skill level:  Easy

My home automation systems use two mechanisms for communication:  Ethernet (both wired and wireless) and 433MHz OOK radio.

433MHz transmitters are readily available and are cheap but unreliable.  Wifi enabled MCUs such as the ESP8266 are also cheap (coming in at around the same cost as an Arduino clone, a 433MHz transmitter and a bag of bits to connect them together), they are reliable enough but extremely power hungry.  If I can plug a project into the mains then I’ll use an ESP8266 and a mobile phone charger for power, if the project needs to run off batteries then a 433MHz equipped Arduino is the way I’ve gone.

Like most people playing with 433MHz radio I found reliability and range of the radio link to be super flaky.  I’ve finally got a more-or-less reliable set-up:

  • A full wave dipole antenna at the receiver
  • A high quality receiver from RF Solutions in place of the cheap ones which are bundled with transmitters. A decent receiver on eBay
  • A big capacitor on the transmitter.  I saw the frequency and amplitude drifting massively during transmission.  Adding a 470µF cap helps.  Allow time for the cap to charge and the oscillator to stabilise, a few seconds delay seemed to do the trick.
  • Using the RCSwitch library on the transmitter:
    • RCSwitch mySwitch = RCSwitch();
    • mySwitch.setProtocol(2); // Much longer pulse lengths = much better range?
    • mySwitch.setRepeatTransmit(20); // Just brute-force it!

With this setup I can get receive a 24bit number from an Arduino running off 2 AA batteries and a coiled 1/2 wave antenna from about 5 meters indoors through walls.  That’s still poor, but it does the job.  Increasing the voltage to the transmitter would probably help.

Once you have a reliable 433MHz receiver setup then you can also buy off the shelf 433MHz enabled home automation gizmos like this smoke alarm or these door sensors.  They have a set of jumpers inside where you can set an ID, which is essentially the same 24bit number that RCSwitch lets you transmit.  For what it’s worth I also have kite-marked smoke detectors in my house, but from the testing I’ve done with a bit of smoldering paper the cheap imports work just fine.

I couldn’t find a cheap Carbon Monoxide which also has 433MHz support so I thought I’d quickly hack one together out of this Carbon Monoxide detector and an Arduino clone and 433MHz radio:

CO Alarm inside

 

 

 

 

 

 

 

 

 

IMG_1238

You can barely notice it!

 

 

 

 

 

 

 

It’s certainly untidy, but it does the job.  If I had PCB facilities at home I’m fairly sure it could be made to fit inside the alarm, along with some more holes in the case for ventilation.

The premise is simple enough.  The Arduino is powered by the 3v3 regulator on the CO alarm PCB.  The cathode of the red alarm LED is connected to pin 2 of the Arduino as an external interrupt.  When the pin goes low the Arduino wakes up and sends it’s 24bit ID number over the radio which is picked up by the receiver which sends an SMS alert, switches the boiler off, etc.  I’ve connected the radio transmitter to directly to the 3 x AA batteries (4.5 volts) via a transistor which is switched by a pin on the Arduino.  In standy-by mode the additional equipment draws a fraction of a milliamp and so I’m not worried about draining the batteries faster.

As with the smoke alarms, this is not my only source of Carbon Monoxide detection.  I’ve yet to test it’s sensitivity.  This is considered to be a “well, if it works, and it turns the boiler off automatically then it’s certainly worth a go, but I’m not relying on it” project.

Snapping Mosquitto MQTT broker

As part of my ever expanding home automation system I wanted to use MQTT to publish data on my network. With the release of the Raspberry Pi 2 I can run Ubuntu Core to create a reliable, secure and easily updated server which is a perfect fit for requirements of an MQTT broker and general HA controller. I asked some Ubuntu friends to help me package Mosquitto as a Snap, and in return I would write down how we did it. Here’s the story…

Start by reading this: https://developer.ubuntu.com/en/snappy/

In summary; a Snappy application is secure because it’s wrapped with AppArmor. It’s easier to install and upgrade because everything is packaged in a single file and installed to a single location. That location is backed-up before you install a new version, and so if the installation goes wrong you can revert to the previous version easily by copying the original files back (or rather, Snappy will do all of that for you). Simplifying things slightly there are two types of Snappy “application”: Apps and Frameworks. Frameworks can extend the OS and provide a mediation layer to access shared resources. Apps are your more traditional top-level items which can use the provided frameworks, or bundle everything they need in to their Snap. This makes things much easier for app providers because they are now in charge – they can be assured that no library will change underneath them.  This is a huge benefit!

Let’s get Mosquitto snapped.

1. Install QEMU to run an Ubuntu Core machine

http://www.ubuntu.com/cloud/tools/snappy#snappy-local

First we install the KVM hypervisor:

sudo apt-get install qemu-kvm

Then check everything is as it should be with:

kvm-ok

Now download the latest Ubuntu Core image from here: http://cdimage.ubuntu.com/ubuntu-core/preview/  At the time of writing this is the newest x86-64 image: http://cdimage.ubuntu.com/ubuntu-core/preview/ubuntu-core-alpha-02_amd64-virt.img

Then launch the virtual machine. This command port forwards 8022 on your local machine to 22 on the virtual machine, so you can SSH to port 8022 on localhost and actually connect to the Ubuntu Core machine. It gives the Core machine 512MB of RAM, nicely achievable on a modest budget (The Pi2 has 1 GB).  We also forward port 1883 from to the VM, which will allow us to connect to the Mosquitto server on our VM once it’s all installed.

kvm -m 512 -redir :8022::22 -redir :1883::1883 <vm image file>

Once it’s booted you can connect to it with SSH. The username and password are “ubuntu”.

ssh -p 8022 ubuntu@localhost

To make things a bit easier, why not use key authentication? On your host machine:

ssh-copy-id -p 8022 ubuntu@localhost

We should also upgrade our Ubuntu Core VM before we start.  SSH in to your box and run:

sudo snappy update
sudo reboot

2. Build Mosquitto in the right way

Back on your host (not the virtual machine you just created above) create some directories to hold the code and download the latest stable source and the build dependencies for Mosquitto:

sudo apt-get install build-essential cmake
sudo apt-get build-dep mosquitto
mkdir -p mosquitto/install mosquitto/build
cd mosquitto
wget http://mosquitto.org/files/source/mosquitto-1.3.5.tar.gz
tar xvzf mosquitto-1.3.5.tar.gz
cd build

Time to build Mosquitto.  Before you run the commands below, a bit of background information.  The cmake line will force cmake to install the binaries to the location specified with INSTALL_PREFIX, rather than /usr/local.  This is required to bundle all of the binaries and other files to the “install” directory we created above, making it possible to package as a Snappy.

cmake -DCMAKE_INSTALL_PREFIX=`readlink -f ../install/` ../mosquitto-1.3.5
make -j`nproc`

make install

nproc spits out the number of processor cores you have, so the make line above will use as many processor cores as you have available.  It’s not required, and for Mosquitto which is fairly small it’s not worth worrying about, but for a bigger job this is quite handy.

If you look in the “../install” directory you’ll see a familiar structure containing all the goodies needed by Mosquitto.

3. Find the libraries needed and copy them in to your Snappy project

Change in to the install/lib directory and use ldd to display the linked libraries for the two main .so files:

ldd lib/libmosquitto.so.1.3.5 lib/libmosquittopp.so.1.3.5 | grep '=>' | awk '{ print $1 }' | sort | uniq

This uses ldd to show the libraries required by Mosquitto, and then sorts them in to a nice list. You’ll see something like this:

libcares.so.2
libcrypto.so.1.0.0
libc.so.6
libdl.so.2
libgcc_s.so.1
libmosquitto.so.1
libm.so.6
libpthread.so.0
librt.so.1
libssl.so.1.0.0
libstdc++.so.6
linux-vdso.so.1

Now, on the Ubuntu Core machine we can run this little script:

for i in `cat`; do find /lib /usr/lib -name $i; done

Copy the list from the previous command to the clipboard and then paste it in to terminal where this command is running and hit Ctrl-D to submit the list.  The script will then search Ubuntu Core for the libraries required.  If it finds them they will be displayed, if it doesn’t then they are not available in Ubuntu Core by default and will need to be included in your Snappy package.

linux-vdso is the Linux kernel and is available on every Linux system by default, so we don’t need to provide that specifically.

libssl, libcrypto, libpthread, librt, libc and libdl are all available in Ubuntu Core by default – so we don’t need those either.

That leaves just libcares to be copied in to our package.

 cp /usr/lib/x86_64-linux-gnu/libcares.so.2.1.0 .

We should already be in the ‘lib’ directory, hence the ‘.’ above.  We are copying libcares in to the lib directory of our Snap, and when we run the Snap we will pass in the library path to make sure Mosquitto can find it.  More on this later.

4. Add the meta data required for the Snappy package

Reference: https://developer.ubuntu.com/en/snappy/guides/packaging-format-apps/

Create the meta data directory inside the install directory (change to the install directory, it should just be cd ..):

mkdir meta

Create the package.yaml file:

nano meta/package.yaml

And this is what we’re putting in it:

name: mosquitto.willcooke
architecture: amd64
version: 1.3.5
icon:
services:
 - name: mosquitto
 start: ./sbin/mosquitto.sh
ports:
 required: 1883

Information about these fields and what they mean is available in the reference linked to above, but they are easily understandable.  A comment on the name though, you need to append .<yournamespace> where your namespace is as you select in your Ubuntu myapps account.  One thing to mention, you can see that to start our Snap we are calling a shell script.  This allows us to pass in extra options to Mosquitto when it runs.

Next we need to create a readme file:

nano meta/readme.md

This file needs to contain at least a couple of non-blank lines.  Here’s what we put in it:

This is a Snappy package for Mosquitto MQTT broker.
Information about Mosquitto is available here:  http://mosquitto.org/
Information about MQTT is available here: http://mqtt.org/

We also need to configure our Mosquitto server, by editing the conf file.  Most of the settings can be left as default, so we will create a new conf file with only the bits in we need.

mv etc/mosquitto/mosquitto.conf etc/mosquitto/mosquitto.conf.ori
nano etc/mosquitto/mosquitto.conf

Add these two lines:

user root
persistence_location /var/apps/mosquitto/current/

We need to change this to run as root.  Since our Snap will be confined there is no risk here.  I expect the ability to run as non-root users when using Snappy will be improved, but really it’s not necessary.

We also need to add a small shell script to start Mosquitto with the right options.  Create a file in install/sbin called mosquitto.sh:

nano sbin/mosquitto.sh

And add this:

#!/bin/sh
LD_LIBRARY_PATH=./lib:$LD_LIBRARY_PATH exec ./sbin/mosquitto -c etc/mosquitto/mosquitto.conf

We are specifying where to find the extra libraries we require and where to find the conf file.  Make that file executable:

chmod +x sbin/mosquitto.sh

5. Build your Snappy package

Add the Snappy PPA to get the build tools, and then install them:

sudo add-apt-repository ppa:snappy-dev/beta
sudo apt-get update
sudo apt-get dist-upgrade
sudo apt install snappy-tools

In your install directory run:

snappy build .

If you see an error about ImportError: No module named ‘click.repository’ then you likely have a clash between the Click library version in the SDK team PPA and the version in the Snappy PPA.  This will be fixed soon, but in the meantime I would suggest installing ppa-purge via apt-get and then running sudo ppa-purge ppa:ubuntu-sdk-team/ppa.

If you see an error about “expected <block end>” in the package.yaml check the whitespace in the file.  It’s likely a copy and paste error.

6. Install your Snappy package

Once you have your .snap file you can install it to your virtual machine like this:

snappy-remote --url=ssh://localhost:8022 install ./mosquitto_1.3.5_amd64.snap

 

7. Test your Snappy package

If everything has gone to plan Mosquitto should now be running on your virtual machine.  In order to test you’ll need to write a test Publisher and Subscriber.  I used the Python Paho library.

Here’s an example Publisher:

#!/usr/bin/python
import paho.mqtt.client as mqtt
from datetime import datetime
from time import sleep
def send_mqtt(topic, message):
 log("Sending MQTT")
 log("Topic: "+topic)
 log("Message: "+message)
 mqttc.reconnect()
 mqttc.publish(topic, message)
 mqttc.loop() 
 mqttc.disconnect()
print "Time server starting up...."
mqttc = mqtt.Client("python_pub")
mqttc.connect("localhost", 1883)
while True:
 tstr = datetime.now().strftime("%Y-%m-%d %H:%M:%S")
 send_mqtt("/information/time",tstr)
 sleep(10)

And here’s an example Subscriber:

#!/usr/bin/python
import paho.mqtt.client as mqtt
import datetime
def on_connect(client, userdata, rc):
 print "Connected with result code "+str(rc)
 client.subscribe("#")
 
def on_message(client, userdata, msg):
 print "Topic: ", msg.topic+'\nMessage: '+str(msg.payload)
 
 
client = mqtt.Client()
client.on_connect = on_connect
client.on_message = on_message
client.connect("localhost", 1883, 60)
client.loop_forever()

 

What’s next?

We’ve built a Snappy package for amd64 (or whatever your native architecture is), but we really need to be cross-architecture to give people the best choice of platform on which to use the package.  This involves cross compiling, which can be tricky to put it mildly.

I spoke to Alexander Sack, the Director of Ubuntu Core, and asked what was coming next for Snappy and I was very excited to hear about easier cross-compilation methods as well as a cool script to help automate gathering the libraries in to your package.  I’ll find out more about these and follow up with another post about

Special Thanks

A huge “Thank You!” to Saviq and Didrocks for doing the actual work and letting me watch.

 

Where to get Snappy Mosquitto

amd64 version:  https://myapps.developer.ubuntu.com/dev/click-apps/ubuntu/1500/
armhf version: https://myapps.developer.ubuntu.com/dev/click-apps/ubuntu/1502/ (please note, I haven’t been able to test the ARM version because of a lack of hardware.  If it doesn’t work let me know and I can fix it.)

Recording screencasts from the Unity 8 Desktop Preview

Obtaining and running Unity 8 Desktop Preview

If you like playing with new toys you might have already downloaded and tried the Unity 8 Desktop Preview (available here:  http://cdimage.ubuntu.com/ubuntu-desktop-next/daily-live/current/)

If you haven’t, you should take it for a spin.  If you have an Intel graphics everything should be fine and dandy, if not YMMV at the moment.

  1. Download the ISO
  2. Find a spare >1GB USB thumb drive
  3. Run “disks” via the dash
  4. Highlight your USB drive and from the cog icon on the right choose “Restore Disk Image…”
  5. Select your ISO and “Start Restoring”  – this will of course erase everything else on your USB stick
  6. Done

You can now boot from your USB stick and have a play with Unity 8.  Right now you’ll be seeing the Phone view of Unity 8, but that will all be changing in time.

Capturing a screencast

Once you’ve got everything up and running you might like to make a few screencasts, so how do you do it?  Well, the Mir developers have provided us with the mirscreencast tool so let’s use that:

Switch to tty1 (ctrl+alt+f1) and log in and run:

mirscreencast --file <output file> -m /run/lightdm-mir-0

Then switch back to tty8 (ctrl-alt-f8) and use Unity.

Your file will now be filling up FAST.  Mirscreencast will be trying to write every raw frame to that file, probably at a rate of 60 frames a second.  To kill mirscreencast I first hit ctrl-z and then:

 pkill -9 mirscreencast

but there is probably a better way.

Capturing a better screencast

There are a few command line options for mirscreencast which will can help us shrink the file size a bit:

mirscreencast --file <output file> -m /run/lightdm-mir-0 -s 683 384 -n 3600

The option “-s” will resize the captured frames.  Note that 683 384 is exactly half my native resolution, so you will need to adjust this to your display.

The option “-n” will capture n frames and then stop.  At 60 frames a second, 3600 frames is one minute.  If you use -n then mirscreencast will exit gracefully at the end.

Playing back your screencast

I am lucky enough to have a spare machine with a touch screen just for running Unity 8 on (http://www.dell.com/uk/dfh/p/inspiron-11-3137/pd) so I SCP the raw video file on to my main machine for playback and editing.

I use mplayer for most of my video playback and encoding needs and it will happily play the raw video file, but it needs a few pointers:

mplayer -demuxer rawvideo -rawvideo fps=60:w=683:h=384:format=bgra <filename>

or, to convert the raw file into something which you can edit in OpenShot try this:

mencoder -demuxer rawvideo -rawvideo fps=60:w=683:h=384:format=bgra -ovc x264 -o <output filename> <filename>

And then you can edit and upload the processed file.  When I export from OpenShot I use the “Web” profile, then target “YouTube-HD”, “HD 720p 29.97 fps” “Med” – it’s a bit overly compressed, but it looks OK.