Raspberry Pi powered heating controller (Part 1)

January 4th, 2014 — 10:02pm — By Will Cooke

In which no Raspberry Pi’s are seen.

TL;DR:  It should be fairly straight forward to add a Raspberry Pi controlled heating and hot water system to a standard UK domestic set up and, more importantly, remove it again without messing with the existing set up.  As a minimum you’ll need a Raspberry Pi and 4 relays.  A few other bits and bobs wouldn’t go a miss though.  The theory checks out, I’ve ordered the bits, come back next time to see what it looks like.

It occurs to me that – for a long time we’ve had a thermostat in our homes which switches the heating off when it gets warm enough, but wouldn’t it be just as useful to have something which turns the heating on when it gets too cold?

This thought, together with a Raspberry Pi that wasn’t doing much and a strong desire to make my home more connected, led me to think about how I might control my heating system from, say, a smart phone.  I’m far from being the first person to think of this idea, and there are loads of really good examples out there, but none of them did quite what I wanted in the way I wanted to do it.  So I’m going to start from first principals and walk through this project design to try and build a removable & non-destructive add-on to an existing system.  I’m writing this at the very start of the project, so I’ve no idea if it will work, if I will break some expensive components on way to getting it working, or if I will just give up before I get to the end.  Let’s see.

Typical domestic hot water and heating systems

This may be UK specific.  Here is a very very crude diagram of a typical home set up:

A crude diagram of how the central heating system works in a typical UK home.

A crude diagram of how the central heating system works in a typical UK home.

The boiler burns gas and heats water.  That hot water is circulated around the system (called the primary circuit) by a pump and can do three jobs.  It can circulate through a heating element in a hot water cylinder and heat more water which is stored in the cylinder.  Note that the water which circulates through the element does not come in to contact with the actual water it is heating, the two are kept separate for water quality reasons.  The second job it can do is circulate through radiators in the home and hear the air.  The third job is to do both.  The hot water from the boiler moves around the primary circuit losing it’s heat to either the hot water in the cylinder or the air and eventually passes through the boiler again, heats up, and goes round and round again.

There are two “header” tanks of cold water in the loft.  One is for the cold water to the bathroom for flushing the loo, filling the bath, brushing your teeth, that kind of thing.  This tank also fills the hot water cylinder.  The other is the header tank for the primary system and ensures that it can’t boil dry.  Both use gravity and water pressure to make sure the water flows to where it is needed.

The system in the diagram is an “open” system.  If the hot water in the cylinder gets too hot it can expand up the vent pipe and dump itself in to the cold water tank.  The cold water tank can over flow to outside.  If the hot water in the primary gets too hot it can expand up in to the header tank ready to be reused to fill the primary when the water cools.

There is such a thing as a sealed pressurised system which doesn’t have these vents.  These are more complex and if you have one please be very careful in tinkering with the control mechanisms.  In an open system, if you get things wrong and the boiler runs and runs you would end up with a lot of steam in the loft.   In a pressurised system things can go pop and blast you with boiling water.  That said, in an open system you could still end up dumping a header tank full of boiling water down on to the bedrooms below.  People have died from this happening, so tinkering with the heating system is not something to be taken lightly.

In summary then; we have three things we can ask a system for:

  1. Make hot water
  2. Heat the house
  3. Make hot water and heat the house

And we have a number of key elements:

  1. Boiler
  2. Hot water cylinder
  3. Primary header tank
  4. Cold water tank

Typical electrical system to control hot water and heating

Once we understand how the wet bits fit together we can take a look at the electrical components:

Y Plan electrical wiring plan for central heating and hot water.

Y Plan electrical wiring plan for central heating and hot water.

There are multiple “standards” for wiring up a heating system.  You can find heaps of information on the excellent DIY FAQ wiki.

My system has been wired in the “Y Plan” configuration and if you have a single 3-port valve in your airing cupboard and a couple of tanks in your loft – then there is a good chance you have too.  I will run through the wiring, and some of the inherent safety systems built in (which is why I’m keen to make sure my controller is a simple replacement for the existing controller, and is not a complete re-wire).  Before we start though, a further word of caution.  Mains electricity is lethal.  You need be comfortable playing with this stuff to consider attempting anything to do with the heating system.  It’s also probably illegal in UK due to some draconian restrictions on what a home owner can and can not do to the wiring in their own home.  Don’t try this at home kids.  A competent tradesman might be able to help you hook it all together.

The incoming mains supply goes through a double pole switch which will disconnect live and neutral when switched off.  In this diagram, the live feed provides power to only the controller (sometimes you might see a parallel (switched and fused) connection to the boiler from that live).  So first and foremost, all power to the components comes through the controller. Neutral is common to pump, boiler and valve and so is earth.

Thermostats are placed in series for both the hot water circuits and the heating circuits.  These will physically break the circuit when a specific temperature is reached.

Let’s consider this example:  I tell the controller to heat the water.  It connects the live feed to the “HW ON” cable via point 6 on the diagram. The current flows to the cylinder stat, which allows the current through since the temperature is lower than the trigger point it is set to.  The pump and the boiler are connected in parallel so you can’t run the boiler with out the pump running too (at least that’s the plan), and they are provided power via the room stat to point 8 on the diagram.  The boiler is told to turn on, and the pump moves that heated water around the system.  The hot water reaches the three port valve.  The valve has an electrical actuator on which moves to set position depending on what electrical connections are made to it.  In our case, no INPUT power is being applied to the valve, so it sits in it’s default position – which just happens to be “Hot water mode”, and so the heated water from the boiler passes through the hot water cylinder only.  When the hot water cylinder get’s to the right temperature the thermostats clicks over to the other contactor and now no power is applied to the pump and boiler via the HW ON output on the controller.  Instead, the grey wire, point 7 on the connector, is energised.  This tells the valve that HW is no longer required.  This system is pretty safe, since as soon as the cylinder stat is triggered power is removed from the boiler and so it would shut down.  Now, thermostats do fail, but they usually “fail safe”, but sometimes they don’t.

What if I want just the heating to run?  The controller connects to the live input to the CH cable via point 4 on the connector.  This passes through the room stat which will allow the current to flow if it’s below the temperature set.  The current ends up at the valve via point 5 on the connector.  In this case, where we only want heating, the “white” wire is live (it’s black on the diagram) and the valve connects the “white” wire to the “orange” wire which goes back to point 8 on the connector, and in turn provides power to the boiler and pump.  At this point the “grey” wire is also energised, as the controller makes it’s “HW OFF” output live when you ask for only heating. The room stat is able to cut power to the circuit when it reaches the set temperature.

If we want both hot water and heating, the controller energises the “CH ON” and “HW ON” outputs.  Here current is provided to the pump and boiler when any of the thermostats indicates that more heating is required.  If the HW reaches it’s temperature first, then the stat energises the grey wire, which tells the valve that no more hot water is required, and so it will move to the CH ONLY position, and current will continue to be provided by the orange wire when the valve reaches the correct position.  The heated water from the boiler will stop circulating through the hot water cylinder and go only through the radiators – concentrating the heating to where it is needed.  Pretty neat!

This system strikes me as being both simple and brilliant at the same time.  It’s also pretty safe, as long as the stats are working as they should do.

In summary then, the controller is able to indicate a requirement for hot water, central heating, or both by linking three outputs to live in the right sequence.  The four states are therefore:

  1. HW OFF, CH OFF (0,0)
  2. HW OFF, CH ON (0,1)
  3. HW ON, CH OFF (1,0)
  4. HW ON, CH ON (1,1)

Confirming my deductions

I’ve looked at the plumbing, and I’ve looked at the wiring, and I’m pretty sure that I know what’s going on.  Next thing to do is apply the scientific method and gather the evidence to back up my assumptions.

Behind the heating controller

The first thing I did was to turn off the power to the heating system.  I’m paranoid, so I turned it off at the fused connection to the left of the controller and also at the fuse box.  I also wore rubber boots, and jumped in the air every time I touched a wire.  Better safe than sorry, eh?  And, rightly so it turns out.  The fused connection unit did actually cut all the power to the heating system, but look carefully at the third connection from the left and you’ll see an earth wire being used to carry live current.  This is against all the regulations.  Whoever installed this system originally was clearly a free spirit.  I was also quite impressed that they’d managed to squeeze all the connections in to a double gang back box.  What a mess.  Remember kids, only a competent person is allowed to fiddle with these things – they do a better quality job you see.

Looking at the zoomed in image you can see 6 terminals:  N, L 1, 2, 3, 4.

N & L are self explanatory.  2 is not connected to anything, and so I don’t need to worry about it.  So that leaves three connections that do something (1, 3 and 4).  Exactly what I was expecting.  One will be CH ON, one HW ON, and one HW off.  Which is which?

Looking at the back of the controller unit it’s self:

Rear of heating controller

My theory is sound!  1 is HW OFF, 3 is HW ON, 4 is CH ON.

It looks like everything is connected as I had expected, but better safe than sorry.  Let’s do a bit more testing:

Testing harness

I wired in a few bits of cable and then (not shown) removed the connections to the rest of the system (labelling where they came from when I removed them!).  I left the L & N connected.  To recap, I removed the existing wires from 1, 3 and 4 and replaced them with my cables which came down to some screw down connector blocks.  The reason I put connector blocks on the end was two fold.  Firstly, to make it easier to probe with my multimeter and secondly to stop me accidentally brushing against one of the cables and giving myself a shock.  I also labelled the permanent live with a bit of red heat-shrink, just so I don’t get confused.

Test harness 2

Putting the controlled back on, I hooked up my multimeter and switched through the options to see what happens when.  My findings are below:






Exactly what I expected.  Point 1 must, therefore be “HW OFF”, point 2 “HW ON” and point 3 “CH ON” – which they are, as we saw from the back of the controller.  I’m now confident enough with the set up to proceed with roughing out a block diagram for the controller and ordering the parts.

The plan

heating_controller_block  Heating Controller Crude

This rather unclear breadboard layout (with the awesome http://fritzing.org/home/) logically lays out what I intend to do.  I’ve also added a crude block diagram for good measure.

First, I will add a real-time clock module.  They’re cheap and easy to fit.  This will provide the Pi with a source of time when it can’t talk to NTP servers, and so it will be able to turn things on and off at the right times, even when the network connection is down.

Next I will add four relays.  I will take the main 240V incoming supply out of the existing controller and put it through relay 1.  This relay will pass the supply on to the existing controller via the “Normally Closed” relay output.  When I switch this relay, the supply to the existing controller will be dropped, and instead routed to the other three relays which will then be able to switch this current.  These three relays will be wired in parallel with the existing controller connections, much like in the image above showing the test harness connected in to the controller connections.  That is to say: one relay will go to point 1, one to point 3 and one to point 4.  The existing safety features (thermostats in series in the circuits) are un-changed and so still offer the same protection.  In order to activate heating or hot water we switch the relays as per the table above.  By adding my new system in parallel and being able to easily switch between the two I can bring the RasPi powered one online gradually.  A few hours here, a few hours there.  And once I’m happy that it’s not going to go crazy I can leave it unsupervised for longer and longer periods.  It also means that if I update the software and break something, we can still wash.

I will also add a number of 1wire temperature sensors.  I will have three on the hot water cylinder: 1 at each of the top, middle and bottom.  This will give me insight in to how much hot water is in the cylinder, and the temperature thereof.  This is not intended to be a safety system.  The temperature readings from these sensors will not be relied upon to switch things off in an emergency, that will be left to the original thermostats, but – we could use these readings as well to help make decisions.  I will also fit a temperature sensors in the cold water tank, the primary header tank and somewhere outside.  This will give me insight in to a couple of things:  Firstly, how cold is the water in the CW tank, and what is the temperature outside?  This has a direct effect on the number of showers that can be had from a given amount of hot water at a known temperature. Useful for trending too.  Secondly, fitting a sensor in the primary header tank can report when the header tank is getting hot.  Really, the header shouldn’t heat up too much.  If it does then either the system is “pumping over” – where the pump is forcing water up the vent pipe pipe OR the water is so hot it has expanded enough to push water out of the vent, or I expect some combination of the two.  Either situation is sub-optimal, and with a sensor in the header tank I get some visibility of what’s going on.  I might also add a sensor to the boiler input and output, so get an idea of how much work the boiler is doing.  Adding a sensor to each room would be a nice addition at some point too.

A couple of switches will be added for manually switching the hot water or heating on/off from the airing cupboard, where the current controller is situated and where the more senior visitors to Whizzy Towers will expect the heating buttons to be.

Shopping list:

Couple that with a few odd bits of wire, some LEDs a bit of Python and we should have ourselves a Raspberry Pi powered heating and hot water controller which is relatively safe, easy to remove and cheap to build.  Let’s see what happens when all the bits turn up.  Should be here in a week or so.

Stay tuned.


Further Updates

  1. Part 2
  2. Part 3
  3. Part 4


3 comments » | linux, Making the world a better place, RaspberryPi

Absolute beginners guide to Google Maps JavaScript v3

December 17th, 2013 — 2:58pm — By Will Cooke

Since I first published my HowTo and the subsequent follow up for novices to get a Google Map on a web page it’s been the most popular post on my site by quite a margin.  Sadly it’s been resting on its laurels, and is now quite out of date and indeed broken.  So spurred on by my recent server replacement and attempt to revitalise this blog I present the new, improved, and generally working… Beginners Guide To Google Maps JavaScript v3.

The premise is simple; you’ve got a requirement to put a map on your site.  You don’t have the first clue how to do it.  You follow these instructions and you get you map.  Hopefully you’ll pick up enough to tweak the map to your requirements, if not – please ask in the comments and I’ll try and help you.  I’m assuming that you’ve got a bit of HTML experience, and that you’ve got a JavaScript debugger available to you (either Firebug for Firefox or Developer Tools built in to Chrome).

I’ve based a lot of this on the official Google tutorial here:  https://developers.google.com/maps/documentation/javascript/tutorial.  (I’m pretty sure they borrowed that from me in the first place, so it’s only fair 😉 ).

Part 1.  Getting the map on the page

Get yourself an API key for Google Maps from here:  https://developers.google.com/maps/documentation/javascript/tutorial#api_key

Let’s start from the end and work back to the beginning.  Save this as an HTML document, change the API key, open it in your browser and you’ll have a map.  I’ve saved you a bit of time by already centring it on Barrow-in-Furness bus depot.

<!DOCTYPE html>
<title>My first Google Map</title>
    <meta name="viewport" content="initial-scale=1.0, user-scalable=no" />
    <style type="text/css">
      html { height: 100% }
      body { height: 100%; margin: 0; padding: 0 }
      #map-canvas { height: 100% }
    <script type="text/javascript"
    <script type="text/javascript">
      function initialize() {
      	var myLatLng = new google.maps.LatLng(54.124634, -3.237029)
        var mapOptions = {
          center: myLatLng,
          zoom: 17,
          mapTypeId: google.maps.MapTypeId.SATELLITE

        var map = new google.maps.Map(document.getElementById("map-canvas"),
        var myMarker = new google.maps.Marker({
          position: myLatLng,
          map: map,
          title: "A place on Earth",
          draggable: true,

      google.maps.event.addDomListener(window, 'load', initialize);
    <div id="map-canvas"/>

Copy and paste that in to a text editor, replace XXXXXXXXXXXXXXXX with your own API key, save it somewhere and the load it up in your browser.  You should see a satellite style map with a marker in the middle. You’re done.  Simple eh?  Read on learn a bit about how it works, and what you can do to change the appearance.

Part 1.1 Understanding the basic HTML

In order to try and guarantee a consistent layout across browsers you need to make sure that your page is rendered in ‘Standards Compliant’ mode as opposed to ‘Quirks’ mode. To do this you need to specify a DOCTYPE at the top of your HTML file.  We’re using a very simple “html” DOCTYPE which tells the browser that we’re HTML5.  In HTML4.x there were a plethora of variations – thankfully we don’t need to care about them anymore.  HTML5 is the way to go, and so the only DOCTYPE we care about is “html”.

We set the title of the page, and then we set some initial viewport settings to help mobile browsers render the page correctly.  This is widely regarded as a good thing, and you can learn more about it from here: http://webdesign.tutsplus.com/tutorials/htmlcss-tutorials/quick-tip-dont-forget-the-viewport-meta-tag/

Skipping over the style and scripts for a moment, we create the main body of the page with a single div element in it.  We give it the id “map-canvas”, and then we close off the bottom of the body and mark the end of the HTML.  You won’t be surprised to read that the div we’ve just created will be where the map will soon appear.

Part 1.2 All about style

In standard HTML5 (remember the DOCTYPE from above) there are a few specifics you need to know about CSS.  If an element specifies a size as a percentage, then that percentage is calculated from the parent objects size.  Imagine you have a nested DIV, called DIV2.  It lives inside DIV1.  Where DIV1 has a fixed size of 500px by 500px, then DIV2 knows that 100% high equals 500px, but what if DIV1 didn’t have a size specified?  In that case, in standards mode, DIV2 would decide that 100% high equals zero px – because it doesn’t know any better.  This has caught people out a few times.  In order to make sure that all our DIVs can inherit a size correctly we set the height of the entire HTML page and the BODY to be 100%.  This is calculated by the browser when the page loads, and then can flow down to the elements within the page correctly.

Once we have the parent elements size set correctly (the page, and the body) we can style our map DIV to be 100% high safe in the knowledge that it has enough information to render and the correct size and not 0 px high.

Part 1.3 The Meat Section

Now we’re going to look at the actual JavaScript and understand what it’s doing and in which order.

First of all, we have the scripts in the HEAD tag.  The browser will load the head part of the HTML first and your scripts will be loaded before the page is fully rendered.  Any functionality that you make available in your scripts should be available to the rest of the page when it comes to need it.  This is generally the right way to do it.

The first SCRIPT tag takes care of loading the Google Maps code.  We tell the browser that the content of the script is text/javascript and then where to find it.  There are a couple of parameters we pass  in to the script through the URL.  The first one is “key” – this is your simple API key for access Google Maps (see above for details of where to get this key).  The second parameter is “sensor”.  This is required and must be “false” if you’re not using a GPS (or similar) to work out where you are.  In our case, we’re just picking a point on and saying “centre the map here” – so we use false.  If were using a GPS to centre the map on our current location, then this would be “true”.

This script gets loaded by the browser and now we can start to make use of the Google Maps JavaScript APIs in the rest of our page.

The second script is a bit more complex, but should be easy to understand:

We create a new function called “initialize”.
Inside that function we create an object called myLatLng.  We use the Google provided API google.maps.LatLng() to create an object which can be understood by the rest of the Google Maps API and pass in the co-ordinates of Barrow-in-Furness bus depot.  We use “var” to limit that object’s “scope” to within the “initialize” function – that is to say, we won’t be able to get access to that particular myLatLng from other functions on the page.
Next we create another thing called mapOptions.  The format of this is as per the spec here: https://developers.google.com/maps/documentation/javascript/reference?hl=en#MapOptions

There are loads of options, most of which we don’t need to worry about, so we’re just setting a few key options:  where the map is centered, how much it is zoomed in, and the type of map we see.  The map types are provided by Google as a set of constants, which are identifiable by being all in upper case.  In our case we’re using SATELLITE, but we could also use HYBRID, ROADMAP or TERRAIN.

Once we’ve set up the various mapOptions we create a new var called “map” which is an instance of a Map object as provided by the Google APIs.  We pass in to it the id of the HTML object where we want the map to appear, as we created in section 1.2, and we pass in the options var which we just created.  This is enough information for the Google APIs to set up the map as we want it and put it on the page.

The last thing we do in our example is add a marker.  A marker is the indicator which you use to highlight a point on the map.  Google provide a lot of icons and colours for us to use, but the default is the red tear-drop one, so we’ll stick with that for now.

To create a marker we create a new instance of google.Maps.Marker and set up some of the options as we did for the map itself.  We tell it the position for the marker to appear.  We use the “myLatLng” object we created earlier.  You might notice that we are using the myLatLng object twice in our example.  Once as the centre point for the map, and once for the position of the marker.  You can probably deduce from this that the marker will appear in the centre of the map.  We also tell the marker which map it should be added to.  We only have one map on our page, but if we had many this is how you’d add a marker to the correct map.  We give it a title, which is simply a string and we make it draggable by setting the draggable option to “true”.  You can read more about the marker options here: https://developers.google.com/maps/documentation/javascript/reference?hl=en#MarkerOptions

That’s very nearly it for our first simple map.  The last thing to do is use a DOM listener to trigger the above JavaScript when the page loads, and so make our map and marker appear.

google.maps.event.addDomListener is provided via the Google APIs and we pass in three pieces of information.  window is the object provided by the browser.  The ‘load’ event is actually is separate from the “onload” event you might have read about.  The load event signifies that the page is fully rendered and any JavaScript which wants to manipulate the DOM can begin work, and that’s what we want to do.  We want to swap the empty div with the id of “map” with the actual map.  So once the page is indeed loaded the command will execute the “initialize” function and all the magic will happen.

And that’s it.  We’re done.  We’ve got a map on the page centred at our chosen location, and there is a little marker to show a specific part of the map.

If you compare this to the original Beginners Guide I think you’ll say that this new version of the API is even easier to use.  I will try and find time of the next few months to jot down a few notes on doing more interesting things with the map but I hope that this will get you started.

2 comments » | Google Maps, Javascript, Making the world a better place

Combining MythTV and Asterisk

December 15th, 2013 — 10:56am — By Will Cooke

I’ve had this idea for a while and with the discovery of the Google Text-to-speech and Voice Recognition AGI scripts from Zaf (http://zaf.github.io/asterisk-googletts/ & http://zaf.github.io/asterisk-speech-recog/) I’ve implemented a quick proof-of-concept.

You can see the results in this YouTube video:

Over the next few days I’ll tidy up the code and write up a blog post about how to do it.  It’s pretty straight forward though, using APIs provided by Google, MythTV and Asterisk and then just glueing them together.


Comment » | asterisk, linux, tv

Convincing MythTV to tune in to DVB-T2 MUXes

November 25th, 2013 — 7:09pm — By Will Cooke

So that I remember for next time, and so I can start writing down some of the issues I’ve had to sort out since re-building my servers, here is what you have to do to tune to a DVB-T2 mux in MythTV (0.27 fixes).

This is the frequency for the PSB3 mux on Sandy Heath, I expect the encoding scheme will work pretty much where ever. At the very least it should get you started:


Frequency:  474200000
Bandwidth:  8 MHz
Inversion:  Auto
Constellation:  QAM 256
LP Coderate: 2/3
HO Coderate: Auto
Trans. Mode: 8K
Guard Interval: 1/32
Hierarchy:  None

If that doesn’t work, and you know it should (as in you are on the same transmitter) try setting the tuning timeouts higher. Also, just blindly re-trying because “WHY DOESN’T IT WORK” seemed to help.

Comment » | linux, tv

And we’re back…

November 22nd, 2013 — 7:51pm — By Will Cooke

I’ve moved to another server, and in the process I’ve had a whole lot of problems.  For anyone that’s read any of my blog before that won’t come as a surprise.

Once I’ve got all the kinks worked out, I’ll tell you how I fixed it.  For now, this is just a test…

1 comment » | Uncategorized

Asterisk UK Caller ID & SMS redux

February 12th, 2013 — 12:41pm — By Will Cooke

Update 28 Dec 2013:  As far as caller ID goes, I might have been barking up completely the wrong tree here.  Have a look here: http://forums.digium.com/viewtopic.php?f=1&t=85028.  I’ve re-implemented that patch for Dahdi and I’m testing it now.  Get in touch if you want to test it as well.  Once I’m happy it’s actually working, I’ll post it here.


tl;dr: Scroll down to “How to fix Caller ID in the UK on Asterisk 11 and Dahdi 2.6.1″

I’ve been having some problems with Asterisk 1.6 for a while now, in that when I’m dialled in to conferences via SIP, or when I call someone via the work Asterisk server from my Asterisk server at home I get cut off every 15 minutes. My colleagues have all kind of accepted this now, it was starting to get annoying. It was a known problem in Asterisk 1.6, and so I was going to have to upgrade.

The Ubuntu Asterisk packages are fairly up-to-date but FreePBX isn’t packaged, and it’s a pain in the backside to get set up with the correct permissions, especially if you install Asterisk from packages. I had a spare machine anyway so I decided that I’d give FreePBX Distro a go and once I’d burnt the ISO to a CD (ya rly) I found the set up process to be simple and quick.

FreePBX Distro is a stripped down CentOS (2.6.32 kernel) which boots quick, has minimal footprint and bundles in loads and loads of FreePBX “apps” like conferences, voicemail, blacklisting and makes setting up extensions a breeze. It also includes things like kernel source for I sell settlement so building extra modules from source is pretty easy. Which is a good job, because it was about to get messy.

If you’ve found this page via Google then this will probably sound pretty familiar:

  • You got a TDM400p four port FXO/FXS interface card, or a clone from someone like OpenVox (which work perfectly by the way – as do those cheap daughter cards off eBay).
  • You’ve upgraded to Asterisk 11, or Asterisk 1.8 perhaps, or changed to the Dahdi 2.6.1 drivers and all of a sudden your ”UK Caller ID” has “stopped working” or at the very least become “intermittent“.
  • It was working fine before, so it’s not a hardware issue and your phone provided have NOT suddenly stopped sending caller ID information, regardless of what people of forums tell you. No, in fact this is clearly a bug which has crept in somewhere along the line.
  • You’ve searched and searched and tried things like setting rxgain=x.x and txgain=x.x or whatever. (Side note: setting txgain and rxgain in chan_dahdi_channels_custom.conf has no effect, the only place is seems to work is in chan_dahdi_groups.conf)
  • You might have even stumbled across a post hinting that cid_rxgain=x.x will solve your problems, only it didn’t

If that sounds like you, dear reader, then read on. I am your salvation.

  1. This page: http://downloads.openvox.cn/pub/drivers/callerid_patches/ (I can only attribute this discovery to divine intervention)
  2. Find a new wctdm.c driver here: http://downloads.openvox.cn/pub/drivers/callerid_patches/2.6.1-wctdm.c
  3. Download the 2.6.1 Dahdi source from here: http://downloads.asterisk.org/pub/telephony/dahdi-linux-complete/dahdi-linux-complete-2.6.1+2.6.1.tar.gz
  4. Replace dahdi-linux-complete-2.6.1+2.6.1/linux/drivers/dahdi/wctdm.c with the version downloaded from openvox
  5. Note it doesn’t compile, add the missing semicolon to line 333 (or thereabouts), successfully build and install the new driver
  6. You might need to change /etc/modprobe.d/dahdi.conf and add:
options wctdm opermode=UK fwringdetect=1 battthresh=4
  1. restart Dahdi and Asterisk and caller ID should now be fixed! Yay!

How to fix Caller ID in the UK on Asterisk 11 and Dahdi 2.6.1

Here is a patch against wctdm.c from stock Dahdi 2.6.1 with the fixes in: wctdm.c.patch

OR: Replace dahdi-linux-complete-2.6.1+2.6.1/linux/drivers/dahdi/wctdm.c with this one: wctdm.c

OR: If you’re running FreePBX distro 64 bit “BETA-3.211.63-5 Release Date-01-24-13″ (or close) here is a binary driver: wctdm.ko  Move it to /lib/modules/2.6.32-279.11.1.el6.x86_64/dahdi/wctdm.ko

It works for me. I have not had to increase the rxgain, or txgain or anything else. It just works.

Once that’s working, we can move on to…

How to send & receive SMS in the UK on FreePBX Distro Beta 3.211

I’ve talked about receiving and sending SMSes before, but that was quite a long time ago, and things have moved on a bit since then. I’ve also learnt a bit more about the Asterisk dial plan, and have a slightly cleaner way of doing it now.

You need to have working caller ID. Hint: see above.
You need to install smsq.
smsq doesn’t come built in FreePBX Distro so you have to build it yourself if you want to send SMS. This is pretty easy:

  1. Download the Asterisk 11.2.1 source from here: http://downloads.asterisk.org/pub/telephony/asterisk/asterisk-11.2.1.tar.gz
  2. Install popt: yum install popt-devel.i686 popt.i686 popt.x86_64 popt-devel.x86_64 (prerequisite for smsq)
  3. In the Asterisk source directory do “./configure”
  4. run “make menuselect”
  5. Scroll down to “Utilities”
  6. In the right hand pane, check “smsq”
  7. Save & Exit
  8. run “make”
  9. DO NOT run “make install” as this will trash your FreePBX config
  10. Once that’s done you should find an executable smsq in the utils directory

Just to be sure, create /var/spool/asterisk/sms/motx and /var/spool/asterisk/sms/mtrx
You can find more details on building Asterisk here: http://blogs.digium.com/2012/11/05/how-to-install-asterisk-11-on-centos-6/

You need to be able to send SMS in order to register with the BT SMS system, otherwise when you receive an SMS you will get a phone call from the BT robots who will read the text to you, badly.

To register with BT try this:

<path to smsq>/smsq –motx-channel=DAHDI/2/17094009 -d 00000 -m “test”

where DAHDI/2 is the channel number of your outgoing line.

If you look in the Asterisk logs you should see the message going out with some TX and RX hex dumps. If it works, a short while later you will receive a phone call from either the SMS system trying to talk modem at you, or the robot. Either way, this means your text went out. Great success!

Now we need to set up receive, which is pretty straight forward.

Edit /etc/asterisk/extensions_custom.conf and add the below. Note – I’ve added a lot of comments to explain what’s going on, it might improve readability if you take them out.

;; This makes sure our new 'app' gets included in the FreePBX dial plan
include => app-rx-sms
;; This is the app to receive an SMS which will be called when we match
;; the caller ID of an incoming call to the BT computer
exten => s,1,Answer()
;; This wait seems to help
exten => s,n,Wait(0.5)
;; We simply use the built in SMS function
exten => s,n,SMS(default,a)
;; Hangup with code 16, normal termination.
exten => s,n,Hangup(16)
;; Done
;; We also extend the from-pstn context with a branch to the SMS app
;; if the caller ID matches, hence why it's necessary to get caller
;; id working first
;; we add ourselves in to the end of the current dialplan
;; priority 7 gets us there...
exten => s,7,GotoIf($["${CALLERID(num)}" = "08005875290"]?app-rx-sms,s,1)
;; pretty simple - if the caller ID matches, go to the sms app
exten => s,n(dest-ext),Goto(ext-group,1100,1)
;; Note: 1100 is my "ring all" group. You will need to change 1100
;; to what ever you use. You should see this at the end of the
;; from-pstn context in extensions_additional.conf

Save that, and in asterisk do a “dialplan reload”. From smsq send an SMS to 00000 saying “register”· If everything works you should get a text file in /var/spool/asterisk/sms/mtrx.

Comment » | asterisk

Device control over HDMI via CEC. libcec FTW.

November 26th, 2012 — 3:28pm — By Will Cooke

Blimey, it’s been a while.  I’ve been a bit busy, and let’s be honest; writing up blog posts always sounds like a good idea, but when you get in to it – it’s really hard work.

Anyway – I finally got round to buying a Pulse-Eight CEC to USB adapter:  http://www.pulse-eight.com/store/products/104-usb-hdmi-cec-adapter.aspx

This awesome little box of tricks makes up for the lack of CEC control in the vast majority of HDMI-Out equipped graphics cards.  It’s the final piece in the jigsaw of a Linux based home entertainment device.  It allows you to talk to the other devices in your HDMI network; your surround sound amplifier and your big screen TV being the best examples (assuming they support CEC of course).

CEC has been around for a long time but for some reason it doesn’t seem to be widely used, or very well implemented, in most consumer electronics devices.  It’s a published standard but with OEM manufacturers wanting to differentiate their products and introduce a bit of vendor lock-in, they all call it something different.  The fact is that your Toshiba Regza Link TV will talk to your Sony Bravia Link Amp just fine, for the most part.  There might be a couple of proprietary things which don’t work, but on, off, volume up, volume down etc will all just work.

Pulse Eight’s USB to CEC adapter lets your computer get in on the act too, and opens up a whole realm of automatic switching, which really cuts down on the number of remote controls you need and the number of buttons you have to remember the purpose of.

The good folk at Pulse Eight have also made libCEC, an open source library to allow pretty much any software to take advantage of the USB adapter.  http://libcec.pulse-eight.com/

It comes with C++, C and .NET interfaces, and a CLI utility called cec-client.  XBMC & MythTV already support libCEC and have some neat features baked right in.  It’s good, but it’s not exactly what I was looking for – I want a bit more control.

Now, I don’t know anything about C++ or C or .NET, so until someone writes some Python bindings, my ticket to this party lies solely with the CLI utility cec-client.  It can do most things on the “transmit” side, so you can send commands to your other CEC devices fairly easily.  Acting on a request, or listening to the CEC traffic is a bit more complex – but not beyond the realms of possibility.

This weekend I wrote (and rewrote and rewrote) a couple of Bash scripts to:

  • Let me control the system volume (i.e. the real hardware, not the mixer on the computer) on the TV & Amp from the PCs remote control
  • Let me switch the TV & Amp on and off
  • Activate proper muting, again not the mixer on the computer – the hardware itself
  • Switch the amp & TV to my MythTV PC
  • Power off all the hardware when the PC suspends, and then switching it all back on again
  • Shut the whole lot down when the screensaver on the PC kicks in.

I make these scripts available for your amusement:

cecsimple is a client of the “server” which itself is a client of cec-client.  Fire up the server and then issue it commands down the FIFO either directly or via the abstraction layer which is cecsimple.sh.


I hooked up the volume control and amp power via “irexec” and lirc.  I tell irexec to execute, for example, “cecsimple.sh volup” or “cecsimple.sh ampon”.  If the server component is already running then these commands are sent very quickly and you don’t really notice the lag.

To switch the TV off when the PC goes in to suspend mode I added a script in /etc/pm/sleep.d which calls “cecsimple.sh tvoff” and then “cecsimple.sh tvon” when it resumes.  In theory if the TV is using the Amp to output surround sound audio then the TV will tell the Amp to turn off, it it’s not – it wont.

To switch things off when the screensaver kicks in, I simply “sudo pm-suspend” from an “xscreensaver-command -watch” script.


The practical upshot is that I can now control 99% of my media centre from a single remote control.  I’ve opted to use the remote connected to the PC as I found it to be the least laggy – using the TV remote to send up/down/left/right etc to the PC was sluggish.


I think it should be possible to parse the log output from cec-client and write a “listener” component too, but it’s probably a better idea to learn some rudimentary C and do it properly.  Or some Python bindings.  Oh yeah, and you know what would be really cool, a hook in to MythTV so that when I’m watching something in surround sound the amp turns on automatically. That would be cool.


UPDATE 3 Dec 2013:  When someone leaves the amp’s HDMI switch set on the PS3 and you switch the MythTV box on from the remote, the amp doesn’t automatically switch to MythTV.  This has been annoying me for a while now, so I fixed it.

In the scripts linked to above the “active source” command does this:

send_command "tx 45 82 11 00"

4 (the MythTV device) to 5 (the amp) – 82 (switch active source) to – but my Sony amp just ignores this request.

I spent some time trying out a few alternatives with cec-client and I’ve found one which works, and it kinda makes sense why:

send_command "tx 45 70 11 00"

4 (MythTV) to 5 (Sony amp) 70 (System Audio Mode) (the input where MythTV is connected)

My assumption is that amp only speaks “system audio” – what with it being an amp.  I’ve changed the “activesrc” with the 45:70:11:00 code and now it works!  (It also switches the amp on, whether I like it or not – so it’s not perfect).



Useful links:


Comment » | linux, Making the world a better place, tv

Fixing “warning: Please check that your locale settings”

August 16th, 2012 — 5:01pm — By Will Cooke

I took an Amazon AWS t1.micro instance for a spin the other day. A free server is not to be sniffed at. Of course I installed Ubuntu 12.04 on it.

I was getting a lot of locale errors, things like this:

perl: warning: Setting locale failed.
 perl: warning: Please check that your locale settings:
 LANGUAGE = (unset),
 LC_ALL = (unset),
 LC_COLLATE = "en_GB.UTF-8",
 LC_CTYPE = "en_GB.UTF-8",
 LANG = "en_US.UTF-8"
 are supported and installed on your system.

I thought this would just go away by itself, but it didn’t – so I had to fix it. Note: I’m in the UK, so I’m using en_GB as my locale, change yours to en_US or whatever.


export LANGUAGE=en_GB.UTF-8
sudo locale-gen en_GB.UTF-8
sudo dpkg-reconfigure locales

And you should be all set.

Comment » | Uncategorized

Monetizing my feeds

August 4th, 2012 — 3:32pm — By Will Cooke

I decided I wanted to add adverts to my RSS feed. Hardly anyone subscribes to this blog, a dozen people or so (Hello! I bet I know exactly who you all are), but I do get a few hundred hits a month on my Beginners Guide To Google Maps tutorial. So adding adverts to the feed shouldn’t really hurt anyone, and as a bonus the bandwidth used serving the RSS feed should reduce.

AdSense lets you easily monetize a feed by sucking in an RSS feed, parsing it and adding the advert code and then spitting it back out again via FeedBurner.  You can do this by logging in to your AdSense account, click “My Ads” and then expand the “Feeds” section on the left. Once you’ve filled in the boxes, it’ll provide you with a new URL for your feed complete with adverts.

All this is pretty easy, but then comes the question of how you get people to see the new monetized feed?

First things first, FeedBurner can generate a little bit of HTML for you to add to your WordPress Sidebar. It’s described here http://support.google.com/feedburner/answer/78487/?hl=en& and for the sake of completeness also described here:

  1. Log in to FeedBurner using the same credentials as you log in to AdSense
  2. Click on Publicize from the menus near the top of the page
  3. Click on Chicklet Chooser
  4. Choose the type of button you’d like to appear on you blog. Personally I think the smaller “Subscribe in a reader” is the best choice
  5. Then the Javascript at the button of the page is automatically updated, so copy it to the clipboard (ctrl-c)
  6. Go to the Admin page of your WordPress blog
  7. Choose Appearance
  8. Choose Widgets
  9. Drag “Text: Arbitrary text for HTML” to your sidebar
  10. Drop down the Sidebar widget and paste the text from above in to the big box
  11. Click Save and click close
  12. You’re done!

That was the easy bit. Now new subscribers to your blog feed can simply click the button and will be taken to the new monetized feed.

But what about existing subscribers? Or 3rd party links which point direct at your “/feed/” URL? There’s the rub.

What you need to do is redirect them to the new URL, but then if you put a global redirect from the old URL to the new URL, how is FeedBurner going to keep itself updated? (It’ll get redirected back to itself and so never see any of your new posts)

The trick is to use mod_rewrite and check for the HTTP_USER_AGENT of “FeedBurner”.

I’m going to assume that you’ve already got mod_rewrite enabled in your Apache config. If you’re running WordPress on your own server then you most likely have it enabled without even realising.

In your blog’s web directory you will find a “.htaccess” file, open this up and you’ll probably see something like this:

# BEGIN WordPress
 RewriteEngine On
 RewriteBase /
 RewriteRule ^index.php$ - [L]
 RewriteCond %{REQUEST_FILENAME} !-f
 RewriteCond %{REQUEST_FILENAME} !-d
 RewriteRule . /index.php [L]

What you need to do is add these lines between “RewriteBase /” and “RewriteRule ^index.php$ – [L]

RewriteCond %{REQUEST_URI} ^/feed/* [NC]
 RewriteCond %{HTTP_USER_AGENT} !FeedBurner
 RewriteRule ^feed/?.*$ http://feeds.feedburner.com/Whizzyorg [R,L]
 #Next rule

How does it work?

Mod_rewrite matches all the conditions in order, when it hits a Rule it decides if all the preceding conditions are met, and if so acts on the rule. If not, if moves on to the next line. The rules are the the separators between the conditions.

So the new logic says:

  • IF you’re trying to get to “/feed/” (the [NC] makes it case insensitive)
  • AND your USER_AGENT is NOT “FeedBurner” (the ! inverts the logic)
  • THEN re-direct people to the new URL

So, if you are FeedBurner you get access to the old standard WordPress RSS feed and keep yourself updated, if you’re anyone else you get redirected and see the adverts.

The adverts are fairly inconspicuous and are easily blocked by those in the know, so I don’t have a problem with it.


2 comments » | Making the world a better place

Over engineering FTW

March 8th, 2012 — 8:26pm — By Will Cooke

Working from home has many advantages.  No commuting, relaxed approach to being dressed, loud music.

One of the drawbacks however, is that you are entirely reliant on your broadband connection.  You could, as many do, have a 3G modem/phone as a backup.  Fine for a few hours, but you’re soon going to run in to data caps, invisible proxies and traffic shapers (more so than a fixed line telco).


So what to do?  My solution has been to get a second line installed.  The big UK telcos are running a lot of offers at the moment to try and secure as many customers as they can in readiness for their Triple & Quadruple Play strategies.  This means you can get a reasonable ADSL line and line rental for about 15 to 20 quid a month.

When you’re as reliant on a connection as I am, that’s money well spent.  It also gives you a perfect opportunity to play with some new toys!

Now, you could simply plug in another wifi router to the new line, and when the primary connection fails move your connections over to the secondary one.  But where’s the fun in that?  What you really want to do is some load-balancing and multi-path routing over both the lines.  You won’t see a sudden doubling in download speeds in normal browsing, but certain activities which open a lot of connections to different IP address should benefit, and you get to play with routing tables.

Here’s what it looks like:


We don’t really get much diversity until we get south of the exchange, and even then we’re probably going to live in the same fibre all the way to The Internet, but we do work round the much more common problem of ISP configuration errors.

Here’s how it works:

Router 1 (R1) and Router 2 (R2) are two independent ADSL modem/routers  connected to a phone line each and the ethernet port connected to a NIC in a server.  These modem/routers also take care of NAT, so that any traffic coming from inside your network will be translated to appear as if it had come from the appropriate router on it’s way to the rest of the internet.  The server has three NICs in total, one each for the two routers and one back to the LAN for everyone else on the network to use as a gateway.  Let’s use the standard naming convention from http://lartc.org/howto/lartc.rpdb.multiple-links.html

$IF0 = eth0
$IF1 = eth1
$IF2 = eth2

The IP addresses associated with those interfaces are:

$IP0 =
$IP1 =
$IP2 =

The routers are configured with these LAN IPs:

$P1 =
$P2 =

As you can see on the diagram, the networks are all /24 ( which gives us network numbers of:

$P0_NET =
$P1_NET =
$P2_NET =

Then we configure the Ubuntu box as a router & NATing gateway:

1.  Enable IP forwarding by editing /etc/sysctl.conf and uncommenting or adding a line like this:


2.  Create two new routing tables.  Edit /etc/iproute2/rt_tables and add a line for each of your providers.  For example:

1 my_isp_name
2 my_other_isp_name

3. Create some routes in each of the routing tables so the server knows where to send data.  They take the general format of:

ip route add $P1_NET dev $IF1 src $IP1 table my_isp_name
ip route add default via $P1 table my_isp_name
ip route add $P2_NET dev $IF2 src $IP2 table my_other_isp_name
ip route add default via $P2 table my_other_isp_name


Using the real numbers and the template above gives us this actual set of commands:

ip route add dev eth1 src table my_isp_name
ip route add default via via eth1 table my_isp_name
ip route add dev eth2 src table my_other_isp_name
ip route add default via via eth1 table my_other_isp_name


In english, that means:  In order to send traffic to the network or the network send it out of eth1 or eth2 from IP address or  These routes are then stored in the relevent routing table; either one called my_isp_name or one called my_other_isp_name.


4. Create some additional routes for communicating between the routing tables and the local interfaces.  This is as much of an efficiency saving as anything, telling each routing table the quickest way to the other interfaces.

ip route add $P0_NET dev $IF0 table my_isp_name
ip route add $P2_NET dev $IF2 table my_isp_name
ip route add dev lo table my_isp_name
ip route add $P0_NET dev $IF0 table my_other_isp_name
ip route add $P1_NET dev $IF1 table my_other_isp_name
ip route add dev lo table my_other_isp_name

5.  Add the routes to the main routing table, so that traffic knows how to find the networks which have the gateways to the internet on:

ip route add $P1_NET dev $IF1 src $IP1
ip route add $P2_NET dev $IF2 src $IP2

6.  Apply the correct routing rules depending on where the traffic is coming from:

ip rule add from $IP1 table my_isp_name
ip rule add from $IP2 table my_other_isp_name

7.  The science bit.  Create a route which will send traffic down each of the internet connections in turn.  This is a global default route, so any traffic which isn’t bound for the local $P0_NET network will use this rule to get to the internet.

ip route add default scope global nexthop via $P1 dev $IF1 weight 2 nexthop via $P2 dev $IF2 weight 1

As you can see, this isn’t a 50/50 split.  We’ve applied some weighting, in this case because I get more GBs with my_isp_name.  We’re actually going to send twice as much traffic down $IF1 as we are down $IF2.

8.  By default, routes to other networks are cached for 10 mins.  This means that once a route is decided on, traffic will always follow the same route for at least the next 10 minutes.  The counter will be reset every time that particular route is used.  This is great for large downloads because it means that your IP address isn’t suddenly going to change half way through getting the file.  But, for something like IRC where it’s possible that no traffic will be sent for 10 minutes, then the IRC server could well see your IP address changing as traffic flips between the two routes.  IRC servers don’t like that.  So you need to fix some routes so that traffic will always follow the same path.

ip route add via $P2

This sets the route to a specific Freenode IRC server to always go via my_other_isp_name.

9.  Last of all, we need to add an extra layer of NAT so that traffic can find its way back to the host on the LAN side.  This is necessary because our internet modem/routers are not really routers at all.  They simply take traffic from one side, rewrite the headers and push it out the other side, while maintaining a look up table of what came from where and doing the reverse when a reply arrives. If you can configure the routes on to your modem, then you don’t need to be reading this in the first place, so let’s assume you can’t.

iptables --table nat --append POSTROUTING --out-interface $IF1 -j MASQUERADE
iptables --table nat --append POSTROUTING --out-interface $IF2 -j MASQUERADE
iptables --append FORWARD --in-interface $IF0 -j ACCEPT

For things like port forwarding you do now have to deal with a bit of an odd NAT situation.  For example, let’s assume I want to run a web server on the same machine that runs the routing.  I decide to only make the server available on the WAN IP address of $R1.  I set up Router 1 to NAT port 80 on the WAN to port 80 on $IP1.  Then I need to configure the web server on the routing server to listen on that interface.  Not too bad really.

And there you have it.  Two internet connections, both available to all the clients on your network.  Make sure that your clients use the IP address of the server as their default gateway and everything should just work.  The additional load on the server is negligible.

If you reboot the server you’ll need to add all the routing information again, so create yourself a Bash and have it run from /etc/rc.local or something.


1 comment » | linux, Making the world a better place

Back to top