Double helping of Pi Hole

In episode 100 of Late Night Linux I talked a little bit about trying out Pi Hole and AdGuard to replace my home grown ad blocker based on dnsmasq and a massive hosts file.

I came down in favour of Pi Hole for a couple of reasons but the deciding factor was that Pi Hole felt a bit more open and that it was built on top of dnsmasq which allowed me to reuse config for TFTP which netboots some devices which needed it.

Now that I’ve been using Pi Hole for a few months I have a much better understanding of its limitations and the big one for me is performance. Not the performance when servicing DNS requests but performance when querying the stats data, when reloading block lists and when enabling and disabling certain lists. I suspect a lot of the problems I was having is down to flaky SD cards.

I fully expect that for most people this will never be a problem, but for me it was an itch I wanted to scratch, so here’s what I did:

Through the actually quite generous Amazon Alexa AWS Credits promotion I have free money to spend on AWS services, so I spun up a t2.micro EC2 instance (1 vCPU, 1GB RAM – approx £10 a month) running Ubuntu.

I installed Pi Hole on that instance along with Wireguard which connects it back to my local network at home. I used this guide from Linode to get Wireguard set up.

The Pi Hole running in AWS hosts the large block files and is configured with a normal upstream DNS server as its upstream (I’m using Cloudflare).

Pi Hole running in AWS configured with Cloudflare as its upstream DNS

I use three Ad block lists:

Pi Hole running on a t2.micro instance is really speedy. I can reload the block list in a matter of seconds (versus minutes on the Pi) and querying the stats database no longer locks up and crashes Pi Hole’s management engine FTL.

The Pi Hole running on my LAN is configured to use the above AWS based Pi Hole as its upstream DNS server and also has a couple of additional block lists for YouTube and TikTok.

This allows me use Pi Hole on a Pi as the DHCP server on my LAN and benefit from the GUI to configure things. I can quickly and easily block YouTube when the kids have done enough and won’t listen to reason and the heavy lifting of bulk ad blocking is done on an AWS EC2 instance. The Pi on the LAN will cache a good amount of DNS and so everything whizzes along quickly.

Pi Hole on the LAN has a block list of about 3600 hosts, whereas the version running in AWS has over 1.5 million.

All things considered I’m really happy with Pi Hole and the split-load set up I have now makes it even easier to live with. I would like to see an improved Pi Hole API for enabling and disabling specific Ad lists so that I can make it easier to automate (e.g. unblock YouTube for two hours on a Saturday morning). I think that will come in time. The split-load set up also allows for easy fallback should the AWS machine need maintenance – it would be nice to have a “DNS server of last resort” in Pi Hole to make that automatic. Perhaps it already does, I should investigate.

Why not just run Pi Hole on a more powerful computer in the first place? That would be too easy.

If you fancy trying out Pi Hole in the cloud or just playing with Wireguard you can get $100 free credit with Linode with the Late Night Linux referral code: https://linode.com/latenightlinux

DNS over HTTPS in a snap

Background Story

With the recent news about the ISP UK association proposing Mozilla as “Internet villain of the year” for enabling DNS over HTTPS (and subsequently changing their mind and dropping the whole category of villain of the year. Good move I think.) I figured it was probably about time that I looked at enabling DoH at home.

Cloudflare have a suite of open source tools called cloudflared which has, among other things, a DNS over HTTPS proxy. By default it points at their 1.1.1.1 service, but you can change that if you want to. Note, at the time of writing there is a bug which seems to stop Google’s DNS service working. If you’re looking to stop people seeing your DNS traffic then Google probably isn’t the right DNS service to use anyway.

I already have dnsmasq running as my DNS server and I have quite a lot of config which I wanted to keep (e.g. adblocking) so I figured I would add cloudflared’s proxy-dns alongside dnsmasq and have dnsmasq use proxy-dns as it’s upstream server, which would in turn pass the DNS lookups to 1.1.1.1 over HTTPS. dnsmasq would then cache the results locally.

So far, so good. I’d built cloudflared on my desktop to test it, now I wanted to move it on to the Raspberry Pi, run it as a service, and ideally have a package so that I didn’t have to mess around rebuilding it in loads of places if I wanted to move to a different box.

Make a snap

Making a snap of proxy-dns would give the the package I wanted, and could allow me to run proxy-dns as a daemon with two words in the YAML. Snapcraft’s build service would build me an ARM binary, as well as loads of others, for free.

I downloaded the source for cloudflared and added three files:

  1. A snapcraft.yaml which describes how to build cloudflared and sets it to be run as a daemon
  2. A configure hook which lets me set some config options
  3. A launcher script which sets the config at run time

None of these are very complicated, as you can see. Hat-tip to Popey for help with the snapcraft.yaml.

The I pushed these back to my project on GitHub and added that project to the Snapcraft.io build service. Now, whenever I push a new change back to GitHub the snap will get rebuilt automatically and uploaded to the store! All I would need to do is a snap refresh and I’d be upgraded to the latest version. All my requirements solved in one place.

How to use the snap

If your Pi is running snapd, it’s dead easy (e.g. Ubuntu MATE or Ubuntu Core):

sudo snap install cloudflaredoh --edge

The snap is currently in the edge channel, meaning it’s not ready for the main stage just yet. Once I’ve spent a bit more time on it, I will move it to stable.

sudo snap set cloudflaredoh address=127.0.0.1
sudo snap set cloudflaredoh port=5053

Configure proxy-dns to listen on 127.0.0.1. If you want it to answer DNS queries from other computers on your network try either the IP address of the box, or just 0.0.0.0 to listen on all interfaces. It will also configure proxy-dns to listen on port 5053. If you want it to answer DNS queries from other computers on your network, use the default DNS port of 53.

sudo snap get cloudflaredoh

This will show you the currently set config options.

sudo snap restart cloudflaredoh

Restart proxy-dns and use the new config.

Now you can use something like nslookup to query the DNS server and make sure it’s doing what you expected.

10 Steps To DNS-over-HTTPS

  1. Get a Raspberry Pi
  2. Download Ubuntu Core and write it to an SD card
  3. Put the SD card in your Pi and boot it
  4. Set up the network on Ubuntu Core (tip: register for an Ubuntu One account first)
  5. sudo snap install cloudflaredoh
  6. sudo snap set cloudflaredoh address=0.0.0.0
  7. sudo snap set cloudflaredoh port=53
  8. sudo snap restart cloudflaredoh
  9. Configure your client’s DNS server as the IP address of your Pi
  10. Have a cup of tea

Update 2019-08-01

I’ve got a new Github repo set up with an improved snapcraft.yaml which pulls directly from the upstream project. I’m aiming to get this hooked up to the Snapcraft build service so that we can package the latest version automatically. More on this later. In the meantime, you can clone this and build the latest version yourself:

https://github.com/8none1/cloudflarednsproxy

Whizzy Labs Wireless Sensor

Sheesh! Making a wireless sensor has proven to be a lot harder than I had expected.

Like a lot of weekend hardware hackers I thought it would be fun to build a wireless temperature sensor. I could use it as a feedback mechanism for my Raspberry Pi heating controller, I could create graphs of temperatures in different rooms (everyone loves a graph right?), and with a little bit of forward planning I could make a fairly useful Arduino breakout board which could be used for lots of other fun wireless projects.

I’ve been through four iterations of the design before I finally found something that worked. I could have avoided some of this if I’d have spent more time testing on breadboard, but then I wouldn’t get to order colourful PCBs!

The Brief

Cheap

In common with a lot of my other weekend projects, this is very likely to be a white elephant, so it needs to be cheap to start with.  There are lots of excellent wireless sensors already available, the Moteino is well regarded.  Using that as a baseline, could I build something similar for less than £5 per unit?  The answer is “nearly”, see the Bill Of Materials below.

Use a ready made microcontroller board

I didn’t want to have to design a board which I would then have to solder an ATmega328/P to myself. Laying out that board would be hard, plus I would have to spec and solder all the supporting components. I could have done that with through-hole components I suppose, but then the board would have been massive, and I very much doubt it would have actually worked out any cheaper.  Instead, I decided that a better idea was to solder on a ready-made Arduino clone, specifically the Pro Mini. I buy them from this supplier on eBay and I’ve never had any problems. They are 3v / 5v switchable, very reliable and turn up quickly.  Power consumption was going to be an important metric and running them off of a 3V supply would be essential.

Use a ready made radio module

As with the MCU, I don’t want to be laying out a radio board or soldering surface mount components.  I needed a ready-made radio module which was cheap, readily available and had low power requirements.  I started off with the NRF24L01.  It had a small footprint, cost around 99p a unit, had a built in antenna and was supported by the RadioHead library.  Initial testing on breadboard showed that the range of these devices would be sufficient for my house.  I designed the first two revisions of the sensor board around this radio (and a CR2032, see the battery section below).  Initial current draw was a bit too high for the CR2032, but I could have worked around that by using AAA batteries instead – but more annoying was that the range of the radios was not at all reliable.  The 2.4GHz spectrum is very noisy and the addition of two baby monitors since the original breadboard test hasn’t helped, but in testing on the PCB I found these radios to be pretty much useless.  I also tried using the version with the power amp as the central transceiver, this helped a little bit but they were still failing to get at least 50% of their transmissions to the other end, even over a couple of meters.  Other people have reported really good range with these boards, but crucially those range tests were done outside.  It’s my considered opinion that these radios and Wifi in the house do not co-exist.

So I gave up with those, and tried the XL4432-SMT.  This is based on the Si443x chipset from Silicon Labs.  They’re readily available on eBay for around £.170 each, so nearly twice the price of the NRF24L01.  They’re well supported by the RadioHead library, can run down to 1.8V, have low current draw when on (virtually nothing when in stand-by), support a wide frequency range around the 433MHz ISM band and in range testing they out-performed the NRF24L01 by a long way.  The downside is that these pre-made boards use 1.27mm pitch / castellated connections.  I had to design an Eagle part to interface with them, but that wasn’t really too hard.  See below for links to the parts I made.  Another drawback was the antenna; being a much lower frequency means they need a much longer antenna so I would need to find a project box which could hold them.
The Si443x also has a temperature sensor and a wake up timer on board.  However, reading the errata from Silicon Labs it seems that the WUT is actually broken and the temperature sensor was returning very strange results.  The datasheet says that you need to calibrate the temperature sensor so I tried doing this but go nowhere fast, and so I opted to use a 1wire sensor instead.

The other option for a radio would have been an ESP8266.  You can get ready made boards cheap on eBay, and I could have done away with a separate MCU altogether but the power consumption of these devices is just too great for a project which needs to run of a couple of batteries for a year.

Run for a long time on batteries

What’s the point in a wireless sensor if you have to plug it in to the mains.  This project must run from batteries, and those batteries need to last a long time – having to change the batteries every few months would quickly get boring and the sensors would be left doing nothing.  Obviously having the MCU go in to a sleep state between runs is going to be necessary, plus a radio which has modest power requirements when running.  We can further reduce the power needs by making sure that the “work” that the sensor has to do is done as quickly as possible.  The default 1wire temperature sensor code you find will typically takes around 2700msec to read.  That’s a very long time.  I changed the code a bit by hard-coding the hardware address of the sensor on the board and by only reading two bytes of temperature data (good for 0.5 degree accuracy).  More information can be found below.

From a size perspective a CR2032 battery looks very appealing.  Some early testing made me think that they would work fine but in real life I had a lot of problems.  In hindsight I think I can put most of the problems down the Brown Out Detector on the Pro Mini being set to 2.8V, more on that in a moment.

UPDATE:  Whoops.  This post has been sat in drafts for nearly 2 years.  Guess it’s not getting finished then.  I’m posting this in case the above is interesting.  Topics that I wanted to cover but haven’t are:

  • Fit neatly in a box
  • Read the temperature
  • Be extensible
  • Reading 1wire temperature sensors quickly
  • Lower the BOD to 1.8V
  • Bill Of Materials