Fixing “warning: Please check that your locale settings”

August 16th, 2012 — 5:01pm — By Will Cooke

I took an Amazon AWS t1.micro instance for a spin the other day. A free server is not to be sniffed at. Of course I installed Ubuntu 12.04 on it.

I was getting a lot of locale errors, things like this:

perl: warning: Setting locale failed.
 perl: warning: Please check that your locale settings:
 LANGUAGE = (unset),
 LC_ALL = (unset),
 LC_MESSAGES = "en_GB.UTF-8",
 LC_COLLATE = "en_GB.UTF-8",
 LC_CTYPE = "en_GB.UTF-8",
 LANG = "en_US.UTF-8"
 are supported and installed on your system.

I thought this would just go away by itself, but it didn’t – so I had to fix it. Note: I’m in the UK, so I’m using en_GB as my locale, change yours to en_US or whatever.

Type:

export LANGUAGE=en_GB.UTF-8
sudo locale-gen en_GB.UTF-8
sudo dpkg-reconfigure locales

And you should be all set.

Comment » | Uncategorized

Monetizing my feeds

August 4th, 2012 — 3:32pm — By Will Cooke

I decided I wanted to add adverts to my RSS feed. Hardly anyone subscribes to this blog, a dozen people or so (Hello! I bet I know exactly who you all are), but I do get a few hundred hits a month on my Beginners Guide To Google Maps tutorial. So adding adverts to the feed shouldn’t really hurt anyone, and as a bonus the bandwidth used serving the RSS feed should reduce.

AdSense lets you easily monetize a feed by sucking in an RSS feed, parsing it and adding the advert code and then spitting it back out again via FeedBurner.  You can do this by logging in to your AdSense account, click “My Ads” and then expand the “Feeds” section on the left. Once you’ve filled in the boxes, it’ll provide you with a new URL for your feed complete with adverts.

All this is pretty easy, but then comes the question of how you get people to see the new monetized feed?

First things first, FeedBurner can generate a little bit of HTML for you to add to your WordPress Sidebar. It’s described here http://support.google.com/feedburner/answer/78487/?hl=en& and for the sake of completeness also described here:

  1. Log in to FeedBurner using the same credentials as you log in to AdSense
  2. Click on Publicize from the menus near the top of the page
  3. Click on Chicklet Chooser
  4. Choose the type of button you’d like to appear on you blog. Personally I think the smaller “Subscribe in a reader” is the best choice
  5. Then the Javascript at the button of the page is automatically updated, so copy it to the clipboard (ctrl-c)
  6. Go to the Admin page of your WordPress blog
  7. Choose Appearance
  8. Choose Widgets
  9. Drag “Text: Arbitrary text for HTML” to your sidebar
  10. Drop down the Sidebar widget and paste the text from above in to the big box
  11. Click Save and click close
  12. You’re done!

That was the easy bit. Now new subscribers to your blog feed can simply click the button and will be taken to the new monetized feed.

But what about existing subscribers? Or 3rd party links which point direct at your “/feed/” URL? There’s the rub.

What you need to do is redirect them to the new URL, but then if you put a global redirect from the old URL to the new URL, how is FeedBurner going to keep itself updated? (It’ll get redirected back to itself and so never see any of your new posts)

The trick is to use mod_rewrite and check for the HTTP_USER_AGENT of “FeedBurner”.

I’m going to assume that you’ve already got mod_rewrite enabled in your Apache config. If you’re running WordPress on your own server then you most likely have it enabled without even realising.

In your blog’s web directory you will find a “.htaccess” file, open this up and you’ll probably see something like this:

# BEGIN WordPress
 RewriteEngine On
 RewriteBase /
 RewriteRule ^index.php$ - [L]
 RewriteCond %{REQUEST_FILENAME} !-f
 RewriteCond %{REQUEST_FILENAME} !-d
 RewriteRule . /index.php [L]

What you need to do is add these lines between “RewriteBase /” and “RewriteRule ^index.php$ – [L]

RewriteCond %{REQUEST_URI} ^/feed/* [NC]
 RewriteCond %{HTTP_USER_AGENT} !FeedBurner
 RewriteRule ^feed/?.*$ http://feeds.feedburner.com/Whizzyorg [R,L]
 #Next rule

How does it work?

Mod_rewrite matches all the conditions in order, when it hits a Rule it decides if all the preceding conditions are met, and if so acts on the rule. If not, if moves on to the next line. The rules are the the separators between the conditions.

So the new logic says:

  • IF you’re trying to get to “/feed/” (the [NC] makes it case insensitive)
  • AND your USER_AGENT is NOT “FeedBurner” (the ! inverts the logic)
  • THEN re-direct people to the new URL

So, if you are FeedBurner you get access to the old standard WordPress RSS feed and keep yourself updated, if you’re anyone else you get redirected and see the adverts.

The adverts are fairly inconspicuous and are easily blocked by those in the know, so I don’t have a problem with it.

Neat!

2 comments » | Making the world a better place

Over engineering FTW

March 8th, 2012 — 8:26pm — By Will Cooke

Working from home has many advantages.  No commuting, relaxed approach to being dressed, loud music.

One of the drawbacks however, is that you are entirely reliant on your broadband connection.  You could, as many do, have a 3G modem/phone as a backup.  Fine for a few hours, but you’re soon going to run in to data caps, invisible proxies and traffic shapers (more so than a fixed line telco).

 

So what to do?  My solution has been to get a second line installed.  The big UK telcos are running a lot of offers at the moment to try and secure as many customers as they can in readiness for their Triple & Quadruple Play strategies.  This means you can get a reasonable ADSL line and line rental for about 15 to 20 quid a month.

When you’re as reliant on a connection as I am, that’s money well spent.  It also gives you a perfect opportunity to play with some new toys!

Now, you could simply plug in another wifi router to the new line, and when the primary connection fails move your connections over to the secondary one.  But where’s the fun in that?  What you really want to do is some load-balancing and multi-path routing over both the lines.  You won’t see a sudden doubling in download speeds in normal browsing, but certain activities which open a lot of connections to different IP address should benefit, and you get to play with routing tables.

Here’s what it looks like:

 

We don’t really get much diversity until we get south of the exchange, and even then we’re probably going to live in the same fibre all the way to The Internet, but we do work round the much more common problem of ISP configuration errors.

Here’s how it works:

Router 1 (R1) and Router 2 (R2) are two independent ADSL modem/routers  connected to a phone line each and the ethernet port connected to a NIC in a server.  These modem/routers also take care of NAT, so that any traffic coming from inside your network will be translated to appear as if it had come from the appropriate router on it’s way to the rest of the internet.  The server has three NICs in total, one each for the two routers and one back to the LAN for everyone else on the network to use as a gateway.  Let’s use the standard naming convention from http://lartc.org/howto/lartc.rpdb.multiple-links.html

$IF0 = eth0
$IF1 = eth1
$IF2 = eth2

The IP addresses associated with those interfaces are:

$IP0 = 192.168.3.100
$IP1 = 192.168.1.1
$IP2 = 192.168.2.1

The routers are configured with these LAN IPs:

$P1 = 192.168.1.254
$P2 = 192.168.2.254

As you can see on the diagram, the networks are all /24 (255.255.255.0) which gives us network numbers of:

$P0_NET = 192.168.3.0
$P1_NET = 192.168.1.0
$P2_NET = 192.168.2.0

Then we configure the Ubuntu box as a router & NATing gateway:

1.  Enable IP forwarding by editing /etc/sysctl.conf and uncommenting or adding a line like this:

net.ipv4.ip_forward=1

2.  Create two new routing tables.  Edit /etc/iproute2/rt_tables and add a line for each of your providers.  For example:

1 my_isp_name
2 my_other_isp_name

3. Create some routes in each of the routing tables so the server knows where to send data.  They take the general format of:

ip route add $P1_NET dev $IF1 src $IP1 table my_isp_name
ip route add default via $P1 table my_isp_name
ip route add $P2_NET dev $IF2 src $IP2 table my_other_isp_name
ip route add default via $P2 table my_other_isp_name

 

Using the real numbers and the template above gives us this actual set of commands:

ip route add 192.168.1.0 dev eth1 src 192.168.1.1 table my_isp_name
ip route add default via 192.168.1.254 via eth1 table my_isp_name
ip route add 192.168.2.0 dev eth2 src 192.168.2.1 table my_other_isp_name
ip route add default via 192.168.2.254 via eth1 table my_other_isp_name

 

In english, that means:  In order to send traffic to the 192.168.1.0 network or the 192.168.2.0 network send it out of eth1 or eth2 from IP address 192.168.1.1 or 192.168.2.1.  These routes are then stored in the relevent routing table; either one called my_isp_name or one called my_other_isp_name.

 

4. Create some additional routes for communicating between the routing tables and the local interfaces.  This is as much of an efficiency saving as anything, telling each routing table the quickest way to the other interfaces.

ip route add $P0_NET dev $IF0 table my_isp_name
ip route add $P2_NET dev $IF2 table my_isp_name
ip route add 127.0.0.0/8 dev lo table my_isp_name
ip route add $P0_NET dev $IF0 table my_other_isp_name
ip route add $P1_NET dev $IF1 table my_other_isp_name
ip route add 127.0.0.0/8 dev lo table my_other_isp_name

5.  Add the routes to the main routing table, so that traffic knows how to find the networks which have the gateways to the internet on:

ip route add $P1_NET dev $IF1 src $IP1
ip route add $P2_NET dev $IF2 src $IP2

6.  Apply the correct routing rules depending on where the traffic is coming from:

ip rule add from $IP1 table my_isp_name
ip rule add from $IP2 table my_other_isp_name

7.  The science bit.  Create a route which will send traffic down each of the internet connections in turn.  This is a global default route, so any traffic which isn’t bound for the local $P0_NET network will use this rule to get to the internet.

ip route add default scope global nexthop via $P1 dev $IF1 weight 2 nexthop via $P2 dev $IF2 weight 1

As you can see, this isn’t a 50/50 split.  We’ve applied some weighting, in this case because I get more GBs with my_isp_name.  We’re actually going to send twice as much traffic down $IF1 as we are down $IF2.

8.  By default, routes to other networks are cached for 10 mins.  This means that once a route is decided on, traffic will always follow the same route for at least the next 10 minutes.  The counter will be reset every time that particular route is used.  This is great for large downloads because it means that your IP address isn’t suddenly going to change half way through getting the file.  But, for something like IRC where it’s possible that no traffic will be sent for 10 minutes, then the IRC server could well see your IP address changing as traffic flips between the two routes.  IRC servers don’t like that.  So you need to fix some routes so that traffic will always follow the same path.

ip route add 174.143.119.91/32 via $P2

This sets the route to a specific Freenode IRC server to always go via my_other_isp_name.

9.  Last of all, we need to add an extra layer of NAT so that traffic can find its way back to the host on the LAN side.  This is necessary because our internet modem/routers are not really routers at all.  They simply take traffic from one side, rewrite the headers and push it out the other side, while maintaining a look up table of what came from where and doing the reverse when a reply arrives. If you can configure the routes on to your modem, then you don’t need to be reading this in the first place, so let’s assume you can’t.

iptables --table nat --append POSTROUTING --out-interface $IF1 -j MASQUERADE
iptables --table nat --append POSTROUTING --out-interface $IF2 -j MASQUERADE
iptables --append FORWARD --in-interface $IF0 -j ACCEPT

For things like port forwarding you do now have to deal with a bit of an odd NAT situation.  For example, let’s assume I want to run a web server on the same machine that runs the routing.  I decide to only make the server available on the WAN IP address of $R1.  I set up Router 1 to NAT port 80 on the WAN to port 80 on $IP1.  Then I need to configure the web server on the routing server to listen on that interface.  Not too bad really.

And there you have it.  Two internet connections, both available to all the clients on your network.  Make sure that your clients use the IP address of the server as their default gateway and everything should just work.  The additional load on the server is negligible.

If you reboot the server you’ll need to add all the routing information again, so create yourself a Bash and have it run from /etc/rc.local or something.

 

Comment » | linux, Making the world a better place

Overriding Outbound Dial Command Options

September 5th, 2011 — 7:03pm — By Will Cooke

I’ve previously noted that you can apply time limits to calls going out of your Asterisk box:

http://www.whizzy.org/2011/02/more-asterisk-hints/

using the L(nnn,mmm,yyy) options for DIAL_TRUNK_OPTIONS. But, what if you don’t want to limit the length of calls for a specific trunk?

Well, FreePBX has a context called [macro-dialout-trunk-predial-hook] which lets you jump in at the very last moment and override any settings you like, which is perfect for this sort of thing.

I got the idea from here:

http://www.freepbx.org/book/export/html/5893

I’ve added this to extensions_custom.conf:

[macro-dialout-trunk-predial-hook]
 exten => s,1,NoOp(Trunk ${OUT_${DIAL_TRUNK}} selected)
 exten => s,n,Gotoif($["${OUT_${DIAL_TRUNK}}" != "SIP/TRUNK_NAME"]?skip)
 exten => s,n,NoOp(Setting DIAL_TRUNK_OPTIONS to Ww)
 exten => s,n,Set(DIAL_TRUNK_OPTIONS="Ww")
 exten => s,n(skip),MacroExit()

Which tests the name of the trunk which is to be used and explicitly sets DIAL_TRUNK_OPTIONS if it matches, otherwise nothing happens. This is a very powerful feature and my extremely crude hack doesn’t do it justice. I wanted to make a note of this before I forgot.

A better way might be to look at the dial prefix? I’ll investigate that option, but for now that will have to do.

Comment » | asterisk

NOOP scheduler – not worth it for SSD

August 16th, 2011 — 9:45am — By Will Cooke

I’ve got a new Thinkpad X220i with a 128GB SSD.

Reading around the internet I found a lot of stuff about squeezing a bit more throughput from your drive.

I did a couple of benchmarks in Ubuntu 11.04:

  • Adding noatime,discard to /etc/fstab

Result:  No change.  Not better or worse, but in theory the “discard” will help in the long term

Decision:  Switch this on.

  • Setting the scheduler to noop in /etc/rc.local

Result: Average read rate increased by 2MB/s but Average access time went from 0.2ms to 0.3 ms.  In itself that’s not a real problem, but the graphs show a different picture.

Without NOOP

Note the scale on the right hand side.

 

 

With NOOP

The access speed is all over the place.

Decision:  Switch this off

 

I’d be interested in hearing if your experience differs, but it seems to me that “Doing nothing” is a valid choice.  Ubuntu, out of the box, doesn’t really require any fettling in order to get the best from your SSD.

 

 

Comment » | linux

Off to pastures new

July 22nd, 2011 — 3:33pm — By Will Cooke

Comment » | Uncategorized

TCP sessions causing lag

July 4th, 2011 — 12:06pm — By Will Cooke

I like to play Call Of Duty online. I find it’s a really good way to get angry and frustrated at inanimate objects. I also run a Bit Torrent client on my network.

I don’t suck at MW2 and I’m not too bad at Black Ops either, but on the PS3 Black Ops has been very unstable to the point of hanging the console.

Anyway, what really gets me cross is Lag. You can tell a match is going to be laggy the second the game opens. It feels as if you’re running through treacle. “GET A MOVE ON” I scream. And then it starts; you get shot while behind brick walls, your shots fail to connect even though “IT WAS CLEARLY A HEADSHOT WHY WONT YOU JUST DIE?”. And so on. You know what I’m talking about.

Originally I had a written a script to ping the PS3 and when it came online and tell Transmission to pause all the Torrents. When the PS3 went offline again it would start them. I thought this would be enough, but it isn’t. Even with Transmission paused and the bandwidth limited to zero kBps it still maintains a connection to other peers.

While this isn’t a significant amount of traffic over the WAN – it really does cause lag. Presumably having something like 1000 NAT sessions in the memory of my router is a bit of a stretch, so now I simply kill Transmission when the PS3 comes online. It takes a few minutes for the sessions to get pruned but it’s made a noticeable difference to the quality of my matches.

Of course, you’re still at the mercy of the internet connection of whoever hosts the match – but that’s beyond your control.

Comment » | Making the world a better place

Trimming Freesat Channels In MythTV

June 8th, 2011 — 12:42pm — By Will Cooke

There are loads and loads of free-to-air channels available on the Astra 28 constellation, the vast majority of which I do not watch.

So to make things a bit easier for me after a full re-scan, I’ve put together a list of the channels I don’t watch and with a tiny bit of SQL I can trim them from my channel list.

To make things a bit easier for you here is a SQL dump of my “unwatched channels” list:

unwatched_channels.sql

And here is the SQL to trim these from your channel list:

update channel set visible=0,useonairguide=0 where name in (select name from unwatched_channels)

You’ll probably want to edit that list yourself to remove and add the channels as you prefer. Generally speaking, my list trims:

  • Regional variations
  • Specialist interest
  • Shopping
  • Games and other text based services

I’ll update this list occasionally, this page will always have my most up to date information.

  • UPDATE: 6 Sept 11.  Refreshed channel list
  • UPDATE: 8 Oct 11. Refreshed channel list
  • UPDATE: 14 Dec 2011.  Refreshed channel list
  • UPDATE: 4 Aug 2012.  Refreshed channel list
  • UPDATE: 29 Dec 2013.  New list of channel IDs available here: unwatched_by_chanid.  Add a new table and import that CSV file.  Then do a “update channel set visible=0 where chanid in (select chanid from <your new table>”

Comment » | linux, Making the world a better place, tv

Maggy Scraggle Loves The Beautiful Ice-cream Man

April 17th, 2011 — 9:07am — By Will Cooke

 

I was fortunate enough to be introduced to this book by my wife reading it to our son.

1 comment » | Uncategorized

More Asterisk hints

February 13th, 2011 — 7:21pm — By Will Cooke

Wow – a day of fixing loads of niggling little Asterisk problems!

  • Max duration

My calling plan gives me unlimited free calls as long as those calls are under an hour in duration. Pretty standard BT stuff. If you do make a call over an hour you don’t just get charged for that bit of the call over the hour, oh no, you get charged for the whole call.
Anyway – we have the technology to defeat them!

In FreePBX under General Settings change your Asterisk Outbound Dial command options to include:

L(3360000,3240000,10000)

which will drop the call after 3360000ms (56 minutes) and should alert you at 3240000ms.

  • Courtesy Tone

The above works very well and drops the calls, but without a bit of extra magic you don’t get the warning in your ear – it just drops the call. To enable the warning tones etc edit this:

/etc/asterisk/features_general_custom.conf

and add

courtesytone=beep

substituting “beep” for what ever noise you want to hear.

  • Call Recording

By also adding:

Ww

to the Dial command options in #1 you can press *1 when you’re in a call to start recording.  It plays the courtesy tone to both parties though.

1 comment » | Uncategorized

Back to top