Cisco 7941, Asterisk and SIP

I got a Cisco 7941 off eBay.  This is a phone which was £400 when new (some time around 2004) but can now be picked up for about £10.  These phones went End Of Sale in January 2010, so even if mine was one of the last phones to roll off the production line it’s still about 7 years old but it’s still working perfectly.  A testament to the good build quality of these phones, and perhaps the previous owner’s careful handling.

Since these devices are no longer supported many companies will be getting rid of them (or probably already have) so there should be some bargains to be had for phone geeks.

Q: Does the Cisco 7941 work with Asterisk?
A: Yes.  You need to load the SIP firmware (the focus of this post) or chan-sccp (out of scope for this post but I’ll check it out at some point).

Q: Does the Cisco 7941 work with SIP?
A: Yes.  You need to flash the correct firmware though.

Q: Is it really hard to get working?
A: No.  If you’re comfortable with Linux and a few command line tools.  And assuming you already have Asterisk set up.

Q: Is a lot of the information on the web about how to set up the 7941 wrong?
A: Yes.  There is a lot of confusion about config files (the 7940 and 7941 use different ones).

Q: Will you tell us how you got your phone to work?
A: Yes!  However – this is what works for me.  You will need to tweak the config in places.

 

The steps to getting this phone working as a SIP extension on Asterisk on Ubuntu / Raspberry Pi:

  1. Set up a TFTP server
  2. Download the SIP firmware from Cisco
  3. Flash the phone with the firmware via the TFTP server
  4. Configure the SIP extension in Asterisk
  5. Write the config files for the phone and upload them via the TFTP server
  6. Make a call!
  7. Optional Extras
    1. Dial plan
    2. Ring tones
    3. Dial tones
    4. Wallpaper
    5. Telephone Directory
  8. Final Tip

Set up a TFTP Server

The phone will download it’s firmware and config via TFTP.  It needs to download it’s config on every boot, so you will always need a TFTP server running.  I think that if the TFTP server is unavailable it will just use the previous config, so it’s possible that you can get away without it, but I haven’t tried.  My recommendation is that you install dnsmasq.  It’s a small and full featured DNS server which also includes a DHCP & TFTP server which are easy to configure and it’s almost certainly packaged for your distro.  You should also (temporarily) disable any other DHCP servers on your local network so that dnsmasq is the only thing offering DHCP addresses.  This will simplify the process of getting the phone to find the TFTP server, since with dnsmasq it will all be automatic.  If you later re-enable your original DHCP server, say on your router, then you will need to configure it to give out the address of the dnsmasq TFTP server and disable DHCP on dnsmasq.  In my opinion, if you’re going to be running a Cisco IP phone on your network you’d be better off moving all DHCP to dnsmasq.

The full configuration of dnsmasq it’s out of scope for this doc, but in a nutshell you need these in your dnsmasq config:

  • Set up a DHCP range
dhcp-range=192.168.1.1,192.168.1.100,24h
  • Enable the TFTP server
enable-tftp
  • Set the TFTP path
tftp-root=/home/<your user>/tftp  (or whatever works for you)

Download the SIP Firmware from Cisco

Usually Cisco require a valid support contract before you can download anything useful from their website, but it seems that since these phones are now out of support they have offered up the firmware free of charge.  You do still need to register an account to download the files.  At the time of writing the latest version is 9.4.2 SR 3 dated 14th February 2017 – so bang up to date, even though these phones are end-of-life.  Bizarre, but good for us.  Thanks Cisco!

Go here: https://software.cisco.com/download/type.html?mdfid=280083379&catid=280789323

Follow the link to the SIP software.

You want to download the “SIP firmware files only”

Unzip that file into the root of your TFTP server (the location you set in the previous step).  You should have 8 files in there:

apps41.9-4-2ES26.sbn
dsp41.9-4-2ES26.sbn
term41.default.loads
cnu41.9-4-2ES26.sbn
jar41sip.9-4-2ES26.sbn
term61.default.loads
cvm41sip.9-4-2ES26.sbn
SIP41.9-4-2SR3-1S.loads

This is everything you need to reflash your phone to the latest SIP firmware.  Now you need to get the phone to reboot in to firmware download mode.

Flash the phone with the firmware via the TFTP server

  1. Unplug the phone from the power.  Make sure that the network cable is still connected (unless you’re using using PoE).
  2. Plug the power back in and hold down the # key
  3. Eventually you will see the “line” lights start to flash orange.  It might take a couple of minutes to get to this stage, don’t give up, just keep holding down #
  4. When the line lights are flashing type 123456789*0#  This will start firmware download mode.
  5. The screen will go black for a moment and then go through the process of getting an IP address and connecting to the TFTP server
  6. Once connected to the TFTP server the software download will start
  7. The phone will reboot once download is complete and present you with an “Unprovisioned” message on the screen.  This is good news!  The phone firmware has now been updated.

I put together a video showing this process.  It’s not very interesting but it will give you an idea of what to expect.  The actual downloading of the firmware section has been sped up 3X.

 

Configure the SIP extension in Asterisk

Now you need to configure the SIP extension in Asterisk.  Do this as per any other SIP extension, but bear this important piece of information in mind:  The Cisco 7941 can only deal with 8 character passwords, so keep your SIP authentication secret to 8 characters.

While you’re in Asterisk configuration mode, take a moment to note down these bits of information as well (in Advanced SIP settings in FreePBX):

  • RTP Port range, start and end.
  • Bind Port (probably 5060)

Write the config files for the phone and upload them via the TFTP server

Please take the time to read this section fully,  this is the part that is most troublesome.  The Cisco 7941 is very picky about it’s config file and even a small mistake will stop the phone from working.  These settings are specific to the 79×1 series of phones running at least version 8.x of the firmware.  If your phone is not a 79×1 and/or is not running v9.x.x of the firmware then these settings are not for you.

Once the phone has loaded it’s firmware and booted, it will go looking for a file called SEP<PHONE MAC ADDRESS>.cnf.xml.  So if the MAC address of your phone is 11:22:33:44:55:66 then the config file needs to be named SEP112233445566.cnf.xml.  This file needs to be in the root of your TFTP server.

You will see mention of a file called XMLDefault.cnf.xml.  If you’ve only got a few phones, don’t worry about this, you don’t need it.

So here is a config file which is about as minimal as I can make it:

<device>
    <deviceProtocol>SIP</deviceProtocol>
    <sshUserId>cisco</sshUserId>
    <sshPassword>cisco</sshPassword>
    <ipAddressMode>0</ipAddressMode>

    <devicePool>
        <dateTimeSetting>
            <dateTemplate>D/M/Ya</dateTemplate>
            <timeZone>GMT Standard/Daylight Time</timeZone>
            <ntps>
                <ntp>
                    <name>#IP ADDRESS OF AN NTP SERVER#</name>
                    <ntpMode>Unicast</ntpMode>
                </ntp>
            </ntps>
        </dateTimeSetting>

        <callManagerGroup>
            <members>
                <member priority="0">
                    <callManager>
                        <ports>
                            <ethernetPhonePort>2000</ethernetPhonePort>
                            <sipPort>#SIP PORT NUMBER FROM YOUR ASTERISK SERVER#</sipPort>
                        </ports>
                        <processNodeName>#IP ADDRESS OF YOUR ASTERISK SERVER#</processNodeName>
                    </callManager>
                </member>
            </members>
        </callManagerGroup>
    </devicePool>

    <sipProfile>
        <sipProxies>
            <registerWithProxy>true</registerWithProxy>
        </sipProxies>
        <sipCallFeatures>
            <cnfJoinEnabled>true</cnfJoinEnabled>
            <rfc2543Hold>false</rfc2543Hold>
            <callHoldRingback>2</callHoldRingback>
            <localCfwdEnable>true</localCfwdEnable>
            <semiAttendedTransfer>true</semiAttendedTransfer>
            <anonymousCallBlock>2</anonymousCallBlock>
            <callerIdBlocking>2</callerIdBlocking>
            <dndControl>0</dndControl>
            <remoteCcEnable>true</remoteCcEnable>
        </sipCallFeatures>

        <sipStack>
            <sipInviteRetx>6</sipInviteRetx>
            <sipRetx>10</sipRetx>
            <timerInviteExpires>180</timerInviteExpires>
            <timerRegisterExpires>3600</timerRegisterExpires>
            <timerRegisterDelta>5</timerRegisterDelta>
            <timerKeepAliveExpires>120</timerKeepAliveExpires>
            <timerSubscribeExpires>120</timerSubscribeExpires>
            <timerSubscribeDelta>5</timerSubscribeDelta>
            <timerT1>500</timerT1>
            <timerT2>4000</timerT2>
            <maxRedirects>70</maxRedirects>
            <remotePartyID>true</remotePartyID>
            <userInfo>None</userInfo>
        </sipStack>

        <autoAnswerTimer>1</autoAnswerTimer>
        <autoAnswerAltBehavior>false</autoAnswerAltBehavior>
        <autoAnswerOverride>true</autoAnswerOverride>
        <transferOnhookEnabled>false</transferOnhookEnabled>
        <enableVad>false</enableVad>
        <preferredCodec>g711ulaw</preferredCodec>
        <dtmfAvtPayload>101</dtmfAvtPayload>
        <dtmfDbLevel>3</dtmfDbLevel>
        <dtmfOutofBand>avt</dtmfOutofBand>
        <alwaysUsePrimeLine>false</alwaysUsePrimeLine>
        <alwaysUsePrimeLineVoiceMail>false</alwaysUsePrimeLineVoiceMail>
        <kpml>3</kpml>
        <natEnabled>false</natEnabled>
        <phoneLabel>#PHONE NAME#</phoneLabel>
        <stutterMsgWaiting>0</stutterMsgWaiting>
        <callStats>false</callStats>
        <silentPeriodBetweenCallWaitingBursts>10</silentPeriodBetweenCallWaitingBursts>
        <disableLocalSpeedDialConfig>false</disableLocalSpeedDialConfig>
        <startMediaPort>#RTP START PORT#</startMediaPort>
        <stopMediaPort>#RTP END PORT#</stopMediaPort>

        <sipLines>
            <line button="1">
                <featureID>9</featureID>
                <featureLabel>#EXT NUM#</featureLabel>
                <proxy>USECALLMANAGER</proxy>
                <port>#SIP PORT#</port>
                <name>#EXT NUM#</name>
                <displayName>#EXT NAME#</displayName>
                <autoAnswer>
                    <autoAnswerEnabled>2</autoAnswerEnabled>
                </autoAnswer>
                <callWaiting>3</callWaiting>
                <authName>#SIP AUTH NAME#</authName>
                <authPassword>#8 CHAR PASSWORD#</authPassword>
                <sharedLine>false</sharedLine>
                <messageWaitingLampPolicy>1</messageWaitingLampPolicy>
                <messagesNumber>#VM NUM#</messagesNumber>
                <ringSettingIdle>4</ringSettingIdle>
                <ringSettingActive>5</ringSettingActive>
                <contact>#EXT NUM#</contact>
                <forwardCallInfoDisplay>
                    <callerName>true</callerName>
                    <callerNumber>true</callerNumber>
                    <redirectedNumber>false</redirectedNumber>
                    <dialedNumber>true</dialedNumber>
                </forwardCallInfoDisplay>
            </line>

            <line button="2">
                <featureID>9</featureID>
                <featureLabel>#EXT NUM#</featureLabel>
                <proxy>USECALLMANAGER</proxy>
                <port>#SIP PORT#</port>
                <name>#EXT NUM#</name>
                <displayName>#EXT NUM#</displayName>
                <autoAnswer>
                    <autoAnswerEnabled>2</autoAnswerEnabled>
                </autoAnswer>
                <callWaiting>3</callWaiting>
                <authName>#SIP AUTH NAME#</authName>
                <authPassword>#8 CHAR PASSWORD#</authPassword>
                <sharedLine>false</sharedLine>
                <messageWaitingLampPolicy>1</messageWaitingLampPolicy>
                <messagesNumber>#VM NUM#</messagesNumber>
                <ringSettingIdle>4</ringSettingIdle>
                <ringSettingActive>5</ringSettingActive>
                <contact>#EXT NUM#</contact>
                <forwardCallInfoDisplay>
                    <callerName>true</callerName>
                    <callerNumber>true</callerNumber>
                    <redirectedNumber>false</redirectedNumber>
                    <dialedNumber>true</dialedNumber>
                </forwardCallInfoDisplay>
            </line>
        </sipLines>

        <voipControlPort>#SIP PORT#</voipControlPort>
        <dscpForAudio>184</dscpForAudio>
        <ringSettingBusyStationPolicy>0</ringSettingBusyStationPolicy>
        <dialTemplate>dialplan.xml</dialTemplate>
    </sipProfile>

    <commonProfile>
        <phonePassword></phonePassword>
        <backgroundImageAccess>true</backgroundImageAccess>
        <callLogBlfEnabled>1</callLogBlfEnabled>
    </commonProfile>

    <loadInformation>SIP41.9-4-2SR3-1S</loadInformation>
    <vendorConfig>
        <disableSpeaker>false</disableSpeaker>
        <disableSpeakerAndHeadset>false</disableSpeakerAndHeadset>
        <pcPort>0</pcPort>
        <settingsAccess>1</settingsAccess>
        <garp>0</garp>
        <voiceVlanAccess>0</voiceVlanAccess>
        <videoCapability>0</videoCapability>
        <autoSelectLineEnable>0</autoSelectLineEnable>
        <webAccess>0</webAccess>
        <spanToPCPort>1</spanToPCPort>
        <loggingDisplay>1</loggingDisplay>
        <loadServer></loadServer>
        <sshAccess>0</sshAccess>
    </vendorConfig>

    <versionStamp>001</versionStamp>
    <networkLocale>United_Kingdom</networkLocale>
    <networkLocaleInfo>
        <name>United_Kingdom</name>
        <uid>64</uid>
        <version>1.0.0.0-4</version> 
    </networkLocaleInfo>

    <deviceSecurityMode>1</deviceSecurityMode>
    <authenticationURL></authenticationURL>
    <servicesURL></servicesURL>
    <transportLayerProtocol>2</transportLayerProtocol>
    <certHash></certHash>
    <encrConfig>false</encrConfig>
    <dialToneSetting>2</dialToneSetting>
</device>

Copy and paste this into a text editor and search and replace the following:

  • #IP ADDRESS OF AN NTP SERVER#  –  with  –  the IP address of an NTP server
  • #SIP PORT FROM YOUR ASTERISK SERVER#  –  with  –  the SIP port of your asterisk server is listening on.  Probably 5060
  • #IP ADDRESS OF YOUR ASTERISK SERVER#  –  with  –  the IP address of your Asterisk server
  • #PHONE NAME#  –  with  –  the text you want to appear at the top right of the phone screen
  • #RTP START PORT#  –  with  –  the RTP port range start from the previous stage
  • #RTP END PORT#’  –  with  –  the RTP port range end from the the previous stage
  • #EXT NUM#  –  with  –  the Asterisk extension number as configured in the previous stage
  • #SIP PORT#  –  with  –  the SIP port of your Asterisk server.  Probably 5060
  • #EXT NAME#  –  with  –  the name you want to give this extension
  • #SIP AUTH NAME#  –  with  –  the username for the SIP extension as configured in Asterisk
  • #8 CHAR PASSWORD#  –  with  –  the password for the SIP extension as configured in Asterisk
  • #VM NUM#  –  with  –  the number you dial for Voicemail.  Probably *98

Note that this config file has two lines configured.  If you just blindly search and replace you’ll end up with two extensions configured the same.

Some comments on what some of the XML tags do:

  • ipAddressMode – 0 is IP v4 only. But this seems to have little effect.
  • registerWithProxy – true – Registers the device with Asterisk, this allows incoming calls to be sent to the phone.  If you’re getting “Unregistered” message on the screen, check you have this set.
  • featureId – 9 is SIP
  • autoAnswerEnabled – 2 – 2 seems to be “off”
  • webAccess – 0 – 0 is on (?!)
  • sshAccess -0 – ditto
  • versionStamp – bump this up every time you make a change.  Something like YYYMMDD001..2..3 etc
  • networkLocale – United_Kingdom – sets the tones to UK, see the optional extras section for more info.
  • transportLayerProtocol – 2 is UDP, 1 is TCP
  • dialToneSettings – 2 is “always use internal dialtone”.  See option extras for more info.

Edit this file as necessary and then save it to the root of your TFTP server with the filename: SEP<MAC>.cnf.xml.  If your phone MAC address was aa:bb:33:44:55:66 then the filename would be: SEPAABB33445566.cnf.xml  Note that it’s case sensitive, letters in the MAC address should be in upper case the extensions should be in lowercase.  You can get the MAC address for the phone from the syslog on your dnsmasq server.

If your phone is still in “Unprovisioned” mode it will have been asking for this config file repeatedly.  Once you save the file you should see the phone reboot shortly afterwards.  It may download the firmware again for some reason, just leave it to get on with it.

Make a call!

If everything has worked you should see your extension listed on the right hand side of the screen near the buttons, and the name of the phone should appear at the top of the screen.  If the icon next to the line buttons is that of a phone without an x through it, then you’re probably good to go!  Press the line button and see if you get a dial tone.  If not, then check the phone logs:

  • Press Settings
  • Press 6
  • Press 1

From these logs you should be able to tell if the phone has loaded your config correctly.  Errors about “updating locale” or “no trust list installed” can be ignored.  If there is a problem with the config file itself a generic error will be listed here.  If the phone won’t load the config file the most likely reason is that there is a typo in your XML file.  Good luck finding it.  You can SSH in to the phone to get more detailed logs and debugging information, but I haven’t tried this yet.  Google is your friend.

Optional Extras

Dial plan

The dial plan tells the phone how to process the digits you type and when to start sending the call.  Without a dial plan the phone simply waits a period of time for you to stop typing numbers before I decides you’re done and starts the call.  By using a dial plan you can reduce the amount of time spent waiting after you’ve finished keying in the number.  Here’s an example plan I’ve edited based on this post on Phil Lavin’s blog (Thanks Phil!) http://phil.lavin.me.uk/2012/11/united-kingdom-dial-plan-xml-for-cisco-phones/

<DIALTEMPLATE>
    <TEMPLATE MATCH="999" Timeout="0"/> <!-- Emergency -->
    <TEMPLATE MATCH="112" Timeout="0"/> <!-- Emergency -->
    <TEMPLATE MATCH="0500......" Timeout="0"/> <!-- Apparently 0500 is always 10 digits -->
    <TEMPLATE MATCH="0800......" Timeout="0"/> <!-- Apparently 0800 is always 10 digits -->
    <TEMPLATE MATCH="00*" Timeout="5"/> <!-- International, 00 prefixed. No fixed length -->
    <TEMPLATE MATCH="0.........." Timeout="0"/> <!-- UK 11 digit, 0 prefixed -->
    <TEMPLATE MATCH="26...." Timeout="0"/> <!-- My local STD numbers start 26 -->
    <TEMPLATE MATCH="\*.." Timeout="0"/> <!-- Asterisk *.. codes -->
    <TEMPLATE MATCH="\*98...." Timeout="0"/> <!-- Asterisk direct VM access *981234-->
    <TEMPLATE MATCH="1..." Timeout="0"/> <!-- Internal numbers -->
    <TEMPLATE MATCH="2..." Timeout="0"/>  <!-- Internal numbers -->
    <TEMPLATE MATCH="*" Timeout="5"/> <!-- Anything else -->
</DIALTEMPLATE>

Save this to the root of your TFTP server, named “dialplan.xml” (lowercase).

Ring tones

Everyone likes novelty ringtones.  You can find plenty of ringtones in a format which is compatible with your phone (raw format, 8000 Hz sample rate, 8 bit, ulaw, max 2 seconds).  These files need to be placed in to the root of your TFTP server.  I tried putting them in a sub-directory but it didn’t work.  Then you need to create a file called “ringlist.xml” also in the root of the server.  The format of this file is:

<CiscoIPPhoneRingList>
    <Ring>
        <DisplayName>#DISPLAY TEXT#</DisplayName>
        <FileName>#FILENAME#</FileName>
    </Ring>
    <Ring>
        <DisplayName>#DISPLAY TEXT#</DisplayName>
        <FileName>#FILENAME#</FileName>
    </Ring>
</CiscoIPPhoneRingList>

Filenames are case sensitive.  Once you’ve save this file, copy it to “distinctiveringlist.xml” as well.  This will allow you to set ring tones for the default ringer and different rings for each line.

Dial tones

By default the 7941 will have a psuedo North American dial tone.  This is annoyingly shrill (yes, it is).  By specifying a NetworkLocale in the phone config we can get it to load a different set of informational tones from a file stored in (per the example XML above) United_Kingdom.  In the root of the TFTP server create a directory called United_Kingdom.  In this directory you need to create a file called g3-tones.xml.  Bizarrely Cisco require you to have a support contract in order to download the correct tones settings for your country, despite giving the phone firmware away for free.  Go figure.  So this means I’m not going to paste the XML here.  If you search hard enough you’ll find an example g3-tones.xml file you can use as a base.  In our phone configuration above we told the phone to always use the internal dialing tone, so this means we only need to change the idial section of the tones file.  The magic numbers are:

  • 31538
  • -780
  • 30831
  • -973

Wallpaper

The phone comes with a single default wallpaper with horizontal lines on it.  This is easily replaced by your own designs with a simple PNG.   Create a directory in the root of the TFTP server called Desktops.  In here create another directory called 320x196x4.

In to this directory you need to place a “List.xml” file:

<CiscoIPPhoneImageList>
    <ImageItem Image="TFTP:Desktops/320x196x4/ubuntu-tn.png"
       URL="TFTP:Desktops/320x196x4/ubuntu.png"/>
</CiscoIPPhoneImageList>

The “-tn” in the file is a smaller thumbnail version of the larger image.  The PNGs need to be sized exactly 320×196 for the large and 80×49 for the thumbnail.  Here’s something to get you started:

Telephone Directory

You will have noticed that the phone has a “Directories” button and a “Services” button.  I haven’t managed to add an extra phone book to the Directories button yet although I think it’s certainly possible, just that the XML file refuses to do anything.  However, I have got a phone directory working on the Services button.

In the main phone config file there is a tag for “servicesURL”.  Point this to a web server on your local network which will serve up an XML file.  For example:

 <servicesURL>http://192.168.1.1/phone/directory.xml</servicesURL>

Assuming you are using Apache 2 to serve that XML file (or it could equally be a CGI script which generates the XML dynamically from a database such as the FreePBX phone book) the format looks like this:

<CiscoIPPhoneDirectory>
   <Title>Whizzy Towers</Title>
   <DirectoryEntry>
       <Telephone>1500</Telephone>
       <Name>Lenny</Name>
   </DirectoryEntry>
   <DirectoryEntry>
       <Telephone>1234</Telephone>
       <Name>Speaking Clock</Name>
   </DirectoryEntry>
</CiscoIPPhoneDirectory>

Important note:  You must tell Apache to serve those files as type “text/xml“.  “application/xml” will not work.

You can do this via your CGI script, or if you want to serve a static file add something like this to your Apache config:

 <Location /phone/>
     ForceType text/xml
 </Location>

Inside your VirtualHost section.

Final Tip

Watch /var/log/syslog on the machine running the TFTP server.  You’ll be able to see exactly what files the phone is asking for.  Bear in mind that it does ask for files it doesn’t strictly need, so don’t worry too much about file not found errors unless it’s one of the above.

Here’s a final video showing the boot up for a fully configured phone

Combining MythTV and Asterisk

I’ve had this idea for a while and with the discovery of the Google Text-to-speech and Voice Recognition AGI scripts from Zaf (http://zaf.github.io/asterisk-googletts/ & http://zaf.github.io/asterisk-speech-recog/) I’ve implemented a quick proof-of-concept.

You can see the results in this YouTube video:

Over the next few days I’ll tidy up the code and write up a blog post about how to do it.  It’s pretty straight forward though, using APIs provided by Google, MythTV and Asterisk and then just glueing them together.

 

Asterisk UK Caller ID & SMS redux

Update 28 Dec 2013:  As far as caller ID goes, I might have been barking up completely the wrong tree here.  Have a look here: http://forums.digium.com/viewtopic.php?f=1&t=85028.  I’ve re-implemented that patch for Dahdi and I’m testing it now.  Get in touch if you want to test it as well.  Once I’m happy it’s actually working, I’ll post it here.

 

tl;dr: Scroll down to “How to fix Caller ID in the UK on Asterisk 11 and Dahdi 2.6.1″

I’ve been having some problems with Asterisk 1.6 for a while now, in that when I’m dialled in to conferences via SIP, or when I call someone via the work Asterisk server from my Asterisk server at home I get cut off every 15 minutes. My colleagues have all kind of accepted this now, it was starting to get annoying. It was a known problem in Asterisk 1.6, and so I was going to have to upgrade.

The Ubuntu Asterisk packages are fairly up-to-date but FreePBX isn’t packaged, and it’s a pain in the backside to get set up with the correct permissions, especially if you install Asterisk from packages. I had a spare machine anyway so I decided that I’d give FreePBX Distro a go and once I’d burnt the ISO to a CD (ya rly) I found the set up process to be simple and quick.

FreePBX Distro is a stripped down CentOS (2.6.32 kernel) which boots quick, has minimal footprint and bundles in loads and loads of FreePBX “apps” like conferences, voicemail, blacklisting and makes setting up extensions a breeze. It also includes things like kernel source for I sell settlement so building extra modules from source is pretty easy. Which is a good job, because it was about to get messy.

If you’ve found this page via Google then this will probably sound pretty familiar:

  • You got a TDM400p four port FXO/FXS interface card, or a clone from someone like OpenVox (which work perfectly by the way – as do those cheap daughter cards off eBay).
  • You’ve upgraded to Asterisk 11, or Asterisk 1.8 perhaps, or changed to the Dahdi 2.6.1 drivers and all of a sudden your ”UK Caller ID” has “stopped working” or at the very least become “intermittent“.
  • It was working fine before, so it’s not a hardware issue and your phone provided have NOT suddenly stopped sending caller ID information, regardless of what people of forums tell you. No, in fact this is clearly a bug which has crept in somewhere along the line.
  • You’ve searched and searched and tried things like setting rxgain=x.x and txgain=x.x or whatever. (Side note: setting txgain and rxgain in chan_dahdi_channels_custom.conf has no effect, the only place is seems to work is in chan_dahdi_groups.conf)
  • You might have even stumbled across a post hinting that cid_rxgain=x.x will solve your problems, only it didn’t

If that sounds like you, dear reader, then read on. I am your salvation.

  1. This page: http://downloads.openvox.cn/pub/drivers/callerid_patches/ (I can only attribute this discovery to divine intervention)
  2. Find a new wctdm.c driver here: http://downloads.openvox.cn/pub/drivers/callerid_patches/2.6.1-wctdm.c
  3. Download the 2.6.1 Dahdi source from here: http://downloads.asterisk.org/pub/telephony/dahdi-linux-complete/dahdi-linux-complete-2.6.1+2.6.1.tar.gz
  4. Replace dahdi-linux-complete-2.6.1+2.6.1/linux/drivers/dahdi/wctdm.c with the version downloaded from openvox
  5. Note it doesn’t compile, add the missing semicolon to line 333 (or thereabouts), successfully build and install the new driver
  6. You might need to change /etc/modprobe.d/dahdi.conf and add:
options wctdm opermode=UK fwringdetect=1 battthresh=4
  1. restart Dahdi and Asterisk and caller ID should now be fixed! Yay!

How to fix Caller ID in the UK on Asterisk 11 and Dahdi 2.6.1

Here is a patch against wctdm.c from stock Dahdi 2.6.1 with the fixes in: wctdm.c.patch

OR: Replace dahdi-linux-complete-2.6.1+2.6.1/linux/drivers/dahdi/wctdm.c with this one: wctdm.c

OR: If you’re running FreePBX distro 64 bit “BETA-3.211.63-5 Release Date-01-24-13″ (or close) here is a binary driver: wctdm.ko  Move it to /lib/modules/2.6.32-279.11.1.el6.x86_64/dahdi/wctdm.ko

It works for me. I have not had to increase the rxgain, or txgain or anything else. It just works.

Once that’s working, we can move on to…

How to send & receive SMS in the UK on FreePBX Distro Beta 3.211

I’ve talked about receiving and sending SMSes before, but that was quite a long time ago, and things have moved on a bit since then. I’ve also learnt a bit more about the Asterisk dial plan, and have a slightly cleaner way of doing it now.

You need to have working caller ID. Hint: see above.
You need to install smsq.
smsq doesn’t come built in FreePBX Distro so you have to build it yourself if you want to send SMS. This is pretty easy:

  1. Download the Asterisk 11.2.1 source from here: http://downloads.asterisk.org/pub/telephony/asterisk/asterisk-11.2.1.tar.gz
  2. Install popt: yum install popt-devel.i686 popt.i686 popt.x86_64 popt-devel.x86_64 (prerequisite for smsq)
  3. In the Asterisk source directory do “./configure”
  4. run “make menuselect”
  5. Scroll down to “Utilities”
  6. In the right hand pane, check “smsq”
  7. Save & Exit
  8. run “make”
  9. DO NOT run “make install” as this will trash your FreePBX config
  10. Once that’s done you should find an executable smsq in the utils directory

Just to be sure, create /var/spool/asterisk/sms/motx and /var/spool/asterisk/sms/mtrx
You can find more details on building Asterisk here: http://blogs.digium.com/2012/11/05/how-to-install-asterisk-11-on-centos-6/

You need to be able to send SMS in order to register with the BT SMS system, otherwise when you receive an SMS you will get a phone call from the BT robots who will read the text to you, badly.

To register with BT try this:

<path to smsq>/smsq –motx-channel=DAHDI/2/17094009 -d 00000 -m “test”

where DAHDI/2 is the channel number of your outgoing line.

If you look in the Asterisk logs you should see the message going out with some TX and RX hex dumps. If it works, a short while later you will receive a phone call from either the SMS system trying to talk modem at you, or the robot. Either way, this means your text went out. Great success!

Now we need to set up receive, which is pretty straight forward.

Edit /etc/asterisk/extensions_custom.conf and add the below. Note – I’ve added a lot of comments to explain what’s going on, it might improve readability if you take them out.

[from-internal-additional-custom]
;; This makes sure our new 'app' gets included in the FreePBX dial plan
include => app-rx-sms
[app-rx-sms]
;; This is the app to receive an SMS which will be called when we match
;; the caller ID of an incoming call to the BT computer
exten => s,1,Answer()
;; This wait seems to help
exten => s,n,Wait(0.5)
;; We simply use the built in SMS function
exten => s,n,SMS(default,a)
;; Hangup with code 16, normal termination.
exten => s,n,Hangup(16)
;; Done
;; We also extend the from-pstn context with a branch to the SMS app
;; if the caller ID matches, hence why it's necessary to get caller
;; id working first
[from-pstn-custom]
;; we add ourselves in to the end of the current dialplan
;; priority 7 gets us there...
exten => s,7,GotoIf($["${CALLERID(num)}" = "08005875290"]?app-rx-sms,s,1)
;; pretty simple - if the caller ID matches, go to the sms app
exten => s,n(dest-ext),Goto(ext-group,1100,1)
;; Note: 1100 is my "ring all" group. You will need to change 1100
;; to what ever you use. You should see this at the end of the
;; from-pstn context in extensions_additional.conf

Save that, and in asterisk do a “dialplan reload”. From smsq send an SMS to 00000 saying “register”· If everything works you should get a text file in /var/spool/asterisk/sms/mtrx.

Overriding Outbound Dial Command Options

I’ve previously noted that you can apply time limits to calls going out of your Asterisk box:

http://www.whizzy.org/2011/02/more-asterisk-hints/

using the L(nnn,mmm,yyy) options for DIAL_TRUNK_OPTIONS. But, what if you don’t want to limit the length of calls for a specific trunk?

Well, FreePBX has a context called [macro-dialout-trunk-predial-hook] which lets you jump in at the very last moment and override any settings you like, which is perfect for this sort of thing.

I got the idea from here:

http://www.freepbx.org/book/export/html/5893

I’ve added this to extensions_custom.conf:

[macro-dialout-trunk-predial-hook]
 exten => s,1,NoOp(Trunk ${OUT_${DIAL_TRUNK}} selected)
 exten => s,n,Gotoif($["${OUT_${DIAL_TRUNK}}" != "SIP/TRUNK_NAME"]?skip)
 exten => s,n,NoOp(Setting DIAL_TRUNK_OPTIONS to Ww)
 exten => s,n,Set(DIAL_TRUNK_OPTIONS="Ww")
 exten => s,n(skip),MacroExit()

Which tests the name of the trunk which is to be used and explicitly sets DIAL_TRUNK_OPTIONS if it matches, otherwise nothing happens. This is a very powerful feature and my extremely crude hack doesn’t do it justice. I wanted to make a note of this before I forgot.

A better way might be to look at the dial prefix? I’ll investigate that option, but for now that will have to do.

g729 in Asterisk

I run Asterisk 1.8 at home for my own amusement.  I’ve just got a SIP trunk running again to SIPGate which had stopped running for some reason.  I’d set the DID for the SIPGate number to check for faxes and then, if it’s not a fax, go to my MOH application, but for some reason it didn’t work.

I also couldn’t access voicemail from the SIPGate trunk either, but I could get it to work with a phone connected to an ATA.

I had a look in the logs:

channel.c: Unable to find a codec translation path from 0x100 (g729) to 0x40 (slin)

Since slin is, I think, the Asterisk native format this is quite bad.  My assumption is that SIPGate sends through G.729 encoded audio regardless of whether you want it to or not (as configured in your allow/disallow lines for the SIP trunk PEER).

G729 from SIPGate works to the ATA because the ATA supports G729 and G729 to G729 pass-through works without any extra requirements from Asterisk.

So – in order to get SIPGate to Asterisk apps working again I installed the G729 codec binaries from here:

http://asterisk.hosting.lv/

and restarted Asterisk.  Now a:

         core show translations

shows I can convert between G729 and loads of other codecs.

Asterisk set up in the UK with SMS

EDIT: 25 Sept 2010
The below specifically refers to Asterisk 1.4. Go to the bottom for an update on 1.6

Why do I always get myself in to these situations?

Many years ago I knocked up a set of scripts to record TV programs from a DVB-T card.  Getting the TV card working was really hard work.  I had to hack about with the driver source code, compile custom kernels, build endless versions of the driver, and so on and so on and so on.  It was a pain in the arse, which was compounded by the fact that at the time very few other people (end users) were trying to get these cards working, so there wasn’t much community support.  My scripts were great and everything and I could record TV, taxes were low and life was, on the whole, pretty good.

Then Stuart put me on to MythTV.  It quickly became evident that although my scripts were functional they were nothing compared to Myth.  So I moved over.  Again with the pain.

The basics were there but DVB support was in it’s infancy and it was a lot of hard work getting it working.  I persevered and learnt a lot about MythTV in the process.  Yay me.  At the beginning MythTV was a foreign country.  By the time I’d got it working the way I wanted it was my home town.

And so it will be with Asterisk, I hope.

Things I have learnt about Asterisk

  1. The cheap FXO cards they sell on eBay are more trouble than they are worth.
  2. The documentation on the voip-info site is often out of date.
  3. UK Caller ID works with the TDM400’s out of the box.
  4. Sending & receiving SMS’s does work, but it’s a bitch, and unreliable.
  5. The logic in Asterisks’ dial plan (extensions.conf et al) is illogical to me
  6. FreePBX is both a blessing and a curse.
  7. I love it.

In a bit more detail then:

1.  The FXO cards you can get from eBay for about 20 quid work perfectly well.  For making calls.  But, there are a few drawbacks.  They don’t support polarity reversal which is the method by which incoming calls and Caller ID are announced to Asterisk.  You’ll still get ringing indications and be able to make calls, but you won’t get Caller ID.  Or rather, you can get Caller ID but you need to patch the Zaptel drivers and the Asterisk source.  The patches are old and don’t apply perfectly.  They are also seemingly unsupported now, in that no one cares if you are having problems with them.  I got the patches applied but Asterisk wouldn’t compile and I don’t know/care enough to fix it.  I was in the market for an ATA anyway so I put that 40 quid towards the cost of the TDM400p11 from NovaVox and if I hadn’t bought the AX-100 then the initial outlay for a fully supported card isn’t so bad.  It also works out cheaper to add another FXS port the to TDM400 than it does to buy another ATA.  The AX-100 also has a US spec. “hybrid” in it which means that the unbalanced UK spec phone lines and the AX-100 are not perfectly matched which can cause wicked echo.  The TDM400 has a tunable hybrid, and that’s a good thing.  All that said – for 20quid I still think it’s worth getting one to play with.  Ask me about doing a swap, if you’ve got any cool toys you don’t want  (I’ve got a DVB-T card going spare as well!). Find the best toy cars for toddlers that are also suitable for kids, 10 and up, here at top 9 rated. I’m mostly looking for more senior level stuff.

2.  The VOIP-info.org site is a really good source of data.  The website might be ugly, but there is a lot of Asterisk data on there.  Unfortunately there is little in the way of information, and some of the data is way out of date.  It acts as a good reference point but isn’t for the beginner because most pages assume a lot of prior knowledge.  Sure – I could go in and edit the Wiki, but I don’t know if what I’m doing is the best way of doing it, or even the correct way to do it.  So for now that’s what this blog post is for.

3.  BT send the Caller ID information before the first ring.  The process goes something like this:

Line polarity reversed -> Caller ID sent as FSK (sort of modem tones) -> Phone starts to ring -> Call Connected -> Charlie Brown's Teacher -> Call Hung Up -> Line polarity reversed

You can see the importance of being able to detect polarity reversal in the call set-up and tear-down.  The AX100 simply doesn’t do it.  The TDM400 does, and it’s fully supported out of the box, as is UK style Caller ID, where the ID is sent before the first ring.  The hack for the AX100 keeps a buffer running all the time and then once a call comes in it looks in this buffer for the caller ID FSK and decodes it.  Shonky, I think you’ll agree.  My advice is that if you want to do caller ID in the UK buy a TDM400.  If you are a masochist then feel free to try with the AX100, I’d love to read your HowTo once you’ve got it working.

4.  SMS.  What a bitch.  I recently discovered my subscription with BT gives me 200 free SMS text messages a month.  So to make sure I was getting best value from the phone subscription I went out and spent 100quid on a PCI card and then spent hours and hours and hours fiddling about trying to get it working.  I don’t know why, I probably send about 3 text’s a month from my mobile, the whole “Because it’s there” thing I guess.  A pox on you, Mallory.

Last night I finally got it working semi-reliably for incoming and outgoing texts.  I can send and receive from my mobile, but Stuart doesn’t receive my messages.  Ho hum.

Firstly, receiving SMS’s. Now, I don’t really understand the extensions.conf language.  It’s all a bit, well, wrong, to my eyes.  All I can tell you is that this is how I got it working.  NOTE:  This WILL NOT work for you if you just copy and paste. Sorry about that.  Perhaps someone can help me re-write it so that it does?

I’m using FreePBX as a GUI and it rewrites some of the Asterisk config files for you.  I’m not sure if the context [from-pstn] is one that I created, or a standard one.  What you need to find out is what route incoming calls from the PSTN take i.e. which contexts they pass through before they start ringing on your internal phones.  You can do this my running the Asterisk console and turning verbosity up to about 6 (core set verbose 6) and then calling in from outside.  You should see a trace of what contexts get called as the call comes in.

For example, my call flow goes something like this:

[from-zaptel] -> [from-pstn] -> [macro-user-callerid] -> 600@[ext-group]

Where 600 is an extensions number I set up with FreePBX to ring all internal phones.

I chose to test for SMS’s in the [from-pstn] context. You can visualise what the [from-pstn] context does when a call comes in by using the Asterisk CLI.  Use the command “dialplan show s@from-pstn” to see the instructions that would be followed when [from-pstn] gets called at position “s”, or start.

Originally, mine went something like this:

[ Included context 'ext-did-0001' created by 'pbx_config' ]
's' =>            1. Set(__FROM_DID=${EXTEN})                   [pbx_config]
2. Gosub(cidlookup|cidlookup_1|1)             [pbx_config]
3. ExecIf($[ "${CALLERID(name)}" = "" ] |Set|CALLERID(name)=${CALLERID(num)}) [pbx_config]
4. Set(__CALLINGPRES_SV=${CALLINGPRES_${CALLINGPRES}}) [pbx_config]
5. SetCallerPres(allowed_not_screened)        [pbx_config]
6. Goto(ext-group|600|1)                      [pbx_config]

You can follow the call progression through this quite easily.  We jump in at position (or priority) 1, we set a variable, then we jump to some cidlookup sub-routine, then if the CALLERID(name) is blank we set it to be the same as the caller id number, then something about CALLINGPRES, not sure what that does, ditto the next line, and then we go to ext-group|600|1.  Ahhaaa!  I know that 600 is the group that I call to ring all the phones, so that must be where the call is handed off to other contexts or functions that let you do talking to people.  Since an incoming SMS doesn’t need to get as far as ringing on a phone, it makes sense to me to interrupt the call flow during [from-pstn].

So I added [from-pstn-custom] to /etc/asterisk/extensions_custom.conf like this:

[from-pstn-custom]
exten => s,3,GotoIf($["${CALLERID(num)}" = "08005875290"]?will-sms,s,1)
exten => s,4,Verbose("Not an incoming SMS")
exten => s,5,Gosub(cidlookup,cidlookup_1,1)
exten => s,n,ExecIf($[ "${CALLERID(name)}" = "" ] ,Set,CALLERID(name)=${CALLERID(num)})
exten => s,n,Set(__CALLINGPRES_SV=${CALLINGPRES_${CALLINGPRES}})
exten => s,n,SetCallerPres(allowed_not_screened)
exten => s,n,Goto(ext-group,600,1)

[will-sms]
exten => s,1,Verbose(=============Entered Will SMS)
exten => s,n,Answer()
exten => s,n,Wait(2)
exten => s,n,SMS(default|a)
exten => s,n,Verbose(=============Done with Will SMS)
exten => s,n,Hangup(16)

How does it work?  The [from-pstn-custom] overrides the [from-pstn] from the main extensions.conf file provided by FreePBX and adds a line that branches to a context called [will-sms] which then uses the SMS application to receive the SMS and then hangup the phone.  (Actually, I think SMS hangs up for you.  But, you know, whatever)

In [from-pstn] in extensions.conf is a line that “includes” the config from “from-pstn-custom”.  By virtue of being imported, the instructions in [from-pstn-custom] take precedence over the other config, because it is imported first.  The config files are read line by line in the order they appear in the file.  But, the imported instructions do not completely replace the config that is already there.  They are sort of merged. That’s why I’ve copied a load of bits from the original call flow in to my new [from-pstn-custom].  If I left them out, then some bits of my new context would be applied and some wouldn’t.  The way I got it to work was to replicate the call flow to the point of it branching off to [ext-group] in my new context.  If you do a “dialplan show s@from-pstn” now, you see this:

[ Included context 'from-pstn-custom' created by 'pbx_config' ]
's' =>            3. GotoIf($["${CALLERID(num)}" = "08005875290"]?will-sms|s|1) [pbx_config]
4. Verbose("Not an incoming SMS")             [pbx_config]
6. Gosub(cidlookup|cidlookup_1|1)             [pbx_config]
7. ExecIf($[ "${CALLERID(name)}" = "" ] |Set|CALLERID(name)=${CALLERID(num)}) [pbx_config]
[pbx_config]
9. Set(__CALLINGPRES_SV=${CALLINGPRES_${CALLINGPRES}}) [pbx_config]
10. SetCallerPres(allowed_not_screened)       [pbx_config]
11. Goto(ext-group|600|1)                     [pbx_config]

[ Included context 'ext-did-0001' created by 'pbx_config' ]
's' =>            1. Set(__FROM_DID=${EXTEN})                   [pbx_config]
2. Gosub(cidlookup|cidlookup_1|1)             [pbx_config]
3. ExecIf($[ "${CALLERID(name)}" = "" ] |Set|CALLERID(name)=${CALLERID(num)}) [pbx_config]
4. Set(__CALLINGPRES_SV=${CALLINGPRES_${CALLINGPRES}}) [pbx_config]
5. SetCallerPres(allowed_not_screened)        [pbx_config]
6. Goto(ext-group|600|1)                      [pbx_config]

[ Included context 'ext-did-catchall' created by 'pbx_config' ]
'_.' =>           1. Noop(Catch-All DID Match - Found ${EXTEN} - You probably want a DID for this.) [pbx_config]
2. Goto(ext-did|s|1)                          [pbx_config]

You can see my new [from-pstn-custom] context at the start, along with the copied commands from ext-did-001 to ensure that they run in the way I expected.  The upshot of this is that I now branch to my new context [will-sms] when the incoming caller ID is the BT 0800 number from where SMS’s originate.  You should note that the 0800 number can change if you have more than one incoming SMS box registered, so don’t do that.  Just register one incoming address  (more on this later).

The [will-sms] is pretty straight forward.  Answer the phone, wait 2 seconds, and then start the SMS application which does all the clever noises.  The key here is Wait.  Without it SMS reception is intermittent at best.  The SMS(default|a) tells the SMS app to receive in to the default queue (this works, don’t understand fully why) and to do an “a” for answer.  I had a load of problems with the example from VOIP-info where it passes in the extension number, which is “s” for start.  If the SMS application sees an “s” it thinks it means “send”.  That won’t work!

If you manage to bodge all this in to your dial plan then you should find your incoming SMS’s in /var/spool/asterisk/sms/mtrx

Make sure that /var/spool/asterisk is writeable by the same user as Asterisk is running as.

Sending SMS’s. Was a whole bunch of no fun as well.  Especially if you don’t understand Asterisk properly.

The command line I’m using to send an SMS is:

smsq --motx-channel=Zap/2/17094009 -d <phone number>  -m "<message>"

That works because zap channel 2 is my PSTN line.  You might need to change this.  17094009 is the SMSC centre number for BT to accept SMS messages for delivery to other networks.

When I was trying to get sending working, which I actually did first, I couldn’t reliably get Asterisk to talk to the SMSC.  Before I had the receive working properly the BT system would ring me up and I’d answer the phone, then not hearing a carrier the BT system would hang up again.  This gave me an idea, if I could emulate the carrier then I could hear what the BT system was sending down the line.  I sent an SMS to the home phone, it rang in on one of the extensions and I whistled at it.  Because I am just so 1337 I hit the right tone and I could hear FSK noises coming down the line, but they were really really quiet.  I tweaked the gain up a little bit in the /etc/asterisk/zapata.conf file:

rxgain=2
txgain=0

and tried again.  It was louder!  Sending a test message to “00000” with the text of “test”  causes the BT system to send a message back to you  (you need to send “register” to 00000 before BT will send you messages as text instead of reading them to you by the way – but if it can’t successfully deliver the message back to you to say it has received your “register” message it will ignore the request and you’ll keep getting the messages read to you by a friendly robot – hence the need to get receiving working first).  It sent!  I saw some hex coming from the SMS application in the Asterisk log.  I sent a text to my mobile, and it arrived.  Yay!  problem solved.

5.  Asterisk’s dial plan.  Bonkers.  I still can’t make sense of them.  Oh well, I’ll get there in the end.

6.  FreePBX is brilliant if you want to get Asterisk configured with some very complex applications (think voicemail) in a matter of minutes.  But, if it makes editing the config files by hand a bit more complicated because you have to use xxxx_custom files.  Thus, all the replication of code when as above when you don’t understand how it all works.

7.  Check out my hold music hacked together from various online sources, and no doubt subject to various copyright restrictions.  I’ve compressed the hell out of it, if you want a better quality version for your own amusement let me know.

Hold Music

I crack myself up, I really do.

Also – think of all the cool things you could do if you could phone or text your computer and get it to do things for you.

Now I’m off to write myself an gmail <-> SMS gateway doodad.

Ok – with Asterisk 1.6 the above is slightly less relevant.  Try this in your dialplan instead:

[will-sms]
exten => s,1,Verbose(=============Entered Will SMS)
exten => s,n,Answer()
exten => s,n,SMS(default,ap(1500))
exten => s,n,Verbose(=============Done with Will SMS)
exten => s,n,Hangup(16)

The SMS application has the option to add a pause which I've set to 1500 ms.  Seems to work.