HOWTO: Add OAUTH to your Alexa Smart Home skill in 10 minutes

Alexa smart home skills require you to provide OAUTH2 so that users can authorise a skill to access the assumed cloud service powering their lightbulbs or any number of other pointlessly connected devices.  This makes sense since OAUTH2 is a standard and secure way to grant access for users from one system to the resources of another.  However, with this come a few caveats which are potential blockers for casual skill developers like me.  If you’re writing a skill for your own personal use, with no intention of adding it to the store, you still have to have a valid and recognised SSL certificate and a whole OAUTH2 server set up somewhere.

The SSL certificate is easy enough to implement, but it’s a bit of a faff (renewing Let’s Encrypt certs, or paying for cert which needs you to deal with the certificate authorities, send in scans of your passport and other tedious red tape) but – in my opinion anyway – setting up an OAUTH server is even more of a faff.  If only there was some way to avoid having to do either of these things….

Using “Login With Amazon” as your OAUTH provider

Since you already have an Amazon account you can use “Login With Amazon” as your skill’s OAUTH server and your normal everyday Amazon account as your credentials.  You’re only sharing your Amazon account data with yourself, and even then we can restrict it to just your login ID.  You don’t actually need to do anything with the OAUTH token once it’s returned since you’re the only user.  I mean, you could if you wanted to, but this HOWTO assumes that you’re the only user and that you don’t care about that sort of thing.  We are also going to assume that you have already created the Lambda function and the smart home skill or are familiar with how to do that.  This is a bit tricky because you can’t test your smart home skill on a real device until you’ve implemented OAUTH, and you can’t complete the OAUTH set-up until you’ve got the IDs from your Lambda function and skill.  If you haven’t written your skill yet, just create a placeholder Lambda function and smart home skill to be going on with.

Much of this information is available from the official Amazon instructions available here: https://developer.amazon.com/public/community/post/Tx3CX1ETRZZ2NPC/Alexa-Account-Linking-5-Steps-to-Seamlessly-Link-Your-Alexa-Skill-with-Login-wit. What follows is a rehash and slight reorganisation of that doc which is hopefully a bit easier to follow.

1. Create a new Login With Amazon Security Profile

From the Developer Console in AWS, go to Apps & Services -> Login With Amazon.  Or click https://developer.amazon.com/lwa/sp/overview.html

Click “Create a New Security Profile”.  Fill out the form along these lines:

screenshot-from-2016-11-30-13-40-23

and hit Save.

You should see a message along the lines of “Login with Amazon successfully enabled for Security Profile.”

Hover the mouse over the cog icon to the right of your new security profile and choose “Security Profile”.

Copy your “Client ID”  and “Client Secret” and paste it in to a notepad.  You’ll need this again shortly.

clientid2

2. Configure your skill to use Login With Amazon

Back in the Developer Console, navigate to the Configuration page for your skill.  (Click on your skill, then click on Configuration).  You need to enable “Account Linking” and this will then show the extra boxes discussed below.

In to the “Authorization URL” box you should put:

https://www.amazon.com/ap/oa/?redirect_url=

and then copy the Redirect URL from further down the page and append it to the end of the Authorization URL.  For example:

https://www.amazon.com/ap/oa/?redirect_url=https://layla.amazon.com/api/skill/link/1234ABCD1234AB

authurl

As far as I can tell Layla is for UK/Europe and Pitangui is for the US.  Use the appropriate one for you.  Also, keep a note of the redirect URL in your notepad, you will need this again later.

In to the “Client Id” box paste your client id from step 1.

You can leave “Domain List” blank for now.

For “Scope” I suggest you use:

profile:user_id

This will give your Alexa Skill access to a minimal amount of information about you from Amazon, in this case just your user_id.  Since you don’t really have any customers for your skill, only you, there is no reason to provide access to any other information.

Further down the page you need to configure the Grant Type:

granttype

Select an “Auth Code Grant

Set the “Access Token URI” to:

https://api.amazon.com/auth/o2/token

and in to “Client Secret” paste your secret from step 1.

You must include a link to your “Privacy Policy URL“.  Since you are the only person who cares you could host a blank file somewhere, or maybe link to a Rick Astley video on YouTube?

Finally hit Save.

3. Link Login With Amazon back to your Skill

Head back to the Login With Amazon page: https://developer.amazon.com/lwa/sp/overview.html

Hover over the cog of your Security Profile and choose Web Settings:

returnurl

In to the “Allowed Return URLs” box paste your Redirect URL from step 2 and hit save.

4.  Login to Amazon from your skill and do the OAUTH dance

From the Alexa app on your phone navigate to your new Smart Home Skill and you see that it says “Account Linking Required“.

img_0473

Click “Enable Skill” and you’ll be asked to login with your Amazon credentials:

img_0474

Once you log in you should see a success message:

img_0475

And you’re done.

 

bravialib – a Python library to abstract the Bravia web API

I posted this to Github, but thought I would mirror it here too. You can download the code from here:

https://github.com/8none1/bravialib

 

bravialib

A Python library for talking to some Sony Bravia TVs, and an accompanying Alexa Skill to let you control the TV by voice. If you like that sort of thing.

These scripts make use of the excellent Requests module. You’ll need to install that first.

bravialib itself

This is a fairly simple library which allows you to “pair” the script with the TV and will then send cookie-authenticated requests to the TVs own web API to control pretty much everything. You can:

  • Set up the initial pairing between the script and the TV
  • Retrieve system information from the TV (serial, model, mac addr etc)
  • Enumerate the available TV inputs (HDMI1,2,3,4 etc)
  • Switch to a given input
  • Enumerate the remote control buttons
  • Virtually press those buttons
  • Find out what Smart TV apps are available
  • Start those apps
  • Enumerate the available DVB-T channels
  • Switch to those channels
  • Send Wake On Lan packets to switch the TV on from cold (assuming you’ve enabled that)
  • A couple of convenience functions

The library tries to hide the complexity and pre-requisites and give you an easy to use API.

I built this for a couple of reasons: 1. Because the TV had an undocumented API, and that tickles me 2. I quite fancied hooking it up to Alexa for lols

Most of the information about how to talk to the TV’s API came from looking at packet captures from the iPhone app “TV Sideview”.

There is a script called testit.py that will give you a few clues about how to use it, but it’s a bit of a mess. I’ve left a lot of comments in the code for the library which should help you.

Really, I think that this library should be imported in to a long-running process rather than be called every time you want to press a remote control button. On a Raspberry Pi, Requests can take a while (a couple of seconds) to import, and then bravialib pre-populates a few data sources, and all of that takes time, like about 20 seconds – so you really don’t want to use this library if you just want to fire a few remote control commands. Also – be aware that the TV takes a long time to boot and accept commands. From cold you’re talking about a minute maybe two, it’s really annoying.

The aforementioned long running process – bravia_rest.py

As the main library takes a while to start and needs a certain amount of data from the TV to work properly it really makes sense to start it up once and then leave it running as long as you can. The bravia_rest.py script does exactly that, and also exposes some of the functionality as a very crude REST interface that you can easily hook it in to various home automation systems.

First you need to add the IP address and MAC address (needed to turn on the TV the first time the script is run, it can be discovered automatically if you just power the TV on for a few minutes before you run the script).

Then run bravia_rest.py.

If this is the first time you have run it you will need to pair with the TV. You will be told to point your browser at the IP address of the machine where the script is running on port 8090 (by default). Doing this will make the script attempt to pair with the TV. If it works you will see a PIN number on the TV screen, you will need to enter this in to the box in your browser. After a few seconds, and with a bit of luck, pairing will now complete. This shouldn’t take too long.

If you are now paired, in your browser go to /dumpinfo for a view in to what the script knows about the TV.

Once everything is running you can POST to these URLs for things to happen (no body is required):

  • /set/power/[on|off] – turns the telly on and off
  • /set/send/<button> – e.g. mute, play, pause, up, down. See dumpinfo for all the key names.
  • /set/volumeup/3 – turn the volume up 3 notches. You MUST pass a number, even it it’s just 1.
  • /set/volumedown/1 – as above.
  • /set/loadapp/<app name> – e.g. Netflix, iplayer. Again /dumpinfo will show you what apps are available.
  • /set/channel/<channel> – e.g. BBC ONE, BBC TWO
  • /set/input/<input label> – You need to have given your inputs labels on the TV, then pass the label here.

Hooking it up to Alexa

Now we can poke the TV through a simplified REST interface, it’s much easier to hook in to other things, like Alexa for example. Setting up a custom skill in AWS/Lambda is beyond the scope of what I can write up at lunchtime, I’m sure there are lots of other people who have done it better than I could. You’ll need to create a custom app and upload a Python Deployment Package to Lambda including my lambda_function.py script (see inside the Alexa directory), a secrets.py file with your info in it, a copy of the Requests library (you need to create a Python Virtual Environment – it’s quite easy) and possibly a copy of your PEM for a self signed HTTPS certificate. I’ve also included the skills data such as the utterances that I’m using. These will need to be adjusted for your locale.

You can read more about Python deployment packages and AWS here:http://docs.aws.amazon.com/lambda/latest/dg/lambda-python-how-to-create-deployment-package.htmlhttp://docs.aws.amazon.com/lambda/latest/dg/with-s3-example-deployment-pkg.html#with-s3-example-deployment-pkg-python

Here’s how it works:

block diagram

  1. You issue the command to Alexa: Alexa tell The TV to change to channel BBC ONE.
  2. Your voice is sent to AWS (the green lines) decoded and the utterances and intents etc are sent to the Lambda script.
  3. The Lambda script works out what the requested actions are and sends them back out (the orange lines) to a web server running in your home (in my case a Raspberry Pi running Apache and the bravia_proxy.py script). You need to make that Apache server accessible to the outside world so that AWS can POST data to it. I would recommend that you configure Apache to use SSL and you put at least BASIC Auth in front of the proxy script.
  4. The bravia_proxy.py script receives the POSTed form from AWS and in turn POSTs to the bravia_rest.py script having done a quick bit of sanity checking and normalisation. The proxy and the rest scripts could live on different hosts (and probably should, there are no security considerations in either script – so ya know, don’t use them.)
  5. bravia_rest.py uses bravialib to poke the TV in the right way and returns back a yes or a no which then flows back (the blue lines) to AWS and your Lambda function.
  6. If everything worked you should hear “OK” from Alexa and your TV should do what you told it.

I could have put bravia_rest.py straight on the web an implemented some basic auth and SSL there – but I think this is something better handled by Apache (or whichever server you prefer), not some hacked up script that I wrote.

Caveats:

  • It doesn’t deal with the TV being off at all well at the moment.
  • I don’t know what happens when the cookies expire.
  • I haven’t done much testing.
  • I have no idea what I’m doing.

DHCP clients not registering hostnames in DNS automatically

To remind myself as much as anything:

I run a dnsmasq server on my router (which is a Raspberry Pi 2) to handle local DNS, DNS proxying and DHCP. For some reason one of the hosts stopped registering its hostname with the DHCP server, and so I couldn’t resolve its name to an IP address from other clients on my network.

I’m pretty sure it used to work, and I’m also pretty sure I didn’t change anything – so why did it suddenly stop? My theory is that the disk on the client became corrupt and a fsck fix removed some files.

Anyway, the cause is that the DHCP client didn’t know to send it’s hostname along with the DHCP request.

This is fixed by creating (or editing) /etc/dhcp/dhclient.conf and adding this line:

send host-name = gethostname();

 

Hacking 433Mhz support into a cheap Carbon Monoxide detector

Skill level:  Easy

My home automation systems use two mechanisms for communication:  Ethernet (both wired and wireless) and 433MHz OOK radio.

433MHz transmitters are readily available and are cheap but unreliable.  Wifi enabled MCUs such as the ESP8266 are also cheap (coming in at around the same cost as an Arduino clone, a 433MHz transmitter and a bag of bits to connect them together), they are reliable enough but extremely power hungry.  If I can plug a project into the mains then I’ll use an ESP8266 and a mobile phone charger for power, if the project needs to run off batteries then a 433MHz equipped Arduino is the way I’ve gone.

Like most people playing with 433MHz radio I found reliability and range of the radio link to be super flaky.  I’ve finally got a more-or-less reliable set-up:

  • A full wave dipole antenna at the receiver
  • A high quality receiver from RF Solutions in place of the cheap ones which are bundled with transmitters. A decent receiver on eBay
  • A big capacitor on the transmitter.  I saw the frequency and amplitude drifting massively during transmission.  Adding a 470µF cap helps.  Allow time for the cap to charge and the oscillator to stabilise, a few seconds delay seemed to do the trick.
  • Using the RCSwitch library on the transmitter:
    • RCSwitch mySwitch = RCSwitch();
    • mySwitch.setProtocol(2); // Much longer pulse lengths = much better range?
    • mySwitch.setRepeatTransmit(20); // Just brute-force it!

With this setup I can get receive a 24bit number from an Arduino running off 2 AA batteries and a coiled 1/2 wave antenna from about 5 meters indoors through walls.  That’s still poor, but it does the job.  Increasing the voltage to the transmitter would probably help.

Once you have a reliable 433MHz receiver setup then you can also buy off the shelf 433MHz enabled home automation gizmos like this smoke alarm or these door sensors.  They have a set of jumpers inside where you can set an ID, which is essentially the same 24bit number that RCSwitch lets you transmit.  For what it’s worth I also have kite-marked smoke detectors in my house, but from the testing I’ve done with a bit of smoldering paper the cheap imports work just fine.

I couldn’t find a cheap Carbon Monoxide which also has 433MHz support so I thought I’d quickly hack one together out of this Carbon Monoxide detector and an Arduino clone and 433MHz radio:

CO Alarm inside

 

 

 

 

 

 

 

 

 

IMG_1238

You can barely notice it!

 

 

 

 

 

 

 

It’s certainly untidy, but it does the job.  If I had PCB facilities at home I’m fairly sure it could be made to fit inside the alarm, along with some more holes in the case for ventilation.

The premise is simple enough.  The Arduino is powered by the 3v3 regulator on the CO alarm PCB.  The cathode of the red alarm LED is connected to pin 2 of the Arduino as an external interrupt.  When the pin goes low the Arduino wakes up and sends it’s 24bit ID number over the radio which is picked up by the receiver which sends an SMS alert, switches the boiler off, etc.  I’ve connected the radio transmitter to directly to the 3 x AA batteries (4.5 volts) via a transistor which is switched by a pin on the Arduino.  In standy-by mode the additional equipment draws a fraction of a milliamp and so I’m not worried about draining the batteries faster.

As with the smoke alarms, this is not my only source of Carbon Monoxide detection.  I’ve yet to test it’s sensitivity.  This is considered to be a “well, if it works, and it turns the boiler off automatically then it’s certainly worth a go, but I’m not relying on it” project.

HOWTO: Very low power usage on Pro Mini V2 (Arduino clone)

Skill level:  Easy enough if you’ve got a soldering iron.

Introduction

The Pro Mini V2 is an Arduino Pro Mini clone available on eBay for, typically, £1.50.  The version I buy is adjustable between 5v and 3.3v and has an ATmega 328 clocked at 8 MHz.  It’s an ideal board for development of IoT remote sensors and great for playing with and learning about the Arduino development environment.

Here’s a link to the version I buy and know works: 3V Pro Mini 2 Arduino Clone

When you want to put a sensor in a remote location the last thing you want to do is have to run a power cable to it. I’ve experimented with solar with generally poor results so battery operation is the obvious solution.  While Li-ion batteries offer higher energy density the sweet spot still seems to be the good ol’ alkaline battery.  They’re cheap, safe, recyclable and readily available.

For what it’s worth the Ikea alkaline batteries offer good value:  http://www.batteryshowdown.com/results-lo.html (I suggest buying as many packets as you can carry, so that you never ever have to go back there.  Unless you like arguing with your wife of course.)

Power Usage

The ATmega 328 has various power saving functions which involve putting it to sleep when not doing anything.  I use the Rocket Scream Low Power library to take care of putting the processor into a low power state, but I wasn’t seeing anything like the low power savings they detail on their site.

Some quick calculations:  Let’s assume a AA battery provides 2000 mAh.  I measured my Pro Mini V2 as drawing 6.7mA when powered up and doing things and 2.8mA when in sleep mode.  As a conservative estimate, let’s say it’s running for 1 hour in every 24 hour period and asleep the rest of the time.  That averages out to about 3mAh of draw.

For a 2000 mAh battery, that would give about 667 hours of runtime, or 28 days. So a standard Pro Mini V2 could run for about a month on a pair of AA batteries.  Not bad, but I think changing the batteries every month is still going to be a bit of a drag. Besides, Rocket Scream are seeing power usages in the micro-amps range when asleep.  There is clearly work to do.

How to dramatically reduce the power consumption

In this photo you can see I’ve identified some sections which are related to the power usage of the Pro Mini.

arduino_highlightsThe red section is the power LED.  This is always on when power is applied and sucks about 0.2mA when lit. If you don’t need this to be lit all the time then you can easily remove it to save some juice.  I found the easiest way was to use a pair of cutters to snip/crush the middle of the LED and then use a soldering iron to remove the bits left over.

The green section is the on-board regulator.  If you are going to be supplying power to the board via a couple of AA batteries (each battery being 1.5v, so two is 3v) then you don’t need the regulator.  You can cut this off too if you like, but.. keep reading, there’s no need to hack it off.

Saving the best until last, the yellow section is the power-selection jumper to switch between 3.3v and 5v.  It passes the power supplied by the RAW pin through the regulator and on to the board.  The regulator is inherently inefficient.  You might think that you could bypass the on-board regulator by powering the board by apply power to the Vcc pin instead, but it still seems to power the regulator.  By simply unsoldering this jumper you can disable the on-board regulator and save loads of power.  Once removed you will need to apply power to the Vcc pin at ~ 3.3V.  I used some solder-wick to clean up but you could just scrape it off with a soldering iron if you need to.

Here’s one I prepared earlier.

IMG_1029

With the jumper and LED removed. (Red and yellow boxes from previous image)

 

With the LED and solder jumper removed I measured the power usage again.  Running current is now down to 3.8mA, pretty much half of what it was.  But, most impressively the power used when asleep is down to 0.004mA.  4 microamps! Yay!

Some more quick calculations based on the same usage as before:  average power consumption drops to 0.17mA.  That gives us 490 days, 1.3 years run time off a pair of AA batteries.  That should allow for 2.5 minutes “work” an hour.  Waking up, taking some readings and sending them off via a radio should take well under 1 minute, which should allow for more power usage by a radio.

 

 

 

Conclusion

Unless you’re going to run your Arduino off a permanently attached serial connector, then just do this.  Get yourself a couple of AA batteries & a battery holder.  Apply the +ve side of the batteries to Vcc and the -ve to ground.  Stick your multi-meter in between the battery and Vcc pin to measure the lovely low current usage.  You can read the battery power being provided with the Secret Arduino Volt-meter trick.

I’ve got quite a few sensors around the house running with this set-up so I will monitor battery usage over the next few weeks or months and report back.

 

Coming Soon…

A write up of my cheapo IoT sensor network, including smoke detectors, door contact sensors, movement sensors, house-plant watering monitors, room temperature sensors and a weather station.  Plus, build an IoT sensor and buy a sausage roll for less than a fiver.