George MacKerron: code blog

Updated May 2012 for Lion

The secret to getting the MySQL gem to install and function with Ruby 1.9.x on Snow Leopard or Lion is:

• Install MySQL using the 64-bit .DMG package installer from dev.mysql.com
• Install Ruby using RVM or (preferably) rbenv
• Add these to lines to ~/.bash_login or ~/.bashrc or ~/.profile as appropriate:

export PATH="/usr/local/mysql/bin:$PATH"
export DYLD_LIBRARY_PATH="/usr/local/mysql/lib:$DYLD_LIBRARY_PATH"

• In a new shell (Terminal window), type gem install mysql as normal.

I’m posting this mainly as a record for myself, having wasted a lot of time in the past trying strange incantations from comments on various other blogs posts.

The recent Gawker passwords leak once again highlights the widespread use of passwords that offer essentially no security.

Some years ago, when working on a secure web app for a large organisation — let’s call them Secret Testing Ltd — I was keen that people shouldn’t choose hopelessly weak passwords. I was particularly concerned by my sysadmin colleague’s fondness for passwords of the form ‘p/\55w0rd’ or ‘S3cr3t-T35t|ng’.

Read the rest of this entry »

My current iPhone app/research project, mappiness, is heavily reliant on Apple’s Push Notification Service (APNS).

We’re now sending about one million notifications a month, and it’s working beautifully. The obvious alternative — using SMS messages — would be costing us tens of thousands of pounds a month.

If you’re using or contemplating using Push Notifications, you may find these four points of interest:

1. Urban Airship offer a well-though-out and reliable service. But if you’re using them just as an intermediary in telling the APNS to beep users X, Y and Z right now — not making use of their scheduling or broadcast features, for example — it might well not be worth it. You’ll cut out a layer of indirection, and save money if you get big enough for it to matter, by talking to Apple directly.
2. There’s a surprising dearth of viable-looking open-source options for interfacing with the APNS. (APNS uses a binary protocol over a secure connection, and doesn’t like this connection torn down and reopened too often, so you can’t easily interface with it directly from a web app or cron job). But there is pyapns, an XML-RPC-speaking APNS provider built on Python’s Twisted networking framework, with client libraries for Python and Ruby. I’ve found this straightforward and rock-solid.
3. The APNS feedback service sometimes reports device tokens as inactive for no obvious reason, so make sure you’re prepared to hear from and reactivate users you’ve previously inactivated. (mappiness got bitten by this early on: a few dozen people were unable to authenticate to our server because they’d been reported as inactive and we’d assumed they were never coming back, despite that fact they had the app installed and Push Notifications switched on).
4. The app delegate method application:didRegisterForRemoteNotificationsWithDeviceToken: seems occasionally to return phantom zero-length device tokens — so don’t let your app or server get flummoxed by these. (We suffered from this on mappiness too: the bad device token got added to the front of a queue for sending from the app to our server; the server wouldn’t accept it; and this blocked all further data getting sent back until we fixed the server to silently ignore the bad tokens).

Ordnance Survey’s Code-Point with Polygons “provides a precise geographical location for each postcode unit in Great Britain”. It’s available in various formats, including ESRI .shp files.

Many UK academics can access the data via institutional subscription to EDINA Digimap. I’m using it in my research into subjective wellbeing and environmental quality.

This post shows how to:

1. import the data files into a PostGIS database; and
2. de-normalise the data into a single table, where there’s a one-to-one mapping of postcodes to rows, and each row contains either all geographical locations covered by a postcode (as a single geometry column, of type multipolygon) or the reason why no such location is available

Read the rest of this entry »

I have a simple site that generates covers for CDs I burn from iTunes purchases and so on (it pre-dates widespread use of JS libraries, and is in much need of prettifying). The site uses Amazon Product Advertising API calls to search and retrieve album cover art and track listings. Since earlier this month, such API calls have to be cryptographically signed.

This is somewhat annoying — the site’s original design has it communicating independently with Amazon (using Amazon’s XSLT API feature to transform their XML data into JSON), and that’s no longer possible with the use of a private key. But it’s not unfixable. The site now sends its API call first to my server, which returns a signed version, and then forwards the signed call on to Amazon.

I found most of what I needed for this on Chris Roos’ blog, but his version still wasn’t quite working for me (the two problems I recall are that Ruby’s CGI.escape doesn’t quite follow Amazon’s requirements, and that times need converting to GMT).

Read the rest of this entry »