George MacKerron: code blog

Updated May 2012 for Lion

The secret to getting the MySQL gem to install and function with Ruby 1.9.x on Snow Leopard or Lion is:

• Install MySQL using the 64-bit .DMG package installer from dev.mysql.com
• Install Ruby using RVM or (preferably) rbenv
• Add these to lines to ~/.bash_login or ~/.bashrc or ~/.profile as appropriate:

export PATH="/usr/local/mysql/bin:$PATH"
export DYLD_LIBRARY_PATH="/usr/local/mysql/lib:$DYLD_LIBRARY_PATH"

• In a new shell (Terminal window), type gem install mysql as normal.

I’m posting this mainly as a record for myself, having wasted a lot of time in the past trying strange incantations from comments on various other blogs posts.

Ever noticed how, in Google Earth, marker pins that overlap each other spring apart gracefully when you click them, so you can pick the one you meant?

And ever noticed how, when using the Google Maps API, the exact same thing doesn’t happen?

This code makes Google Maps API version 3 map markers behave in that Google Earth way. Small numbers of markers (up to 8, configurable) spiderfy into a circle. Larger numbers fan out into a (more space-efficient) spiral.

More »

This post from June 2011 was updated in November 2011 and January 2012.

This recipe makes wonderful bread: crusty, open-textured, moist, and beautiful.

It needs no kneading, but this doesn’t imply any sort of trade-off. It’s a recipe for a perfect loaf which happens to be effortless, and an effortless recipe which happens to make a perfect loaf.

It’s based on Jim Lahey’s recipe from the New York Times in 2006 (also the subject of an article and accompanying video, and now a full-length book). But it’s even easier, as there’s no awkward middle stage with linen cloths.

In the eight months since we started using it, we’ve made virtually all our own bread.

More »

Here’s a less generic and slightly different nearest-neighbour function based on Regina’s generic nearest-neighbour function at Boston GIS.

It follows the same basic idea of using series of enlarging search radii to restrict distance calculations to a manageable subset of things-that-might-be-near. The difference is that it uses a geometric progression of sizes (x, x * y, x * y^2, x * y^3, ...) instead of an arithmetic one (x, x + y, x + 2y, x + 3y, ...).

For some distributions of things-that-might-be-near, and tuned with the right parameters (x, y), this turns out substantially faster (I’ve used it to locate the nearest UK postcode to each mappiness response).

More »

For a recent study two colleagues needed to elicit a cumulative probability distribution function (CDF) from survey respondents.

We decided it would be nice to allow respondents to interact with this CDF after providing some key values, and implemented this in the web browser with <canvas> (falling back on FlashCanvas for IE).

In the process, we implemented three kinds of cubic spline calculation in the ever-wonderful CoffeeScript: natural, clamped and (what we actually needed) monotonic cubic splines.

You can play with some examples below: click-and-drag the round handles, or double-click to enter values directly. Feel free to borrow the code (ideally with attribution) if it’s of use to you.

More »

The recent Gawker passwords leak once again highlights the widespread use of passwords that offer essentially no security.

Some years ago, when working on a secure web app for a large organisation — let’s call them Secret Testing Ltd — I was keen that people shouldn’t choose hopelessly weak passwords. I was particularly concerned by my sysadmin colleague’s fondness for passwords of the form ‘p/\55w0rd’ or ‘S3cr3t-T35t|ng’.

More »

PostGIS has an st_askml function. This turns geometries into fragments of KML, and thus takes you most of the way to easy visualisation of spatial queries using Google Earth. But not all the way: these fragments have then to be assembled into a complete document.

I’ve written some wrapper and aggregate functions to automate this. They’re probably deeply inefficient — I wouldn’t advocate building your next web service on them — but for one-off eyeballing of query results I find them really useful.

The key functions are called as_kmldoc; you could see them as the missing aggregate versions of st_askml.

More »

My current iPhone app/research project, mappiness, is heavily reliant on Apple’s Push Notification Service (APNS).

We’re now sending about one million notifications a month, and it’s working beautifully. The obvious alternative — using SMS messages — would be costing us tens of thousands of pounds a month.

If you’re using or contemplating using Push Notifications, you may find these four points of interest:

1. Urban Airship offer a well-though-out and reliable service. But if you’re using them just as an intermediary in telling the APNS to beep users X, Y and Z right now — not making use of their scheduling or broadcast features, for example — it might well not be worth it. You’ll cut out a layer of indirection, and save money if you get big enough for it to matter, by talking to Apple directly.
2. There’s a surprising dearth of viable-looking open-source options for interfacing with the APNS. (APNS uses a binary protocol over a secure connection, and doesn’t like this connection torn down and reopened too often, so you can’t easily interface with it directly from a web app or cron job). But there is pyapns, an XML-RPC-speaking APNS provider built on Python’s Twisted networking framework, with client libraries for Python and Ruby. I’ve found this straightforward and rock-solid.
3. The APNS feedback service sometimes reports device tokens as inactive for no obvious reason, so make sure you’re prepared to hear from and reactivate users you’ve previously inactivated. (mappiness got bitten by this early on: a few dozen people were unable to authenticate to our server because they’d been reported as inactive and we’d assumed they were never coming back, despite that fact they had the app installed and Push Notifications switched on).
4. The app delegate method application:didRegisterForRemoteNotificationsWithDeviceToken: seems occasionally to return phantom zero-length device tokens — so don’t let your app or server get flummoxed by these. (We suffered from this on mappiness too: the bad device token got added to the front of a queue for sending from the app to our server; the server wouldn’t accept it; and this blocked all further data getting sent back until we fixed the server to silently ignore the bad tokens).

Perl, Ruby, and some other languages support a readable ‘extended’ regular expression syntax, in which literal whitespace is ignored and comments (starting with #) are available. They also support a multi-line mode where the . character matches anything, including a newline.

JavaScript does neither of these: it doesn’t recognise the extended syntax, and its version of multi-line only allows the ^ and $ characters to match the beginnings and ends of lines within a string (it will never allow the . to match a newline).

So I wrote the following function to convert extended and fully-multi-line RegExp source strings to the basic syntax that JavaScript understands.

More »

I recently switched to TextMate for editing Stata .do files: unlike Stata’s built-in editor on the Mac, it has syntax highlighting and other goodies via Timothy Beatty’s bundle (now hosted by Dan Byler).

One thing it doesn’t have, though, is tab completion. Or rather, it didn’t, until now.

More »