Cubic splines in JavaScript (via CoffeeScript)
For a recent study two colleagues needed to elicit a cumulative probability distribution function (CDF) from survey respondents.
We decided it would be nice to allow respondents to interact with this CDF after providing some key values, and implemented this in the web browser with <canvas> (falling back on FlashCanvas for IE).
In the process, we implemented three kinds of cubic spline calculation in the ever-wonderful CoffeeScript: natural, clamped and (what we actually needed) monotonic cubic splines.
You can play with some examples below: click-and-drag the round handles, or double-click to enter values directly. Feel free to borrow the code (ideally with attribution) if it’s of use to you.
Blocking the weakest passwords
The recent Gawker passwords leak once again highlights the widespread use of passwords that offer essentially no security.
Some years ago, when working on a secure web app for a large organisation — let’s call them Secret Testing Ltd — I was keen that people shouldn’t choose hopelessly weak passwords. I was particularly concerned by my sysadmin colleague’s fondness for passwords of the form ‘p/\55w0rd’ or ‘S3cr3t-T35t|ng’.
as_kmldoc: easily visualise PostGIS queries as KML in Google Earth
PostGIS has an st_askml function. This turns geometries into fragments of KML, and thus takes you most of the way to easy visualisation of spatial queries using Google Earth. But not all the way: these fragments have then to be assembled into a complete document.
I’ve written some wrapper and aggregate functions to automate this. They’re probably deeply inefficient — I wouldn’t advocate building your next web service on them — but for one-off eyeballing of query results I find them really useful.
The key functions are called as_kmldoc; you could see them as the missing aggregate versions of st_askml.
Four things I’ve learned using Push Notifications
My current iPhone app/research project, mappiness, is heavily reliant on Apple’s Push Notification Service (APNS).
We’re now sending about one million notifications a month, and it’s working beautifully. The obvious alternative — using SMS messages — would be costing us tens of thousands of pounds a month.
If you’re using or contemplating using Push Notifications, you may find these four points of interest:
- Urban Airship offer a well-though-out and reliable service. But if you’re using them just as an intermediary in telling the APNS to beep users X, Y and Z right now — not making use of their scheduling or broadcast features, for example — it might well not be worth it. You’ll cut out a layer of indirection, and save money if you get big enough for it to matter, by talking to Apple directly.
- There’s a surprising dearth of viable-looking open-source options for interfacing with the APNS. (APNS uses a binary protocol over a secure connection, and doesn’t like this connection torn down and reopened too often, so you can’t easily interface with it directly from a web app or cron job). But there is pyapns, an XML-RPC-speaking APNS provider built on Python’s Twisted networking framework, with client libraries for Python and Ruby. I’ve found this straightforward and rock-solid.
- The APNS feedback service sometimes reports device tokens as inactive for no obvious reason, so make sure you’re prepared to hear from and reactivate users you’ve previously inactivated. (mappiness got bitten by this early on: a few dozen people were unable to authenticate to our server because they’d been reported as inactive and we’d assumed they were never coming back, despite that fact they had the app installed and Push Notifications switched on).
- The app delegate method
application:didRegisterForRemoteNotificationsWithDeviceToken:seems occasionally to return phantom zero-length device tokens — so don’t let your app or server get flummoxed by these. (We suffered from this on mappiness too: the bad device token got added to the front of a queue for sending from the app to our server; the server wouldn’t accept it; and this blocked all further data getting sent back until we fixed the server to silently ignore the bad tokens).
Extended and properly-multi-line Regular Expressions for JavaScript
Perl, Ruby, and some other languages support a readable ‘extended’ regular expression syntax, in which literal whitespace is ignored and comments (starting with #) are available. They also support a multi-line mode where the . character matches anything, including a newline.
JavaScript does neither of these: it doesn’t recognise the extended syntax, and its version of multi-line only allows the ^ and $ characters to match the beginnings and ends of lines within a string (it will never allow the . to match a newline).
So I wrote the following function to convert extended and fully-multi-line RegExp source strings to the basic syntax that JavaScript understands.
Tab completion for Stata variables in TextMate
I recently switched to TextMate for editing Stata .do files: unlike Stata’s built-in editor on the Mac, it has syntax highlighting and other goodies via Timothy Beatty’s bundle (now hosted by Dan Byler).
One thing it doesn’t have, though, is tab completion. Or rather, it didn’t, until now.
Using OS Code-Point Polygons in PostGIS
Ordnance Survey’s Code-Point with Polygons “provides a precise geographical location for each postcode unit in Great Britain”. It’s available in various formats, including ESRI .shp files.
Many UK academics can access the data via institutional subscription to EDINA Digimap. I’m using it in my research into subjective wellbeing and environmental quality.
This post shows how to:
- import the data files into a PostGIS database; and
- de-normalise the data into a single table, where there’s a one-to-one mapping of postcodes to rows, and each row contains either all geographical locations covered by a postcode (as a single geometry column, of type multipolygon) or the reason why no such location is available
Signing Amazon Product Advertising API calls in Ruby
I have a simple site that generates covers for CDs I burn from iTunes purchases and so on (it pre-dates widespread use of JS libraries, and is in much need of prettifying). The site uses Amazon Product Advertising API calls to search and retrieve album cover art and track listings. Since earlier this month, such API calls have to be cryptographically signed.
This is somewhat annoying — the site’s original design has it communicating independently with Amazon (using Amazon’s XSLT API feature to transform their XML data into JSON), and that’s no longer possible with the use of a private key. But it’s not unfixable. The site now sends its API call first to my server, which returns a signed version, and then forwards the signed call on to Amazon.
I found most of what I needed for this on Chris Roos’ blog, but his version still wasn’t quite working for me (the two problems I recall are that Ruby’s CGI.escape doesn’t quite follow Amazon’s requirements, and that times need converting to GMT).
O2 mobile broadband on a Mac: tips for you, suggestion for O2
Until Monday, O2 is offering a half-price USB dongle for pay-as-you-go mobile broadband. Buy it via Quidco, and you get not only the dongle but also a month’s access (or 3GB, whichever is the sooner) absolutely free.
So: free modem. No further commitment. And the opportunity to buy access a day at a time for £2, on a train journey, for example, or in an airport lounge, where the cheapest paid-for WiFi option is rarely less than £5, and often nearer £10. Handy.
Update. O2 is no longer doing the modem at half price, but the Quidco cashback has increased so that it still fully covers the modem and a £15 top-up.
Growling Mac backups with rsync
Between Time Machine and services like Dropbox, paranoid levels of backup are surprisingly painless to achieve on the Mac these days.
Still, just one more copy of your data, in just one more continent, surely can’t do any harm, right? One that won’t burn down with your house, but also isn’t just wafting vaguely in the Cloud at someone else’s whim. One that elevates your backup system from sensibly paranoid to borderline OCD. One, in this case, brought to you by rsync, find and Growl.