George MacKerron: code blog

Since light can affect happiness, two important pieces of environmental data I add to the Mappiness data set during analysis are: (1) whether a response was made in daylight; and (2) day length when and where the response was made.

To derive these, I need the date, time and location of the response, and sunrise and sunset times for that date and location. R’s StreamMetabolism library provides sunrise/sunset calculations based on NOAA routines. And since my data is in PostGIS, it’s handy to use PL/R to access these.

I set this up as follows on my Ubuntu 12.04 GIS box.

In bash:

sudo aptitude install postgresql-9.1-plr
sudo R

In R:

install.packages('StreamMetabolism', dependencies = TRUE)

In Postgres (9.1):

create extension plr;
create table plr_modules (modseq int4, modsrc text);
insert into plr_modules values (0, 'library("StreamMetabolism")');
 
create or replace function 
  _r_daylight_period(lat double precision, lon double precision, datestring text, 
                     timezone text)
returns setof integer as $$
  return(as.integer(sunrise.set(lat, lon, datestring, timezone)))
$$ language plr immutable strict;
 
create or replace function 
  sunrise(location geometry, moment timestamp without time zone, timezone text)
returns timestamp without time zone as $$
  select to_timestamp(min(s)) at time zone $3 from _r_daylight_period(
    st_y(st_transform($1, 4326)), -- 4326 = WGS84
    st_x(st_transform($1, 4326)),
    to_char($2, 'YYYY/MM/DD'),
    $3
  ) s
$$ language sql immutable strict;
 
create or replace function 
  sunset(location geometry, moment timestamp without time zone, timezone text)
returns timestamp without time zone as $$
  select to_timestamp(max(s)) at time zone $3 from _r_daylight_period(
    st_y(st_transform($1, 4326)), -- 4326 = WGS84
    st_x(st_transform($1, 4326)),
    to_char($2, 'YYYY/MM/DD'),
    $3
  ) s
$$ language sql immutable strict;
 
create or replace function 
  is_daylight(location geometry, moment timestamp without time zone, timezone text) 
returns boolean as $$
  select (sunrise($1, $2, $3), sunset($1, $2, $3)) overlaps ($2, cast('0' as interval));
$$ language sql immutable strict;
 
create or replace function 
  hours_in_the_day(location geometry, moment timestamp without time zone, timezone text) 
returns double precision as $$
  select extract(epoch from sunset($1, $2, $3) - sunrise($1, $2, $3)) / 3600.0;
$$ language sql immutable strict;

These new functions can be used like so:

select 
  is_daylight(geom, moment, 'Europe/London'), 
  hours_in_the_day(geom, moment, 'Europe/London') 
from mytable;

(Note: what I actually do with the Mappiness data is to use a single call to _r_daylight_period, and calculate the other quantities as a second step. This speeds things up a lot, because I have millions of rows to process and Postgres doesn’t appear to do as much caching of immutable function results as one would like here).

We have an old MacBook (the original white Intel model from 2006) running EyeTV as our telly.

Apple’s Magic Trackpad makes a handy remote control for this setup. Unfortunately, Bluetooth on the old MacBook is highly temperamental. Built-in Bluetooth regularly fails: out of the blue, the Mac decides that it has no Bluetooth module after all, puts a wavy line through the menu bar icon, and ignores the trackpad. The only fix for this is to shut the computer down, and leave it off for several minutes. This is annoying.

I had an old D-Link Bluetooth dongle (DBT-120), so I tried using this instead. This is better, in that the failure mode is less annoying. Using the dongle, the trackpad’s Bluetooth connection only fails after a prolonged sleep. It looks as if it’s still connected, with a dotted line across the Bluetooth icon, but it’s unresponsive. This can be fixed by simply unplugging and replugging the dongle. But that’s still a pain, and is rather a compromise of the ‘remote’ in remote control.

It turns out that the post-sleep unresponsiveness may also be fixed by restarting the Bluetooth daemon, by typing sudo killall blued in the Terminal.

This is good news, because we can automate this action using sleepwatcher.

If typing sudo killall blued in Terminal solves your Bluetooth issues after sleep, then you may want to use sleepwatcher too. (Note that after a sudo command, you may be asked for a password. Nothing will show up as you type, but that’s OK: just type your password and press Return).

More »

When converting coordinates between WGS84 (GPS) and OSGB36 (UK National Grid), using OSTN02 can gain us a few metres in accuracy over the basic parametric transformation provided by PostGIS’s st_transform via PROJ.4.

Happily, Ordnance Survey now distribute an NTv2 version of the OSTN02 transformation, courtesy of the Defence Geographic Centre, which can be used by PROJ.4 and, therefore, PostGIS.

More »

I recently needed to calculate a SHA1 hash in an iOS app.

In iOS4+ it’s possible to use CommonCrypto, but Mappiness has always supported iOS3. I therefore added NSData and NSString categories to a public domain C implementation instead. This remains public domain: do with it what you will.

It relies on a hex-encoding category on NSData, which you can also consider public domain.

More »

PostGIS 2 has some exciting new goodies — including raster support — that I’m keen to use in the analysis of Mappiness data.

But the PostGIS package provided by Ubuntu 12.04 is still only at version 1.5, and the GEOS and GDAL packages are also too old to support the new version.

So — this is how I compiled PostGIS 2.0.1 2.0.3 and its dependencies on my GIS server.

More »

I’ve just released Spindlytext on Github. It’s the library that powers the live data display in Pigeon Sim, by creating KML linestrings in the shape of letters.

More »

Update, 10 February: Sorry for some serious reliability issues over the last few days. The streaming server is now hosted in-house at CASA, which should be a lot more robust.

At CASA we’ve been looking at using a Kinect or three in our forthcoming ANALOGIES (Analogues of Cities) conference + exhibition.

We’ve been inspired in part by Ruairi Glynn‘s amazing work here at UCL, and Martin has been happily experimenting with the OpenKinect bindings for Processing.

Meanwhile, I recently got to grips with the excellent Three.js, which makes WebGL — aka 3D graphics in modern browsers — as easy as falling off a log. I’m also a big fan of making things accessible over the web. And so I began to investigate prospects for working with Kinect data in HTML5.

There’s DepthJS, an extension for Chrome and Safari, but this requires a locally-connected Kinect and isn’t very clear on Windows support. There’s also Intrael, which serves the depth data as JPEG files and provides some simple scene recognition output as JSON. But it’s closed-source and not terribly flexible.

The depthcam

So I decided to roll my own. I give you: the depthcam!

More »

There are plenty of ways to get spatial data from a PostGIS database into a Processing sketch.

You can export to CSV or SVG and load it from there; you can query the database directly; or, depending on context, you might choose to generate Processing commands directly, which is the route I went to display a background map of the UK in a recent visualization project.

More »

Need to protect something with a passcode in an iPhone app you’re developing?

Then you may find my MIT-licensed passcode view controller — as seen in the mappiness app and in the short screencast below — of use.

See Github for the code and (scant) documentation.

Imagine you want compare various locations in terms of the availability of a certain type of environment, such as fresh water.

You might want to use a measure of the proximity of that environment — such as the nearest neighbour distance.

You might want to use a measure of the quantity of that environment in the vicinity — such as the proportion of land within a specific radius that is of that type.

Or you might ideally like a measure that combines both of these: one that incorporates the quantity of that environment, but gives greater weight to areas that are nearer, and lesser weight to those that are further away.

In that third case, what you probably want is a kernel-weighted proportion.

More »