George MacKerron: code blog

Today, I needed a JavaScript equivalent of Ruby’s Array#compact (which returns the array stripped of any nil values).

A standard JavaScript implementation looks like this:

Array.prototype.compact = function() {
  var x, _i, _len, _results;
  _results = [];
  for (_i = 0, _len = this.length; _i < _len; _i++) {
    x = this[_i];
    if (x != null) {
      _results.push(x);
    }
  }
  return _results;
};

But since I’m using CoffeeScript, I get this monstrosity of line-noise and distraction for free when I type this:

Array::compact = -> x for x in @ when x?

This is some of the best of CoffeeScript: array comprehensions, implicit return, the existential operator ?, and more. Taking 11 lines of JavaScript busywork and turning it into 40 pithy, elegant characters.

Preamble

In the last few years, Objective-C has become enormously DRYer. For example: in the past, adding a property to a class meant adding several of: an ivar, a @property, a @synthesize, a getter/setter, and a release call in dealloc. Worse, renaming or deleting the property meant updating all these. This was error-prone busywork, and made iOS development frankly pretty tedious.

Nowadays a single @property is often all that’s needed. Plus of course we’ve lost all that retain / release / autorelease noise, and got fast enumeration, literals for NSNumber, NSArray, NSDictionary and so on. This is great (and is one reason why I’ve come back to Xcode after a brief dalliance with RubyMotion).

Anyway, I really do hate repeating myself, and one of the more annoying remaining areas where this has been necessary has been string constants. These are pretty widely used in Cocoa for NSNotification names, dictionary keys, and so on.

Problem

In the past I’ve used #define in my headers. For example,

This seems like fairly innocuous repetition, but it’s still annoying. And the idiomatic/Apple way is worse. You do this in the header file:

Backed up by this in the implementation file:

We type the same identifier (and its accompanying line noise) no fewer than THREE times.

Solution

The best solution I’ve found involves a macro plus a one-line sed command.

The macro goes like this:

And the one-line sed command (added as a build phase straight after ‘Target Dependencies’) goes like this:

Now all you do for a string constant is type this in your header file:

The macro converts this to the right form for the header, and the sed command searches all your headers to create a single implementation file that contains all the corresponding definitions (you’ll need to manually add this file to the project when it’s first created).

If you think this is too much magic, I’ll understand. But for me, it’s a necessary battle in the war on busywork.

For Cocoa developers: I’ve just put a small category on NSMutableAttributedString on Github.

More details at https://github.com/jawj/NSMutableAttributedString-InlineStyles

For reasons that may become clearer in future, I needed to use a bandpass filter in an iOS app. The DSP part of Apple’s Accelerate framework makes this lightning fast both for the programmer to implement and for the machine to execute … if the programmer knows how use vDSP_deq22, which is documented, at best, concisely.

The following functions produce the five-coefficient filter definition you need to pass to vDSP_deq22. Since this took me a little while to put together, I thought I’d share.

void makeBandpassFilterWithFcAndQ(float* filter, double Fs, double Fc, double Q) {
 
  // Fs = sampling rate, Fc = centre freq
  // with thanks to http://www.earlevel.com/main/2013/10/13/biquad-calculator-v2/
  // and https://github.com/bartolsthoorn/NVDSP
 
  double K = tan(M_PI * Fc / Fs);
  double norm = 1.0 / (1.0 + K / Q + K * K);
  filter[0] = (float)(K / Q * norm);
  filter[1] = 0.0f;
  filter[2] = -filter[0];
  filter[3] = (float)(2.0 * (K * K - 1.0) * norm);
  filter[4] = (float)((1.0 - K / Q + K * K) * norm);
}
 
void makeBandpassFilterWithFreqRange(float* filter, double Fs, double Fbtm, double Ftop) {
 
  // with thanks to 
  // http://stackoverflow.com/questions/15627013/how-do-i-configure-a-bandpass-filter
  // -- this sets Q such that there's -3dB gain (= 50% power loss) at Fbtm and Ftop
 
  double Fc = sqrt(Fbtm * Ftop);
  double Q = Fc / (Ftop - Fbtm);
  makeBandpassFilterWithFcAndQ(filter, Fs, Fc, Q);
}

And you use it like so:

float filter[5];
makeBandpassFilterWithFreqRange(filter, sampleRate, filterLoRate, filterHiRate);
vDSP_deq22(rawFloats, rawStride, filter, filteredFloats, filteredStride, numRawFloats - 2); 
  // rawFloats and filteredFloats are pointers, of course

I recently posted a HOWTO based on my experience moving a Xen domU from Linode to my own Xen Dom0 setup at Hetzner.

Since this machine is only a development server, I more recently decided to turn the same machine into a VMware VM, running locally (in VMware Fusion 4 on my MacBook Pro). Here, I document the steps necessary for that transformation.

More »

I’ve been playing with the much-enhanced raster support in PostGIS 2.1 in the last few days. Amongst other things I’ve been producing maps with ST_MapAlgebra, which has changed a little since 2.0.

The example callback function in the documentation is somewhat unhelpfully bare-bones: it always returns 0, touching none of its arguments. So here’s an equally useless — but rather more informative — example of a callback function for the simple one-raster, one-band, one-pixel-at-a-time case:

create or replace function 
callback_fn(pixel float[], pos integer[], variadic userargs text[]) 
  returns float 
  language plpgsql 
  immutable  -- careful: this function is immutable, yours may not be
as $$
  declare
    pixval float;
    inputx integer;
    inputy integer;
  begin
    pixval := pixel[1][1][1];  -- pixel indices: [raster #][xdistance][ydistance]
    inputx := pos[1][1];       -- pos indices:   [raster #][x = 1, y = 2]
    inputy := pos[1][2];       --                (raster #0 is the output raster)

    return pixval + inputx + inputy;
  end;
$$;

-- example call:

select st_mapalgebra(rast, 1, 'callback_fn(float[], integer[], text[])'::regprocedure) 
from raster_table;

And here’s something that caught me out: unless you’re passing some userargs, don’t declare your callback function to be strict. If you do, your callback function will never be called because its userargs argument is always null.

Since light can affect happiness, two important pieces of environmental data I add to the Mappiness data set during analysis are: (1) whether a response was made in daylight; and (2) day length when and where the response was made.

To derive these, I need the date, time and location of the response, and sunrise and sunset times for that date and location. R’s StreamMetabolism library provides sunrise/sunset calculations based on NOAA routines. And since my data is in PostGIS, it’s handy to use PL/R to access these.

I set this up as follows on my Ubuntu 12.04 GIS box.

In bash:

sudo aptitude install postgresql-9.1-plr
sudo R

In R:

install.packages('StreamMetabolism', dependencies = TRUE)

In Postgres (9.1):

create extension plr;
create table plr_modules (modseq int4, modsrc text);
insert into plr_modules values (0, 'library("StreamMetabolism")');
 
create or replace function 
  _r_daylight_period(lat double precision, lon double precision, datestring text, 
                     timezone text)
returns setof integer as $$
  return(as.integer(sunrise.set(lat, lon, datestring, timezone)))
$$ language plr immutable strict;
 
create or replace function 
  sunrise(location geometry, moment timestamp without time zone, timezone text)
returns timestamp without time zone as $$
  select to_timestamp(min(s)) at time zone $3 from _r_daylight_period(
    st_y(st_transform($1, 4326)), -- 4326 = WGS84
    st_x(st_transform($1, 4326)),
    to_char($2, 'YYYY/MM/DD'),
    $3
  ) s
$$ language sql immutable strict;
 
create or replace function 
  sunset(location geometry, moment timestamp without time zone, timezone text)
returns timestamp without time zone as $$
  select to_timestamp(max(s)) at time zone $3 from _r_daylight_period(
    st_y(st_transform($1, 4326)), -- 4326 = WGS84
    st_x(st_transform($1, 4326)),
    to_char($2, 'YYYY/MM/DD'),
    $3
  ) s
$$ language sql immutable strict;
 
create or replace function 
  is_daylight(location geometry, moment timestamp without time zone, timezone text) 
returns boolean as $$
  select (sunrise($1, $2, $3), sunset($1, $2, $3)) overlaps ($2, cast('0' as interval));
$$ language sql immutable strict;
 
create or replace function 
  hours_in_the_day(location geometry, moment timestamp without time zone, timezone text) 
returns double precision as $$
  select extract(epoch from sunset($1, $2, $3) - sunrise($1, $2, $3)) / 3600.0;
$$ language sql immutable strict;

These new functions can be used like so:

select 
  is_daylight(geom, moment, 'Europe/London'), 
  hours_in_the_day(geom, moment, 'Europe/London') 
from mytable;

(Note: what I actually do with the Mappiness data is to use a single call to _r_daylight_period, and calculate the other quantities as a second step. This speeds things up a lot, because I have millions of rows to process and Postgres doesn’t appear to do as much caching of immutable function results as one would like here).

We have an old MacBook (the original white Intel model from 2006) running EyeTV as our telly.

Apple’s Magic Trackpad makes a handy remote control for this setup. Unfortunately, Bluetooth on the old MacBook is highly temperamental. Built-in Bluetooth regularly fails: out of the blue, the Mac decides that it has no Bluetooth module after all, puts a wavy line through the menu bar icon, and ignores the trackpad. The only fix for this is to shut the computer down, and leave it off for several minutes. This is annoying.

I had an old D-Link Bluetooth dongle (DBT-120), so I tried using this instead. This is better, in that the failure mode is less annoying. Using the dongle, the trackpad’s Bluetooth connection only fails after a prolonged sleep. It looks as if it’s still connected, with a dotted line across the Bluetooth icon, but it’s unresponsive. This can be fixed by simply unplugging and replugging the dongle. But that’s still a pain, and is rather a compromise of the ‘remote’ in remote control.

It turns out that the post-sleep unresponsiveness may also be fixed by restarting the Bluetooth daemon, by typing sudo killall blued in the Terminal.

This is good news, because we can automate this action using sleepwatcher.

If typing sudo killall blued in Terminal solves your Bluetooth issues after sleep, then you may want to use sleepwatcher too. (Note that after a sudo command, you may be asked for a password. Nothing will show up as you type, but that’s OK: just type your password and press Return).

More »

When converting coordinates between WGS84 (GPS) and OSGB36 (UK National Grid), using OSTN02 can gain us a few metres in accuracy over the basic parametric transformation provided by PostGIS’s st_transform via PROJ.4.

Happily, Ordnance Survey now distribute an NTv2 version of the OSTN02 transformation, courtesy of the Defence Geographic Centre, which can be used by PROJ.4 and, therefore, PostGIS.

More »

I recently needed to calculate a SHA1 hash in an iOS app.

In iOS4+ it’s possible to use CommonCrypto, but Mappiness has always supported iOS3. I therefore added NSData and NSString categories to a public domain C implementation instead. This remains public domain: do with it what you will.

It relies on a hex-encoding category on NSData, which you can also consider public domain.

More »