Lab Notes
WordPress Docs 29th Apr 10
With a couple of days between projects, I had a chance to work on my latest 20% project.
RailsAPI‘s sdoc format is very nice in that it allows searching over all methods and classes, and is generally an intuitive way of doing things.
WordPress is full of PHPDoc comments, the Codex is invaluable, and of course being able to read the source is absolutely necessary much of the time. All in all, WordPress is quite well documented, though there’s really no coherent place that brings that all together. So I am very happy to have this new tool available before my next WordPress jaunt: WordPress Docs.
Posty – our new gem for geolocation in Ruby 13th Apr 10
Just last week Ordnance Survey released a tonne of data. This is of course a wonderful thing for everybody, and software making use of this data has only started appearing in the past week. This morning, in fact, we noticed a gem, pat by Stef Lewandowski which uses Pezholio/Stuart Harrison’s very useful UK Postcodes.
We took a look at it, and decided to make a gem that uses a local database (usually a bit faster than using a Web-based API), but which is similarly easy to use.
Three hours later, we present to you posty!
The data it provides from CodePoint Open is:
- postcode
- quality
- latitude
- longitude
- country
- nhs_region
- nhs_health_authority
- county
- district
- ward
In all these cases, rather than proving the name of the county/district/ward/etc, it only provides the identifier used in the data set. This is a case where UK Postcodes would be a better option.
To install it,
sudo gem install postyTo set up the database (which can be downloaded automatically from mySociety, or you can download the latest version yourself and use the -c option),
sudo posty-init -gThis should take about 8 minutes to load all the CSV into the database.
To use it,
require 'posty' Posty.postcode('SE1 1EN')
If you’re going to be using it frequently from the same process, you should probably use the following instead,
require 'posty' posty = Posty.new postcode = posty.postcode('SE1 1EN') postcode = posty.postcode('SE1 7PB')
The magick going on here is that the database is stored in the gem (in lib/posty/codepointopen.sqlite3), and then it’s available without having to connect manually.
Have a fiddle with posty and tell us what you think.
Migrating from Gitorious to Gitosis 12th Apr 10
Gitorious is very nice for social coding, and I thought it would be perfect for us at first. Apparently its git cloning is not quite adequate for us, as we need git pulling and git pushing and some of those cloning shouldn’t be able to write (such as deploy users).
Since we decided Gitorious wasn’t for us, we’ve just left it as it is (frequently restarting ActiveMQ and the poller, as they crash a lot). We just never had an excuse to spend the time to migrate until today when Gitorious broke completely for no apparent reason.
The process was fairly trivial, but I’ll share my code anyway as it was a somewhat interesting endeavour.
First of all then, Gitorious stores its repositories in quite an annoying way if you want to get at them outside of Rails. It stores them as some kind of hash, but split up to prevent the directories from containing trillions of entries. For instance, git@thedextrousweb.com:wordpress-plugins/wp-recaptcha is stored in /var/git/repositories/e27/004/48f5b0e17f3403aa15efd0ea8d47ce874a.git. The database then holds a mapping from the human-readable path to the actual path.
Gitosis, on the other hand, uses a mapping from wordpress-plugins/wp-recaptcha to ~/repositories/wordpress-plugins/wp-recaptcha.git, which is much more obvious.
So we need the mapping, for which we shall have to prod Rails’ database:
require File.dirname(__FILE__) + '/../config/environment' Repository.all.each do |r| name = r.project.slug + '/' + r.name path = r.hashed_path puts "#{path} #{name}" end
All that does is prints out the physical path to the repository, followed by the human-readable path, separated by a space. So redirect that to some file, and enter gitosis’ repository directory (possibly as the git user, or chown -R later). I’ll be using grep -v wiki here because we never used the wikis. So we first create the directories because git won’t mkdir -p:
grep -v wiki files | cut -d' ' -f2 | cut -d/ -f1 | sort | uniq | xargs mkdir
And rather than using cp -R or rsync, I thought it would be best to let git do the job:
grep -v wiki files | perl -pe 's/ /.git /;s/^/git clone --mirror \/var\/git\/repositories\//;s/$/.git/' | sh
Since we’ve only got a few users, we’re not going to bother importing all the users and groups, so we can just cut the list of repositories, remove the newlines, and append it to our writeable.
Testing WordPress with Cucumber 19th Nov 09
I should probably introduce myself first: I’m Tom, Dextrous Web employee #1.
We do quite a bit of work with WordPress, and one of the problems we’ve found is that it doesn’t lend itself to the kind of automated testing that we do with the rest of our projects. The code for themes easily gets pretty cluttered, and it’s hard to know what the consequences of installing a WordPress upgrade or new plugin might be.
One of the tools that we use for our Ruby on Rails projects is Cucumber, a Behaviour Driven Development tool that makes front-end testing of Rails applications really easy. There’s an example below, but in brief: it lets you write human-readable tests and then run them on your application to make sure it’s working properly. Because Cucumber tests the front-ends of web applications, it’s not tied to any one technology — so we thought, why not use it to test WordPress?
So I spent some time writing a little bit of configuration magicks and step definitions, and here it is: cucumber-wordpress.
To get started you’ll want to install the gem:
gem install cucumber-wordpress --source http://gemcutter.org
Then copy the example features directory from the gem into the root of a WordPress installation:
cp -R /usr/lib/ruby/gems/1.8/gems/cucumber-wordpress-1.0/examples/features .
Configure your test database, and where the site is being served from:
vim features/support/config.yml
And test:
cucumber
Here’s a sample:
Background:
Given WordPress is installed
Scenario: Submitting a post
Given I am logged in as "admin"
And I am on admin dashboard
When I follow "Add New" within "#menu-posts"
And I fill in "title" with "I <3 cucumber"
...I’ve been testing a plugin with this for a week now, and it’s been very smooth.
Here at The Dextrous Web we’re committed to backporting the Rails culture to WordPress.
Data: Weekly Fuel Prices 11th Aug 09
James Darling and I met up for a chat about some data-related stuffs this afternoon, and came across this data on average fuel prices via the Office of National Statistics. This struck us both as being very useful (any hauliers out there who want to make some a nice visualisations?) so we threw together a script to convert it into (much) more useful formats.
Check out Weekly Fuel Prices, in more formats than you can shake a stick at, here.
ScenicOrNot: want to play with the data? 26th Jun 09
mySociety have added a data dump to ScenicOrNot, the site we built for them a couple of months ago. It’s got the photos and all the votes for each of the 181,300 places that have received 3 or more votes since the site launched.
If you’re one of the many people who had something to say about the voting system that ScenicOrNot uses, we hope you might have some fun playing with the raw data! If you do make something, let us know how you get on…
DFID Procurement Opportunities 30th Dec 08
One of the things that I mentioned in our last post was the lack of RSS feeds for various parts of the DFID’s new website. Today, someone else picked up the same point in a Twitter update.
This is just the kind of thing we like to do, so we knocked up a script to generate an Atom feed from DFID’s procurement page. If you’re interested in DFID’s procurement opportunities, do add it to your feed reader, and let us know how you get on.