Archive for the ‘Comment’ Category
The great big website bonfire 25th Jun 10
So, the Coalition is going to close lots of unnecessary government websites. Hurrah. The Web Rationalisation project has been going for quite some time and though it’s not without its problems, it’s broadly a good thing. The coalition seems to be taking it up with renewed vigour, which is great. But making websites is cheaper and easier now, so we need to be mindful about how we do it.
There are serious problems with the way Government procures, develops and manages websites, and unless we change that, closing down websites will only be a short-term solution. People working in departments will always want new websites to do useful, valuable things, and they’ll usually find ways to make them. To call these “vanity websites” isn’t really fair. And pushing all citizen-facing content onto Directgov, NHS Choices or BusinessLink isn’t really sensible. Such large platforms bring a necessary degree of inflexibility which isn’t helpful to people who are trying to innovate.
I think that, as with many of the difficulties facing Government, IT and the Web, the problems begin with procurement. Government’s traditional suppliers just aren’t very good at making websites. Government needs to make sure that the new breed of suppliers can get their collective feet in the door. We need procurement processes that make it practical for SMEs to bid for work, civil servants who are keen to try a new approach and project management that takes account of the fact that a lot of “best practice” just isn’t, anymore.
But I think the most important thing to bear in mind today is that this response from the Coalition is a reaction to the problems of the past. Lots of Government sites are genuinely bad, and totally useless. And some of them date back to an era where that’s more or less all we could expect: because the ideas, the theory, the business processes and technology weren’t there. Expensive websites were the only websites there were. Bad was the best we could do. It’s a bit like comparing an AMC Gremlin to a modern car and expecting it to stack up.
But that’s emphatically not the case anymore. Technology and software development processes have improved substantially over the last decade. It really is possible to produce exemplary websites at a fraction of the cost that would have been unavoidable 6 or 7 years ago. NGOs and the private sector have seized on these technologies and ideas to unleash a new wave of products and services that have transformed the way we communicate and think about what the web can do. All the Government has to do is start commissioning it.
I hope that that’s what we’ll see happening over the next couple of years — and I especially hope that the problems of the past decade won’t blind us to the extraordinary opportunities of the next.
With thanks to Rory Cellan-Jones for the title!
The wraps come off data.gov.uk! 1st Oct 09
The UK’s version of data.gov, ably put together by the Cabinet Office, has just launched in private beta. We got to have a sneak peak, and it’s great!
The site is a blend of the US’s equivalent, data.gov, and Directgov | Innovate. It’s got a listing of available data packages, powered by the Comprehensive Knowledge Archive Network, and user-generated lists of apps and new ideas. This is just right: the data you need, combined with a way to promote the things you make and a place to get ideas if you’ve got itchy typing fingers but lack inspiration.
It’s not perfect. Conspicuously missing is an organised way to browse data sets: but that’s coming, along with some other tweaks and twiddlings that’ll improve the site’s usability.
The site is powered by Drupal, with packages catalogued and hosted by CKAN. Meanwhile, data.gov.uk hosts a data store powered by Talis that can scale to 100 billion triples and is hosted on Amazon EC2. The system is federated, so departments can add and control their own data, lots of which is available as RDF, with the remainder downloadable in spreadsheet form.
Speaking of spreadsheets, they’ve even written an app that departments can deploy in-house to convert spreadsheets into RDF (kudos to John Sheridan!) which makes it much easier for departments to produce structured, linked data.
This is all working now, and was put together by the team at the Cabinet Office in the last three months. This is a massive achievement, and it sounds like it’s just the beginning: they have big plans. User submissions for new datasets. Metadata to describe provenance. More data sets on the site. More data as RDF. Organised browsing for packages. Source code releases. The list goes on.
This is such an encouraging thing to see. No expensive procurement exercises for clunky, bespoke sites: instead, we have the right tools for the job, joined together. Simple things that do one job well, combined to form a more complex whole. It’s the Unix philosophy in action.
This is how all Government IT should work.
Our hearty congratulations go out to the team at the Cabinet office, with special thanks to Richard Stirling for spilling some of the beans. I had lots of questions and nitpicks, and every single one of them was answered reassuringly.
They’ve got a plan, and it’s a good one.
NHS Choices have a new hospital rating tool 11th Aug 09
Just discovered (via Twitter) that NHS Choices have released a new tool for users to rate hospitals. Fantastic. Government should embrace user generated content more often than it does, and feedback on Hospitals is a great application for it — but their implementation could do with a bit of tweaking.
First: the tool has been implemented as a new feature within the existing NHS choices website. Its design is clean, but very text heavy. This is not helped by the use of very long titles in the sidebar boxes that contain statistics — including the user-generated stuff — about the hospital you’re viewing. I suspect that they are this long because of a desire to make it ultra-clear which content is drawn from official statistics, and which is user-generated.
That’s a reasonable concern, but the site as it stands smacks of paranoia. I think that most people can tell which bits are official and which aren’t, even if the bits are in the same box. The distinction between “75% of people would recommend this hospital to a friend” and “1.33 MRSA infections for ever 10,000 bed stays” is obvious. There’s also a very strange chart in the user-generated sidebar: it appears in the middle of a sentence. I’d call myself a fairly seasoned web user, but even I found that confusing. It just looks broken.
Second: the comments aren’t prominent enough. They’re too far down the page, and relegated to a sidebar. They’re one of the most useful parts of the page — personal stories will always speak louder than dry statistics — so I’d give them a bit more importance, and put them in the main content area, beneath the hospital’s description. As well as being more prominent, it’s more consistent with how comments are usually presented.
If you click through to read all the comments, you find that they are presented in exactly this way on the next page, which is excellent: it displays all the comments in a way that makes it easy to absorb the ratings at a glance, summarises them right at the top of the page and has a prominent call to action for people who want to post feedback. Even more important than that, it has replies from the hospitals, which is fantastic. Government forays into the social web rarely ever result in real two-way communication. To see it being done is encouraging.
In short — despite being a bit rough around the edges — this is a nice bit of work, and definitely a big step in the right direction.
Andrew Stott — the new Director of Digital Engagement 13th May 09
I was slightly bemused when the Cabinet Office announced that it was going to create a new £160k position for the Director of Digital Engagement.
The job seemed like a tall order: a list of requirements that it would be hard for any one person to fulfill, and a very big job to do with very limited resources. It seemed like a strange move to make when creating two positions at £80k a piece would probably still attract very qualified people, and give you more time and knowledge for your money.
Nonetheless, I watched with interest, and now, a tad later than expected, the position has been filled by Andrew Stott. My initial reaction was along the same lines as Emma Mulqueeny’s — more bemusement — but actually, I think Andrew is a good choice. Not who I’d have expected, but good nonetheless. As numerous people have said, he is very qualified, does have a brain the size of a planet, and has lots of experience pushing through the kind of change that we need. More than that, though, he’s practical.
I worked with Andrew briefly in 2008. One of the things we were looking at at the time was the quasi-XML version of the Civil Service Yearbook, which has lots of useful data in it. As is usually the case, though, it wasn’t proper XML — it’s variously broken, inconsistent and badly written. We spent a satisfying ten minutes at the end of the day bemoaning such irritations, and the next morning Andrew showed up at the office having spent all the previous evening writing a bunch of code to take the nasty XML and make it into useful data.
That, I think, is indicative of the man.
Reforming Ordnance Survey 25th Apr 09
People have been talking about the Ordnance Survey rather a lot recently.
It’s a very strange beast. It has mountains of really useful data: electoral boundaries, postcode databases and the locations of all sorts of buildings, not to mention roads, railways and green spaces. Unfortunately, it’s a quasi-independent body, which has to pay its own way. It charges heaps of money for access to all these datasets, and makes them available under strange and deeply inappropriate licences.
This has long been a problem for civic hackers, and by implication, everyone else: if you want to make a service that needs to turn a postcode into a geographic location, you need to use their data, and most of the time, you can’t. It costs an arm and a leg.
Fortunately, it’s clear to everyone that the OS needs to be significantly reformed. They own datasets like electoral boundaries, that are fundamental to the operation of a democracy. They’ll charge you if you want to use them, even if you’re the government and you’re organising an election. That’s just nuts, and we all know it. So what to do?
Thankfully, people have been working on it. The recent Power of Information Taskforce report had lots of good things to say: Recommendation 7 was almost perfect. Liberate postcodes and boundary data. Make basic mapping data available for free to all, for modest use. Simplify licensing. Its only real downside was the absence of any mention of derived data.
Put simply, if you use an OS map to create a database of something — like postboxes, hospitals or parks — then OS share copyright in that data. In this modern age of user generated content, that position is completely unsupportable. It’s a shameless grab for intellectual property, motivated by their desire to receive extra royalties. For a private company, that might be fair enough, but it’s reprehensible for a body that exists to provide a public service.
For anyone who thinks this is merely a theoretical problem, it may be of interest that all those lovely crime maps launched by police forces across the UK are probably in violation of the OS’s licensing terms.
This, then, was the background to the latest budget. Having heard that it would contain some new announcements, we were waiting for its publication with baited breath, and indeed: it promised reform. The following day, Ordance Survey published their new commercial strategy. It is underwhelming.
There are some good changes: more data, including boundary information, will be made available through OpenSpace, their API. They’re going to revise their definition of “commercial” so that sites that use their data can carry advertising without being required to pay for licences. But that, more or less, is it. The rest of the strategy revolves around converting people using free licences into ones that become financially sustainable so they can pay. Fair enough, but hardly groundbreaking.
The real problems remain. OS still own electoral boundary data and postcode boundaries & locations. They still decide if you’re commercial, and you still have to accept their onerous licences to use their data. OS still maintain a stranglehold on any data that they consider to be derived from theirs. They’ll still charge you royalties to use that data, of at least £1000 a year. You still won’t be able to do anything with that data that’s not acceptable under their licence, like adding it to OpenStreetMap.
This new strategy is progress, but only just. It is at best a fractional improvement upon what we had before.
A lot more needs to be done.
The Office of National Statistics and Postcodes 12th Mar 09
Here’s a story from FreeOurData which is, quite frankly, incredible. The Office of National Statistics, in preparing for the next census, has found that the postcode databases offered by the Royal Mail and Ordnance Survey aren’t accurate enough for their purposes. Their solution: to build their own database. This is fair. The postcode database is not amazingly accurate, and ONS have different requirements anyway.
Unfortunately, Royal Mail and Ordnance Survey make good money from selling the postcode databases to other organisations. These datasets are very valuable: you’ve probably made use of them whenever you’ve put your postcode into a website. Royal Mail and Ordnance survey did not — apparently — like the idea of ONS making another postcode database with which they’d presumably have to compete. So, rather than take that nice dataset and do useful things with it — like giving it back to us taxpayers — the ONS have pledged to build the database, use it for the census, and then destroy it.
Postcode databases are almost a holy grail. Of all the datasets in the country, liberating the postcode database for free reuse would probably create more value than any other. The thought of spending £12m on a new, super-accurate postcode database and then destroying it is wasteful, a huge missed opportunity and to be frank, completely idiotic.
We implore you: don’t do it.
Comment on the Power of Information Taskforce's report 11th Feb 09
The Power of Information Taskforce have been figuring out how to liberate public sector information, how to facilitate better use of the modern, social web in government, and how to support the efforts of those outside government who are doing worthy things. All in all, they’re a great bunch of people, doing great work. They’ve just published their draft report. This is notable for two reasons.
First, the report has been published using Commentariat, a WordPress theme from the folks at DIUS that makes it easy to browse and comment on big documents. It is fantastic — it really works well — and you’d never know it was WordPress, unless you checked. It’s a great example of WordPress’s flexibility.
For me, this informal consultation exercise is characterised by its ease. I read the report online. When I had something to say, I could just fill out the comment box. I could read other people’s comments, which helped to clarify my own thoughts. The PoI team have been posting comments too: responding to people, thanking them for their feedback, letting us know when they’ve made changes. Brilliant. It’s a real conversation between people who genuinely want to seek out ideas. It stands in stark contrast to the process of formal consultation, which is stifled, slow, more or less one-way, and frequently happens after all the important decisions have already been made. This exercise couldn’t be more different: agile, easy, conversational and public. I have no doubt that the report will be better because of it.
Second, the report itself is great: it hits all the right boxes. Be active in other people’s networks. Make sure civil servants have decent ‘net access — it’s astonishing how many of them are so filtered that they’re almost useless. Support third sector developers. Invest in innovative new ideas, even if they’re high risk. Liberate geodata from burdensome licensing and fees. There are 25 recommendations in all, and they’re all great.
If you have some time, read the beta report and leave some comments. It’s a great document, and great opportunity to get your thoughts in front of people who will listen.
Innovation in Government: SchoolClosures.org.uk 3rd Feb 09
I was at the UKGovWebBarcamp last weekend, and among the talks I attended was one by the Directgov Innovate team. This team has been recently formed, and is a really good development. In their own words:
Directgov have created the innovate.direct.gov.uk developer network to inform the greater developer community about available resources, to provide a platform to connect with one another, and to showcase new ideas with the aim of supporting and encouraging innovation.
Over time we will provide content feeds and API’s allowing people to develop new and interesting ideas and applications for use by the greater community.
Among my questions was: what will this team actually do? I was very glad to hear that they plan to develop new sites, make APIs, make data available to people and create a community of developers who are interested in this field. Great. This is just what’s needed. What’s even better is that they’ve already delivered on that promise.
Yesterday, Tom Watson tweeted that we should have a site where people can check to see if their school is closed. Brian Hoadley and Paul Clarke at Directgov took up the challenge, and just a couple of days later, launched a new site: SchoolClosures.org.uk.
It’s pretty rough around the edges: there doesn’t seem to be much RSS support, and there’s no access to the underlying data, and — well — it doesn’t tell you whether your school is closed… but it is still useful, and it’s very impressive that it appeared so quickly, and with such little prompting.
Kudos to all involved — this is a fantastic and very encouraging start.
DCSF Statistical Releases, the BBC and Better Data Formats 15th Jan 09
Simon Dickson picks up an interesting story from the BBC’s Editors’ blog about official releases of statistics.
Usually, when the Department of Children, Schools & Families releases new statistics, they’re given to the media in advance. The media need this lead time to be able to format all their articles and tables and make sure everything is correct and works properly: this is fair enough, given that they’re the data they’re working with lives in lots of Excel spreadsheets, with multiple sections, differing layouts and everything else you really don’t want if you’re tasked with this kind of job.
Given what they have to work with, the BBC’s anger is understandable, but perhaps misplaced: why are we still dealing with bunch of spreadsheets in the information age? Why isn’t there an API that allows this data to be queried, or at the very least, a standard data format that doesn’t change from year to year, and doesn’t reply on proprietary technologies that are hard to work with?
An API or standard data format would allow media organisations to write code which generates the statistics they need every year. They wouldn’t have to create new tools to be tweaked and tested every time there are new statistics. Better still, it would create a market for someone to create a tool that did this for them, saving them money. Even better than that, it would allow anyone who wants to do something innovative with these statistics to do so far more easily.
I think I’m not alone in saying that the case for releasing data properly — in reusable formats, to everyone, for free, whenever it is possible — has been made, has been heard and has been widely accepted as valid.
Why are we still fiddling with messy spreadsheets, and bemoaning the fact that we only have days to do what should take hours?
Crime Maps 8th Jan 09
Maps to display levels of crime, nationwide, were promised by the Government last year. It’s great to see that they’ve finally launched. This is a step in the right direction.
Unfortunately, it’s only a small one. Some forces have collaborated, but most have their own maps — which seems very strange. Why not a central service? We have the National Crime Reporting Standards, so there must be some consistency in the data from individual forces. It seems rather strange to implement 40-odd websites that all do the same thing, not to mention inconvenient: people who live in areas that adjoin other police forces will have to check two or more sites to get a good idea of the levels of crime in their neighbourhood.
It also means that the quality of these sites is extremely variable: from the good, to the average, to the downright mediocre. Why didn’t the Met simply share — or even, sell — its nice Google Maps mashup to the other forces around the country? Why waste money reinventing the wheel?
We would also love to see an API, or at least some way to get these stats in a machine-readable format. We’d like to be able to take this data and play with it. We’d like graphs that compare levels of crime to population, mean income, number of CCTV cameras, number of police officers per person, proximity to alcohol retailers, or anything else that takes our fancy. Of course, not all this data is readily available, but getting hold of crime stats would be a great improvement.
Finally: there’s some speculation over whether these maps break the Ordnance Survey’s licensing terms. Said terms are extremely bad: expect programmers who work in this sector to go into histrionics if you mention them. The Home Office have said, more or less, that they’ll sort it out. Quite how, or what was decided, is unclear: Did the OS get a shedload of money, or did the government come to some arrangement? If it’s the latter, I hope it may go some way towards helping third-sector projects too.
In any case, despite all these gripes, this is a great development. Congratulations are due to all involved: hopefully, this is just the first of many such innovations to come.