Chief among the Indian projects was some amazing mapping and visualisation work. David McCandless has done a great write-up on the Guardian Data Blog. Another fabulous project was National Election Watch, who create public data on candidates by entering the details from their affadavits. This site has one of the most awesome features I’ve ever seen on a political website: a crime-o-meter for each candidate.

But the most interesting thing which came up was the widespread use of missed calls as a way to communicate. Entire services have sprung up built on the idea that you call a number, let it ring once, and then hang up. Which is free. And then the service calls you back. The lack of any privacy regulation also allows these services to enter agreements with the telecomms companies to obtain location data on callers.

This led me to wonder: could a system be made, powered by missed calls, to report incidents of police abuse? Like an ultra-light-weight Ushahidi? So, along with a couple of the Indian devs, I put together a mock-up:

The pins need to be converted into coloured areas, but hopefully that conveys the general idea: that people who experience problems with the police — like being assaulted, or forced to pay a bribe — could anonymously report it, allowing statistics to be built up about where police abuses take place. These could then be compared to the official statistics, or to the numbers of official complaints made. And could allow people to get a real sense of the scale and geography of the problem.

Such a system could work — despite most Indians’ lack of internet access — because it’s powered by mobile phones, which almost everyone has. Even more than that, by taking advantage of missed calls, it becomes free to use and to operate, unlike SMS. And missed calls are a practical approach, because they’re already known and used by Indians in other contexts.

Unfortunately, because we only had the morning, I couldn’t do much more than that basic mock-up. But if anyone in India is reading this, and would like to take over the project, let me know! I’d love to see it up & running.

Update: I’ve just done a podcast interview about the trip with Steph Gray of Helpful Technology.

From my loose understanding, India certainly faces some interesting and unique challenges. But I don’t know what they are, or how best the web can solve them. I’m sure there’s a broad section of Indian society for whom web applications are not a very practical solution to any of the challenges they face. At least for the time being. So, what are the problems experienced by Indians who are digitally included? And how can we use the web to solve them?

Should we be focusing on applications for mobile phones? Or on low-cost devices like the curiously iPad-like $35 laptop?

Should we aim to produce a quasi-public-service like FixMyStreet, or tools for accountability and transparency, like Armchair Auditor?

What sort of public data is available in India, both officially and for scraping? And how might we be able to use it to influence Indian public policy for the better?

As always, the aim for the day is to have something tangible, useful and interesting that we can show to people. Even if it’s only a screenshot or a very raw prototype.

Very grateful for your ideas, thoughts and advice!

There are serious problems with the way Government procures, develops and manages websites, and unless we change that, closing down websites will only be a short-term solution. People working in departments will always want new websites to do useful, valuable things, and they’ll usually find ways to make them. To call these “vanity websites” isn’t really fair. And pushing all citizen-facing content onto Directgov, NHS Choices or BusinessLink isn’t really sensible. Such large platforms bring a necessary degree of inflexibility which isn’t helpful to people who are trying to innovate.

I think that, as with many of the difficulties facing Government, IT and the Web, the problems begin with procurement. Government’s traditional suppliers just aren’t very good at making websites. Government needs to make sure that the new breed of suppliers can get their collective feet in the door. We need procurement processes that make it practical for SMEs to bid for work, civil servants who are keen to try a new approach and project management that takes account of the fact that a lot of “best practice” just isn’t, anymore.

But I think the most important thing to bear in mind today is that this response from the Coalition is a reaction to the problems of the past. Lots of Government sites are genuinely bad, and totally useless. And some of them date back to an era where that’s more or less all we could expect: because the ideas, the theory, the business processes and technology weren’t there. Expensive websites were the only websites there were. Bad was the best we could do. It’s a bit like comparing an AMC Gremlin to a modern car and expecting it to stack up.

But that’s emphatically not the case anymore. Technology and software development processes have improved substantially over the last decade. It really is possible to produce exemplary websites at a fraction of the cost that would have been unavoidable 6 or 7 years ago. NGOs and the private sector have seized on these technologies and ideas to unleash a new wave of products and services that have transformed the way we communicate and think about what the web can do. All the Government has to do is start commissioning it.

I hope that that’s what we’ll see happening over the next couple of years — and I especially hope that the problems of the past decade won’t blind us to the extraordinary opportunities of the next.

With thanks to Rory Cellan-Jones for the title!

The Dextrous Web was founded with a specific mission in mind. We want to build exemplary, smart web projects for the public sector. We think that Government can get much more out of the web than it does, and create much better websites than it has, and we want to help make that happen.

We’ve spent the last couple of years doing all sorts of things: from building simple but important websites for consultations and programmes to crowdsourcing datasets for MySociety, munging PDFs into XML to make interactive consultation tools and helping Government departments to put RDFa into their web pages.

We’re looking for a very specific person to come on board. You’re a programmer, you’ve probably been one for quite a while, and you might be the best programmer you’ve ever met. You write code all the time, because you enjoy it. You’ve probably made a website or two that do interesting things. You might have made some desktop applications, too, and you almost always finish what you start.

You display the classic and indispensable qualities of excellent programmers everywhere: laziness, impatience and hubris. You enjoy learning to use new languages, platforms and frameworks, and you’re quick about it. You have a fanatical obsession with usability, and writing beautiful, elegant applications makes you all happy on the inside – but you’re pragmatic enough to JFDI when things need to get finished. You understand that real people use computers too, and that telling the client to learn to use rsync just won’t fly.

More concretely, you’ve made stuff with Ruby on Rails, PHP and Javascript. You know RESTful architectures and the HTTP protocol. You’re an XHTML/CSS wizard, you understand why this comic is funny and if your website got Slashdotted, there’s no way it would fall over. You know your way around a terminal. You’ve used JSON, YAML, XML, ATOM, RDF and all their cousins and you understand them reasonably well.

For the right candidate, we’re offering a competitive salary, limitless coffee, a nice computer to work on and lots of interesting things to do.

If you’re this person, we’d love to meet you. First read the job description, then pop your CV over for us to see. Make sure you tell us about something you’ve made (preferably a website, and where to find it). If more than one person was involved, let us know what your bit was. If there’s anything else you want to tell us about yourself, include that too!

We’re looking forward to meeting you. The closing date for applications is July 30th 2010.

Agencies: Thanks, but no thanks. We’re only accepting direct applications.

There’s lots I didn’t like about the old site. It was never meant to last for long, but lingered for ages. It didn’t really describe us very well: we’ve been thinking about how and where we should fit into the UK’s digital landscape, but the site hasn’t been keeping pace with the ideas. We were overdue for a change.

This site isn’t finished: there’s more we want to add about us, our plans, our team and our ideas about building the kind of web supplier that Government needs and wants. But it’s better than what was here before, and hopefully introduces us a bit more clearly.

As always — if you have thoughts or ideas about the site, or us, or anything else, we’d love to hear them. Please do let us know what you think.

As is often the case with this kind of work, the deadlines were short and the team rushed. The consultation was only five weeks long, and there was even less time to analyse the responses after it closed. With this in mind, we came up with a format that gave respondents a meaningful way to respond, while also making the responses as easy as possible to analyse.

Most of the questions have yes/no answers, and a few have options you can choose between. Each page has a box for free text responses, instead of each question — and the box is limited to 500 characters. We were a bit nervous about limiting input in this way, but hopefully it’s a good compromise. Very few people contacted us with extra things they wanted to say, so it hopefully most people didn’t feel constrained.

Given the number of questions, the length of the text and the constraints of the form, it was really important to make the site as intuitive as possible. The text boxes make it very obvious if you put too much in them. The final form at the end confirms that you really are finished, and the buttons are subtly differentiated using icons and colour. You’ll also find that using the sidebar menu to jump around is safe — even if you’re in the middle of writing a response. We’re using javascript to make sure the form gets submitted. And everything’s unobtrusive, accessible and standards-compliant.

The site was implemented using WordPress. It’s a fantastic platform to work with. There’s no doubt that WordPress’s amazing flexibility is what made it possible to launch this quickly — and on time, and on budget. Everyone seemed pleased with the way it came out (including a real user) so we’re pretty pleased!

I spent most of the morning in Ivo Gormley’s ThinkPublic session. We tried out some of the techniques they use when they’re helping clients to improve service delivery. We had talks from the operators of three sites, and then tried to apply the lessons learned from those sites to our own example scenarios — in our case, an old persons’ home. We took on various personas and tried to apply the good practices from the talks to our own problems. It was a great way to think about things with a new perspective.

One of the sites from that session was particularly interesting: HorsesMouth.co.uk is an online coaching and mentoring site which has established a pretty impressive community of mentors and help-seekers. They take great pains to ensure to preserve their members’ anonymity and to make the site a safe environment to ask sensitive questions. Rather like StackOverflow, but for personal problems. Really good to see. They deserve much more attention than they’re getting.

In the afternoon, I went along to Paul Clarke and Mark O’Neil’s session on bringing together official and unofficial services. I have to say: they were a marvellous double-act. Though I did get appropriated as the representative of all developers everywhere (quite an honour!). They had several Socratic debates followed by discussions covering reliability, incentives, sustainability, ecosystems for innovation — the whole gamut, really. I wish there was video of it. This was immediately followed by another session run by James Munro on “The elephants in the room: the questions people are avoiding”, which addressed lots of the same questions.

It was a really good day, with interesting and inspiring talks and a great mix of people. Mostly, though, there was just a wonderful energy about the place. Enthusiastic, passionate people all talking about practical ways to improve public services. A grass-roots conference, organised by people on the front lines of health services, attended by people who care.

Congratulations to Patient Opinion for their spectacular success. I hope there’s another one next year!

At the beginning of the year, Jeffrey Peel asked us to organise a panel for the day on digital engagement. It’s an area we have some experience of — we made TellThemWhatYouThink.org and ConsultationXML, and are helping the COI to deploy RDFa to describe consultations documents on central government websites. We had a lively panel and discussion about consultation and how to do it, which you can now watch:

And here are my feelings on the subject — in a bit more detail than we could fit in on the day.

Consultation is an area in which there is intense activity. Departments consult on almost everything they do, local government has a statutory obligation to consult residents on a wide variety of issues, and the rise of social media in government has brought into sharp focus the things that the Web makes possible. But despite all this activity — very much including innumerable panels at conferences — we’re not achieving the mass participation in policymaking that is, for many, the goal of digital engagement. So what’s wrong?

I think that some very big assumptions have been made about digital engagement, its potential, and the right way to do it. Everyone’s been excited by the possibilities, myself included, but I think we’ve failed to really look at the people we’re trying to engage, their level of interest, motivation and available time.

The reality is that formal consultation is simultaneously necessary (in that a deliberative, evidence-based policymaking process is valuable) and expensive in terms of the investment of time and energy that people must make to participate. We have conflicting goals: to reduce the barriers to participation — make it quicker and easier — while also maintaining an informed policymaking process. Formal consultation is far from perfect and we should work to make it better, but it’s not obsolete.

The solution we’ve adopted so far is to try and make it easier to dip one’s toes into a formal consultation. This has been valuable, and we’ve learned a lot from it, but I don’t think it’s workable.

Such approaches can substantially increase the number of responses that consultations receive, but they’re usually not the right kind of responses. A formal consultation doesn’t much benefit from large volumes of anecdotal correspondence about personal experiences. That kind of input is extremely valuable, but by the time a policy has reached formal consultation, it’s too late to use it. That kind of engagement has to happen earlier.

It also has to happen more often. It’s simply no good to pick a time — essentially arbitrarily — to ask people about their experience of, for example, public services. A consultation on the NHS probably wouldn’t be of much interest to me 5 years after my operation, but if I’m asked straight away, I’d be much more likely to respond. The issues would be fresh and immediate, and I wouldn’t have moved on with my life.

Those experiences happen all the time. They constitute the raw reality of our society and the value, or lack of it, that Government succeeds in generating for people. They are innumerable, chaotic, disorganised, neverending and personal: just the unstructured, unrepresentative things that you don’t want in a formal consultation, but that have the potential to create real, valuable change in the way ministers, Parliamentarians, policymakers, civil servants and front-line staff do their jobs.

We need to pick apart these strands. First, we must take formal consultation on to the web, away from paper and PDFs, and engage those people who are interested in investing their time and effort in the process. Second, we must embed into government a culture of engagement, so that those who have stories to tell can tell them to the right people at the right time. Engagement is not the exclusive province of web, press and comms teams. It’s everyone’s job, and everyone must make time for it.

After all, “engagement” is just another word for “talking to people and finding out what on earth’s going on”.

Who couldn’t get behind that?

The site is a blend of the US’s equivalent, data.gov, and Directgov | Innovate. It’s got a listing of available data packages, powered by the Comprehensive Knowledge Archive Network, and user-generated lists of apps and new ideas. This is just right: the data you need, combined with a way to promote the things you make and a place to get ideas if you’ve got itchy typing fingers but lack inspiration.

It’s not perfect. Conspicuously missing is an organised way to browse data sets: but that’s coming, along with some other tweaks and twiddlings that’ll improve the site’s usability.

The site is powered by Drupal, with packages catalogued and hosted by CKAN. Meanwhile, data.gov.uk hosts a data store powered by Talis that can scale to 100 billion triples and is hosted on Amazon EC2. The system is federated, so departments can add and control their own data, lots of which is available as RDF, with the remainder downloadable in spreadsheet form.

Speaking of spreadsheets, they’ve even written an app that departments can deploy in-house to convert spreadsheets into RDF (kudos to John Sheridan!) which makes it much easier for departments to produce structured, linked data.

This is all working now, and was put together by the team at the Cabinet Office in the last three months. This is a massive achievement, and it sounds like it’s just the beginning: they have big plans. User submissions for new datasets. Metadata to describe provenance. More data sets on the site. More data as RDF. Organised browsing for packages. Source code releases. The list goes on.

This is such an encouraging thing to see. No expensive procurement exercises for clunky, bespoke sites: instead, we have the right tools for the job, joined together. Simple things that do one job well, combined to form a more complex whole. It’s the Unix philosophy in action.

This is how all Government IT should work.

Our hearty congratulations go out to the team at the Cabinet office, with special thanks to Richard Stirling for spilling some of the beans. I had lots of questions and nitpicks, and every single one of them was answered reassuringly.

They’ve got a plan, and it’s a good one.

In March 2009, the Department of Business, Innovation and Skills launched a white paper setting out their policy on Informal Adult Learning — which is any kind of learning that doesn’t end in a qualification (like Yoga). Part of their policy was to promote existing informal adult learning events, which they did by running the Learning Revolution Festival, in October 2009.

The web and policy teams at BIS had a bold vision for the kind of website they wanted. Not just another dry campaign site, but something more engaging and useful. They approached at the end of July 2009 with the idea of creating a site where users could submit their events for publication, and could search for events in their area.

We thought this was a great idea. We built and launched the site during August, and it ran for the length of the festival.

At the end of October the festival came to a close, and so the site is no longer running under its Directgov URL — but you can check out our demo version here.

Update: We’re now in the process of turning this site into an open source product that people can download and reuse for their own campaigns. Watch this space…