To recap, the concept of open sourcing government is essentially about allowing third parties (citizens, companies, non-profits, etc.,) direct access via APIs to government data, so:

that individuals, communities and businesses are able to interact with government web applications in ways that are useful to them.
5 principles for Govt 2.0

The UK Cabinet Office released a report in June last year that looked at what this would mean for their public management system. The economic impact alone, as the report makes clear, had the potential to make a significant contribution:

2006 figures from the Office of Fair Trading that estimate that improved availability of information to re-users could double the direct market value of public sector information to £1.1 billion per year.
The value of government information

The Princeton paper authors, however, seem to be going some way further, calling for the US federal government to reduce the role it plays in presenting online information to citizens. The authors argue that the key role for federal agencies should be opening up their data, rather than building websites that provide a platform (and by inference, not a particularly good one) and a filter for people to access the information.

Their premise is an attractive one. That government

should focus on creating a simple, reliable and publicly accessible infrastructure that “exposes” the underlying data. [... The Government should] require that federal websites themselves use the same open systems for accessing the underlying data as they make available to the public at large.
Government Data and the Invisible Hand, p1.

Effectively, this means that government agencies could focus their attention (and taxpayer dollars) on the quality and accessibility of the information, and not on the presentation level. It’s a compelling proposition, particularly when you consider the current state of the namespace (in any jurisdiction, really).

The authors argue that the market, in the form of these third parties developing applications that reuse the government data, will drive economies of innovation and accessibility that are beyond anything that government itself could hope to provide. They list some of these advanced features:

• advanced search
• RSS feeds
• links to information sources
• mashups with other data sources
• discussion forums and wikis
• data visualization
• automated content and topic analysis
• collaborative filtering and crowdsourcing analysis

It is worth pointing out that some of these advanced features are already part of the namespace here; albeit with mixed success. Advanced search, RSS feeds, and wikis are all essential elements of the .govt.nz space.

There are a couple of concerns that I have. The authors acknowledge that in some case there will be no private actor willing to step forward and create a compelling website based on the data. The notion that government makes all information accessible, irrespective of it’s apparent value, is a fundamental one. Abrogating that responsibility to third parties seems fraught with potential to disadvantage some sections of the community. Mike W leaves a comprehensive comment to this effect on Ed Felten’s blog (one of the authors of the paper).

As an aside, ensuring that people can access a government data set via a visualization (like EveryBlock, for example) using a screenreader can be done, but it is hard work…

More of a concern, however, is the notion that we can either continue to try to build usable websites or simply outsource innovation in the namespace. I don’t see, as the authors apparently do, that the two are mutually exclusive. Indeed, there is an argument that government should retain and build more capability for innovation, rather than adopt practices that would encourage this sort of activity to atrophy.

Those concerns aside, however, what the authors of the paper are proposing is both an effective and efficient approach to transformed government. We should be seriously considering the same here in New Zealand.

Photo: kool_skatcat

Of course, this is not limited to the public sector but, given our obligation to make information available and to keep it available as outlined in the Policy for Government Held Information, we are in a particularly invidious position.

One of the obvious ways to address this content crush is tagging. However, I can’t help feel that this is only part of the answer to the problem or, less optimistically, at best a short term solution.

With only 600 odd items in my delicious account, I already feel a sense of dread at the cognitive overload when I look through my list of tags trying to locate a page saved a couple of months ago. Looking at Hamish’s bookmarks renders me dumb with equal parts awe and despair.

The Answer

As I suggested in 5 Principles for Govt 2.0, the answer is Search. Or, in Jon Udell’s far more elegant phrase,

we’re in the midst of a long transition from container-based to query-based storage and retrieval
Tagging and Foldering.

An important point to qualify here is that the term long transition shouldn’t be taken to imply a process that is either leisurely or painless; in government, we can rest assured it will be neither.

I should also point out that, for the government namespace, the demand side search issue has already been solved. The new search on newzealand.govt.nz is frighteningly good (if you are a content owner in this namespace, just search your domain and you will see why it is frightening — everything you have posted in now surfacable).

As an example, when I wanted to pull up the Government Held Information Policy, I searched the SSC site using the, what I thought, obvious terms: “government held information” “policy framework” and finally (I am a lazy typist) I went for the whole phrase. Result? Nothing on the first page of the three searches. An advanced search on newzealand.govt.nz, using the phrase “government held information” restricted to “ssc.govt.nz” and, bingo! straight there.

Supply Side Search

The real issue for government is not the search tool. It is preparing the content for search — making it discoverable. This is where we will have to reinvent business processes and essentially apply a new model to the creation and publication of government held information.

However, until we have XML editors installed across government, and we continue to rely on the thoroughly broken model of trying to convert Word documents to valid HTML, then we will need to pay far more attention to these supply side techniques.

So, in the meantime, we need to focus on:

• semantic markup
• seo
• folksonomies, and last but most importantly
• validate your pages.

All these techniques will address the symptoms, but to get to the heart of the issue will require a more fundamental and profound change.

Conclusion

The exponential growth in the amount of content on government websites should be viewed as an opportunity to transform the namespace by creating social and economic value. There isn’t much point talking about the long tail if it is squashed under a fat rump of poorly marked up, irrelevant and dated or superseded content…

Photo: singleframe.