webofevil | (no subject)

China and Iran are criticised for how much the state controls information. How different is it in this country?

It's less different than we’d like to think. For example, a group of computer programmers was trying to get hold of Hansard, the parliamentary record. They asked for it politely, they didn’t get access to it. They ended up scraping it off the web. The parliamentary officials couldn't stand this; they thought they should have a right to control who had access to this information. These computer programmers had a huge battle to get access to this supposedly public data so we could see how our MPs voted and when our MPs had attended debates.

Interview with Heather Brooke, Guardian

I really haven’t the faintest idea what she’s talking about, and by the sound of it neither does she. The entirety of Hansard is on the web so that absolutely anyone at any time can access it—or “scrape it off”, as she puts it—and search it for exactly the information that she’s describing. Whatever battle these “computer programmers” faced either involved some whole other information that isn’t a matter of public record or resulted from massive confusion on their part.

Another option is that the programmers demanded expensive hard copies stretching back years, which no-one was going to be arsed to provide them with, and didn’t know that this information was freely available elsewhere. Still, that’s where an investigative reporter might be expected to shine.

I actually admire Heather Brooke for the part she played in unearthing the expenses scandal but holy shit, if this is indicative of the competence of her investigations, maybe all of that needn’t have taken five years after all.

Flat | Top-Level Comments Only

I think this is an oblique reference to TheyWorkForYou who demanded that Parliament put Hansard online in such a way that they could re-present it more easily. That's a hell of jump from saying that somehow "the public" couldn't see how MPs voted or find out what they'd said in debates.

I find Brooke a know-nothing self-publicist, and this has done nothing to change my mind.

Plus, I don't understand why, exactly, but hacks seem almost pathological in their compulsion to lie about Hansard. I saw Rentoul in the Indy blithely asserting that Cameron's press office got the record changed for a reason that didn't even make any sense.

My reading of this is that TWFY (or a similar org) asked for access to the Hansard API. As it's probably like most Big IT, a bit cobbled together, loaded with private AJAX calls and almost certainly not properly documented, the answer was an understandable "no". So, TWFY just started polling Hansard for the displayed data and built their own API around it. Perfectly reasonable, and precisely the sort of thing I've been involved with in the past. Arguably naughty, admittedly, but hardly the massive conspiracy Ms Brooke seems to want to make it out to be.

Ah, entirely possible. But yes, the displayed data is freely available and in no way subject to the strict controls evident in, for example, China and Iran. Still, good article, everyone.

What she's saying is they weren't given access to the raw data in some easily machine-readable format. Instead, they had to do a lot of programming to convert from the final Hansard HTML back into a proper database (this is what she means by "screenscraping"). I know the TheyWorkForYou people and this is accurate.

I'm pretty sure they didn't demand hard copies.

Obviously this isn't exactly Chinese levels of secrecy, because Hansard is publically accessible, but it takes a lot more work to figure out how your MP voted from Hansard than from TheyWorkForYou, to the extent that most people wouldn't bother.

I think the reason the Hansard IT people were reluctant to release the information to TWFY is because it would show up the poor quality of their programming and website design, rather than due to pressure from above.

Edited 2010-04-19 21:56 (UTC)

Exactly, MySociety went through so much crap getting that set up, and for all the wrong reasons. There're still MPs that want their sites shut down.

Dear Know-Nothings,
The story in its entirety is told in Chapter 5 of The Silent State. Parliament actually threatened breach of copyright against the programmers which certainly fits my definition of draconian levels of control freakery and censorship.

Perhaps next time you'll do your research before defaming someone.

Regards,
Heather Brooke

lol you actually say computer programmers I was giving you the benefit of the doubt that that was a "helpful" Guardian sub.

which bit of parliament precisely threatened this, was it in a position to do so, and did it actually do anything towards doing so other than threatening?

also does your book talk about how computer programmers made an internet of parliaments?

Parliament does indeed own the copyright on what is said in the building. This could arguably be described as “control freakery” and should probably be changed. However, threatening to enforce that copyright but then giving up amounts to neither a draconian measure nor successful censorship.

Meanwhile, I suggest you don’t equate “criticism” with “defamation”. It makes you sound like Michael Martin.

If it is a reference to They Work For You, it's entirely possible that Hansard gave them free access to its code. Not that I'd know anything about it. Heather should ask and find out...

I think TWFY and The Guardian should be invited to guest edit an issue of Hansard each. Piece of cake! I reckon they could both do it in half the time Hansard manage it and have it up on the internet at least twice as quickly.

What do you reckon, Heather? Is this idea a goer?

Also, what about a public API for Guardian copyrighted content, available free of charge to bloggers and whoever?

(no subject)

no subject

no subject

no subject

no subject

no subject

Ripping into a reporter for no good reason

Re: Ripping into a reporter for no good reason

Re: Ripping into a reporter for no good reason

Re: Ripping into a reporter for no good reason

no subject

no subject