The idea that "information" should be freely available has become a central feature of what, in the 21st century, we expect from a well-functioning democracy. Words such as "openness" and "transparency" litter the rhetoric of public policy discourse. They are also characteristics that for many define good science.

Why, then, has Freedom of Information (FoI) legislation, a key tool in facilitating openness and transparency, developed such an uncomfortable relationship with UK scientific research? Were researchers "asleep at the wheel" when FoI came calling? Or, to turn the question around, was FoI legislation ready for science?

The UK's Freedom of Information Act, which was passed in 2000 and came into force on 1 January 2005, aims to ensure that public bodies are publicly accountable. It legally obliges public authorities to provide information in response to FoI requests within 20 days.

Although some would argue that it is still not used widely enough, FoI seems to have been embraced by at least some parts of the British public, with about 30,000 requests for information flooding in each year. The majority of these requests are aimed at local and national government - with perhaps the most famous example being the publication of MPs' expenses in 2009 after a protracted legal battle fought by journalists and FoI campaigners, although ultimately, the information was leaked.

ÖйúAƬ

However, universities can be the target of FoI legislation too, as demonstrated by the illegal release of emails from the Climatic Research Unit (CRU) at the University of East Anglia last year, which revealed controversial correspondence between researchers on the subject of FoI requests for climate change data.

In another high-profile clash between FoI campaigners and academia in April this year, Queen's University Belfast was ordered by the Information Commissioner to hand over 40 years' worth of research data on tree rings after a three-year campaign by a climate sceptic.

ÖйúAƬ

At East Anglia, a seemingly endless series of inquiries has established that none of the more outlandish allegations made against the staff was true: data were not fabricated and there was no "smoking gun" that proved climate change to be a lie. But as even the staunchest supporters of the beleaguered research team at the CRU have had to admit, the way in which FoI requests were dealt with was unsatisfactory. The Independent Climate Change Email Review found this summer that the scientists did show "a consistent pattern of failing to display the proper defence of openness", particularly around FoI.

According to Fred Pearce, the environmental journalist who covered the story in most detail, an important lesson emerged from the debris of the "Climategate" debacle: science simply was not ready for FoI legislation.

"Science as a community did not see that the FoI laws would impact on science. Like many of us, they thought it was designed to uncover self-serving Whitehall secrets and our own personal files (held by public bodies). I regard the failure to see what was coming down the track as a real failure of the science community."

Similarly, an event on FoI and academia held last year by the Research Information Network noted "lethargy" when it came to researchers' approach to the legislation - not only in terms of their reactions to outside requests, but also in making use of FoI in their own work (see box, below).

At the root of all this is a profound question about the nature of scientific knowledge itself. Is a request for "information" a productive way of opening up scientific knowledge?

The sort of knowledge that can be easily extracted using FoI requests is far-reaching but also inherently limited to information that is explicit. Numbers, calculations, reference lists - and, of course, emails - can all be placed squarely in the public domain. With enough of this type of explicit information, some aspects of the scientific process can be recreated. If you have someone's raw data, you know the calculations they made and you can see their results, you are in a position to confirm or challenge their conclusions. But to what extent does this fully capture scientific knowledge?

Mike Hulme, professor of climate change at the University of East Anglia, argues that scientific knowledge is built on more than mere data, computer codes or theories.

"With complex issues like climate change, sophisticated forms of expert knowledge assessment are necessary to weigh conflicting, incomplete or ambiguous evidence. The Intergovernmental Panel on Climate Change (IPCC) is a good example of this. Of necessity, such assessment is discursive and deliberative, and cannot be captured in data, theory or even in formalised recorded words. Here, FoI - if it is being used to reveal the foundations and construction of knowledge - reaches its limits. If scientific knowledge is to continue to warrant public trust, then expert deliberations, eg, the IPCC, should be made public events."

ÖйúAƬ

Hulme - who has publicly criticised the CRU researchers involved in the Climategate affair - is clearly not suggesting that the norms of scientific research provide a means of wriggling out of accountability. But at a fundamental level, the nature of scientific "knowledge" seems at odds with the "information" that can be revealed through FoI requests. FoI legislation can skim the surface of knowledge and cream off information - but to get at the heart of a scientific dispute requires scientific expertise.

Anyone can audit a list of expenses. Only someone with the right knowledge can settle a technical dispute in a satisfactory way. Expecting FoI requests to be able to arbitrate between competing knowledge claims is no more plausible than asking the social services to use FoI to uncover instances of "bad parenting" - it's the wrong tool for the job.

At a very practical level, FoI also seems unsuited to the current institutional culture of scientific research. Many scientists are uneasy about sharing their precious intellectual resources unless it is on their own terms - not least for fear that they will be misrepresented.

"I think there's certainly a tension, in that scientists see freeing their information rather differently to how FoI works," says Martin Griffiths, national coordinator for science journalism training at the Royal Statistical Society. "FoI goes beyond just the data and allows the release of correspondence between scientists. These may not make much sense to outsiders and lead to the kind of language problems we saw in Climategate - with (the word) 'tricks' (being used) and so on - that are everyday parts of science being blown out of proportion."

Gabrielle Bourke, a researcher at University College London's Constitution Unit, makes a similar point about the tensions between academic work and FoI.

"The ways in which scientists go about their work, such as peer review, don't necessarily sit very well with the FoI policy," she says. "The usual way is not to hand over your data before you have done work on it."

Martin Robbins, a science writer, posted a provocative blog post in April arguing that he "couldn't see why academic data should be covered by the FoI Act". He railed against the idea that because the public have paid for research, they should have access to it.

"We actually haven't paid for it," he wrote. "The public pay for research to be done. They do not pay for peer review, or publication, or data archiving, or indeed any sort of public dissemination of information except where it's explicitly set out in the funded proposal."

In addition, while most research is publicly funded, not all of it is - and the FoI legislation does not apply to research funded through private channels. This means industry lobbyists or other groups can use FoI as a tactic to delay publicly funded research programmes.

A great deal has already been written about the motivations of the army of bloggers who bombarded the University of East Anglia with FoI requests. Were these part of a genuine desire to find, use and share information, or designed to mess researchers around and stop them from getting on with other work?

Legislation designed to make publicly funded research open and accountable may be abused to suppress particular pieces of work - potentially for political ends. This problem was perhaps made most evident in the controversy surrounding the US' Data Quality Act. And at the University of East Anglia, it certainly seemed as if some of the FoI requests satisfied the Information Commissioner's definition of "vexatious", although other (apparently legitimate) requests do also appear to have been ignored by the scientists.

Of course, many public bodies could construct a compelling argument for why they should be exempted from the demands of FoI, but there does seem to be something special about the academic case, because of the nature of scientific knowledge. However, because the academic community did not spot the possible implications of FoI for scientific activity when the legislation was written, says Pearce, "we now have law that is so badly drafted it makes no distinction between the needs of scientific discourse and the demands of angry bloggers".

ÖйúAƬ

UCL's Bourke notes that in the US, there is a set of exceptions for universities, which came into effect in 1999 after a consultation process in which the universities participated.

"Maybe UK academics and academic organisations just need to have a conversation about FoI," she suggests. "Just because there is a tension doesn't mean universities and FoI can't be reconciled. Indeed, there are lots of movements to open data."

To return to the question of whether science was prepared for FoI or the other way around, the answer seems to sit somewhere in the middle. At a minimum, the bodies controlling publicly funded science were not engaged enough with the policy process. In 1999, while UK academics were ignoring FoI legislation, their US counterparts were helping to improve theirs. Since then, there is evidence that even in the political domain, FoI is not the silver bullet that its proponents envisaged.

In a review of the impact of FoI legislation on UK governance, Ben Worthy, also from UCL's Constitution Unit, argues that while FoI has achieved the core objectives of increasing transparency and accountability, it has not increased public participation in governance issues. According to Worthy, FoI is simply not a powerful enough tool to tackle the complex, deep-rooted issues that prevent increased participation, understanding or trust.

While his argument is aimed at governance, the logic applies just as well to science.

FoI legislation alone cannot be a panacea for public trust in science. A meaningful relationship between science and the world beyond the ivory tower is unlikely to be predicated on the superficial level of transparency achieved by FoI requests. At best, they will act as a useful audit of data, uncovering the odd mistake or oversight. But left alone, there is a danger that public engagement with the scientific process will be reduced to an exchange of mutually distrustful FoI correspondence, or a rather limited interaction between the public and the university PR office.

Science needs to learn to live with FoI, regardless of whether it can deliver a meaningful assessment of scientific knowledge to the public. But the key challenge is for scientists to find ways of increasing openness that are more proactive and less confrontational than the FoI process (see box, below). Dialogue-based processes of deliberation and interaction are more powerful ways to build trust between scientists and the public, and the questions that can be asked can go deeper than an acrimonious audit of a model or dataset.

The role of the public in shaping the work scientists do is critical - public engagement is vital to ensure scientific programmes are not only technically sound, but socially beneficial.

So public engagement with science is worth fighting for, but accessing science through FoI legislation is unlikely to lead to a satisfactory outcome for scientists or the public.

According to a review of FoI legislation in Canada, Australia and New Zealand in 1987, its introduction led to greater scrutiny of ministers' expenses rather than of their management of economic policy. Are we more interested in reading scientists' emails or in shaping the values that guide their work.