QUOTE: "Many of the scientists working with models appear to have
forgotten that science is about testing predictions against data. They
seem to have fallen victim to the trap long-recognised at IBM, where it
used to be said that simulation was like self-stimulation: if one
practised it too often, one began to confuse it for the real thing."
May 7, 2008
http://www.jennifermarohasy.com/blog/archives/003025.html
Last Wednesday was the launch of a new initiative between the University
of Queensland and the Institute of Public Affairs for environmental
research.
There is some re****ting of the program in today's The Australian
newspaper under the title 'Climate Sceptic's $350,000 grant to uni has
no strings attached'. [I have received comment that the article includes
some snide remarks about me - hopefully not related to my critique of
the national newspaper's 'Save the Murray Campaign'.]
The Australian also includes a column by the Perth-based philanthropist,
Bryant Macfie, who's generosity has made the partner****p possible.
The column is entitled 'Blessed are the sceptics' and in it he explains
the im****tance of ****ning the hard light of reason and critical thinking
on our environmental problems, aided by multiple skills and points of
view.
After the launch at the University, Aynsley Kellow, Professor and Head
of the School of Government at the University of Tasmania, gave an
address to member and friends of the IPA at the Brisbane Club. His talk
was entitled 'All in a Good Cause: Framing Science for Public Policy'.
He said:
"The history of science is replete with error and fraud.
Environmental science is no exception. Indeed, this area of science
provides a hyperabundance of examples, thanks to the presence of two
factors: a good cause and extensive reliance upon modelling, especially
that involving sophisticated computer models.
The good cause - one that most of us sup****t - can all too readily
corrupt the conduct of science, especially science informing public
policy, because we prefer answers that sup****t our political
preferences, and find science that challenges them less comfortable.
We would all wish to preserve the spiralled-horned ox, Pseudonovibos
spiralis, because it is on the Red List of endangered species. Problem
is, is doesn't seem to have existed in the first place.
And we might not have minded the apparent planting by US Federal Fish
and Wildlife Department officers of fur from endangered Canadian lynx in
Wenatchee and Gifford Pinchot National Forests in the Pacific Northwest
in 2002.
When found out, the officials claimed that they were merely trying to
test the reliability of testing methods, by covertly seeing whether the
testing laboratories could identify real lynx fur if not told in
advance. Critics suspected the samples had been planted in an effort to
protect the national forests from logging, mining and recreation. The
Executive Director of the Forest Service Employees for Environmental
Ethics termed this response 'a witch hunt in search of a false
conspiracy'.
This Executive Director, Andy Stahl, had what is known in policing
circles as 'form'. In the 1980s, during the controversy over the logging
in the Pacific Northwest, Stahl was involved in sponsoring the
production of peer-reviewed science to sup****t the Spotted Owl campaign
to reduce old-growth logging.
Stahl put mathematical modelling entymologist Russell Lande in touch
with scholars who suppled the data, and then helped find reviewers to
produce a peer-reviewed publication. This was necessary because the only
'science' then available on the spotted owl was an incomplete doctoral
dissertation.
The Lande paper was created to suit the political campaign and was used
together with the notion of precaution to win the day. Whereas it
assumed an owl population of 2,500 and further assumed that logging
old-growth forest would cause its extinction, subsequent research showed
the species was far more numerous and, if anything, preferred regrowth
forest. Regrowth forest provided more prey and more conducive hunting
conditions than old-growth forest.
Remarkably, the leading journal Nature editorialised in sup****t of those
who had faked the Canadian Lynx evidence - which tells us something
about scientific journals.
The combination of the precautionary principle with endangered species
legislation is a particularly seductive one, but it is the use of models
into which value-laden assumptions can be smuggled that is particularly
pernicious - as a recent Australian example shows.
A case involving the Orange-Bellied Parrot in 2006 saw the merest hint
of a parrot, together with some mathematical modelling (and the
precautionary principle) used by the then Australian Commonwealth
Environment Minister to disallow the construction of a wind farm that
was environmentalists' preferred response to climate change, but was
opposed by residents in a marginal Coalition government constituency.
Modelling for the Bald Hills wind farm on the Orange-bellied Parrot
assumed the birds spent time at most of the sites of wind farms in
Victoria, despite the fact that the birds had not been recorded at 20 of
the 23 sites along the coast of Victoria, and despite active searches
having been conducted. Only one or two sightings had been made at the
other three sites.
The authors then assumed that the birds would remain present within a
single wind farm location for six months-the longest possible period the
migratory species could remain at a winter site, and longer than any
bird had been recorded at any site. They also assumed the parrot would
make two p***** through the Bald Hills site. They did all this to err on
the side of caution.
So, while no parrot had been sighted within 50 kilometres of the
proposed site, the minister then acted in accordance with the
precautionary principle (and an election promise) to block Bald Hills on
the basis of ***ulative impact-compounding the precaution already
embedded in the assumptions underlying the modeling.
I have proposed in what I call Kellow's Law that sightings of endangered
species are clustered around the sites of proposed developments. This
reflects not just the cynical uses of endangered species for political
purposes, but partly also the fact that research for environmental
*****sments frequently finds species because the site has never
previously been surveyed.
This 'noble cause' corruption of science - named for the 'framing' by
police of suspects 'known' to be guilty is helped not just by the
virtuous cause, but by the virtual nature of both the science and the
context within which it occurs. Both conservation biology and climate
science rely on virtual science. The former has seen people in check
****rts counting deer scat give way to physicists and mathematicians,
while the latter (unlike more traditional meteorology) has always
involved more computing than fieldwork.
James Hansen, of NASA's Goddard Institute, for example, wrote his
doctoral thesis on the climate of Venus, and - contrary to what some of
his critics might think - it's clear he has never visited another
planet.
Computer models fed by scenarios based on economic models are the norm
in climate science, and when we are dealing with climate impacts on
biodiversity, we are often dealing with species-area modelling fed by
the modelled results of the impact of climate models on vegetation.
It is im****tant to understand the way in which the revolution in
information technology has transformed the conduct of science. Its
impact has come not just in the ability to model complex phenomena of
which scientists a decade or so ago could only dream - though that is
part of the problem. Computer models are always subject to the Garbage
In - Garbage Out problem and they can never be a substitute for
hypotheses tested against the cold, hard light of observational data.
Many of the scientists working with models appear to have forgotten that
science is about testing predictions against data. They seem to have
fallen victim to the trap long-recognised at IBM, where it used to be
said that simulation was like self-stimulation: if one practised it too
often, one began to confuse it for the real thing.
One problem with observational data in areas like climate science is
that they themselves are subject to substantial massaging by computers
before they are of any use. Even data collection, therefore, provides
op****tunities for subjective assumptions to intrude into the adjustments
made to data to make them useful.
This highlights the im****tance of quality assurance processes, and there
are no greater guarantors of quality assurance in science than
contestation and transparency - full disclosure of data properly
archived and of methods, including computer code.
Society deems this fundamentally im****tant when we are dealing with
science such as drug trials, which are conducted under fully transparent
conditions, ideally with separate teams making up doses, administering
them, diagnosing effects and analysing data. We insist on regulatory
guidelines, and we audit laboratories. We know that even when
researchers are fastidious in pursuing impartiality, subjective
assumptions can find their way into what become 'data'.
There are similar requirements imposed by stock exchanges for data such
as core samples relating to mineral resources. Standards govern the
collection, archiving and analysis of data . In Australia, these are
laid down as standards by JORC - the Joint Ore Reserves Committee. Even
then, mistakes occur and there are consequences: shareholder value is
destroyed or created.
In areas such as climate science we have made no similar demands. Data
are routinely gathered, manipulated and modelled by the same research
teams and the discipline has not insisted on anything like full
transparency. Many of the people engaging in this science are then
acting as advocates for particular policy responses. James Hansen is
perhaps the most notable example in this regard, but there are numerous
others, such as Stephen Schneider at Stanford.
The Intergovernmental Panel on Climate Change then allows the same
people to act as lead authors, sitting in judgment on their own work and
that of those who might differ with them. This corrupts the scientific
process.
The work of former mining industry analyst Steve McIntyre in exposing
the debacle of the Hockey Stick controversy in climate science and in
finding that Hansen's computer generation of mean temperatures for the
US had a Y2K problem (that meant that the hottest year ****fted
conveniently from the 1930s to the 1990s) are good examples of what is
needed. But it is significant that these necessary correctives came from
outside the climate science community.
The ****ft of the 'warmest year' in the US was in itself a small change
in the totality of climate science. But most of the mistakes tend to be
in one direction, and that is in a politically convenient one. This
underscores my point about the need for openness, transparency and
sceptical challenging of science, especially where data collection, data
preparation, data adjustment, modelling and interpretation all take
place in the one institution.
Again, it is noteworthy that an amateur scientist, Anthony Watts, is
responsible for a web-based audit of sites that generate data for that
record, and he and his 'citizen auditers' have found many sites that are
likely to have produced a recent warming trend through poor siting or
site maintenance.
It is worth re****ting that Watts visited NOAA recently, and not only was
he given a warm reception, but he found that the walls of the offices of
those responsible for maintaining temperature records were covered with
photographs of the stations he and his sup****ters have photographed.
NOAA is grateful for the work they have done (at no cost to it), and the
result is likely to be better data in the future. But its surface
records continue to be based on flawed instrumentation that is subject
to adjustment and compilation.
I would suggest that the need for sceptical auditing is even greater
when the senior spokesman for the institution concerned is also a
vociferous advocate for a particular policy position. James Hansen
claims to have been muzzled by the Bush administration - though
Republicans were unkind enough to point to the 1400 or so media
interviews he seems to have managed, and he managed to throw off the
muzzle for long enough to endorse John Kerry in 2004.
The point about all this is that, while Michael Crichton once famously
observed that 'data is not Democrat or Republican, it's just data', we
need to ensure we have institutions that prevent data from acquiring
partisan characteristics.
Steve McIntyre was aware of the case of Bre-X, where gold assays were
fabricated and now applies his considerable skills to auditing climate
science-to our enormous collective benefit. The proposition with climate
change policy is that we are being asked to make substantial social
investments in an enterprise that does not have the standards of
transparency and accountability stock exchanges insist upon to prevent
Bre-X situations, nor situations where subjective beliefs have intruded
into analyses.
But to return to the impact of IT on all of this, we must recognise how
the IT revolution has also revolutionised both the conduct of science
and the way in which it is interpreted - the way in which it enters
politics and the policy process.
One of the impacts has been on peer review, the cornerstone of quality
assurance in science. Publication after anonymous peer review in quality
journals does not guarantee that the science is accurate, but it helps
guard against inaccuracy.
Some journals in which key pieces of climate science are published do
not maintain the standards of strict double blind refereeing that we
take for granted in the social sciences. Geoscientists I raised this
with thought that this would inhibit debate between authors and
reviewers that might lead to fresh insights. Perhaps - but if society is
to take such science seriously, such conversations have to be secondary
to quality assurance. We are well past Victorian gentlemen discussing
interesting fossils they have found.
That problem aside, the internet has made it much more likely that the
identity of an author can be tracked down, breaking down the anonymity
that focuses reviewers on the quality of the reason and evidence
presented in the paper.
Indeed, the internet has made possible increased international
collaboration among scientists, while the increasing specialisation of
knowledge has narrowed the circle of likely referees. Not only does the
internet (and cheap air travel) increase the likelihood that authors are
known to potential referees, it increases the likelihood that they have
worked together. The IPCC has assisted this process, by engaging many of
them on a common task and producing that enemy of all good science, a
consensus.
Edward Wegman performed a social network analysis of those working on
multiproxy reconstructions of climate when examining the Hockey Stick
controversy and found that there was a clear network of co-author****p
between the Hockey Stick authors and almost all others working in the
field, including those most likely to have been selected as a referee by
an editor.
There was neither true independent verification of results nor peer
review, and the possibilities for (at the very least) what we call
'groupthink' were great. When Professor David Deming re****ted receiving
an e-mail some years earlier from a senior climate scientist stating
that there was a need to do something about the inconvenient truth
presented by a Medieval Warm Period warmer than the present, the need
for scepticism is obvious.
Scepticism can guard against such results, but unfortunately leading
scientific journals seem to have lost their sceptical zeal and become,
at least on occasions, boosters for good causes. Let me give you two
examples from what many regard as the best journals of all: Nature and
Science.
A 2004 paper in Nature using the species-area model to predict species
distribution in response to modelled climate change (in turn based upon
emissions scenarios) concluded its abstract with a call to action:
'These estimates show the im****tance of rapid implementation of
technologies to decrease greenhouse gas emissions and strategies for
carbon sequestration.' The paper itself presented neither reason nor
evidence for such conclusions.
The problem is confined to neither climate science nor modelling.
Science, for example, not only published the fraudulent research on
cloning of Dr Woo Suk Hwang, but rushed it into print after short review
so that it appeared in an electronic version, accompanied by a press
release that ensured media coverage, on the eve of a key vote in the US
Congress to overturn an administrative order of the Bush Administration
prohibiting the use of federal funds for cloning research. Not only did
it seem such research was more promising than was the case at that time,
but South Korea was seemingly passing the US by.
Not only have leading science journals yielded to the temptation of the
need for 'relevance', but the ramparts of the prevailing paradigms are
now defended using information technology to marshal the troops.
'Swarming' is not confined to partying adolescents in yellow sungl*****,
enjoying their 15 megabytes of fame, but is to be seen whenever ideas
emerge to challenge the consensus. The white cells of the immune system
of the dominant paradigm are despatched electronically, dealing with the
infectious ideas with all means at their disposal, including (but by no
means limited to) typically anonymous posters to internet discussions.
One of the means commonly employed is the use of the term 'denier', a
rhetorically powerful signifier quite deliberately first used (as far as
I can tell) by a couple of defenders of the faith reviewing Bjorn
Lomborg's The Sceptical Environmentalist for Nature. It was used quite
deliberately by Jeff Harvey and Stuart Pimm to liken Lomborg to a
holocaust denier for daring to question the highly questionable
estimates of the number of species extinctions that supposedly occur
every year.
The computer-based estimates of species extinction range all the way
from a few tens of thousands to 50-100,000 (if you can believe
Greenpeace). The actual do***ented number accepted by the International
Union for the Conservation of Nature is around 800 over the 500 years
for which we have records.
While I'm prepared to accept we have missed more than a few, and I'm a
passionate advocate for the conservation of charismatic megafauna (such
as tigers and orangutans), I think the use of the term 'denier' tells us
more about the person using it than about the target. I think the use of
it amounts to an example of Godwins's Law of Internet Discussions, which
holds that eventually someone will liken someone else to Hitler, at
which point rational debate is over. (Implicitly, the person using it
loses).
Unfortunately, the use of the term is rife in debates over climate
change, where those on one side seem finally to have cottoned on to the
point that scepticism in science is actually a good thing, and it was
even used last year by the now minister responsible.
If it has served any purpose, this use of illiberal name calling serves
to remind us of what is needed to ensure that noble cause corruption
does not afflict the science informing public policy.
Those of us who see value in both social democracy and liberal
democracy - who are committed to humanist ideals but are open to
evidence-based reasoning rather than ideology in determining how we are
to advance them - must acknowledge that it is from liberal views of the
celebration of different points of view, and the battle of contending
ideas, that good science derives.
The philosopher of science, Paul Feyerabend, warned that scientists
might engage in all manner of devices - from the rhetorical to the
reprehensible - to have their points of view prevail. It seems to me
that the only protection against any kind of corruption in science is to
celebrate the liberalism inherent in Karl Popper's philosophy of
science, regardless of whether we share his political liberalism -
though separating the two might be difficult in practice. Feyerabend's
prescription was a kind of anarchism and a rejection of any kind of
marriage between science and the state.
As I said at the beginning of this lecture, the history of science is
replete with error and fraud. In science, the best kind of quality
assurance is to celebrate sceptical dissent and to reject any attempt to
tell us that we should bow to a consensus, that 'the science is settled'
on principle - not just even, but especially when it sup****ts our
preferences. Because as Carl Sagan once put it, 'Where we have strong
emotions, we're liable to fool ourselves.'"
--
Warmest Regards
Bonzo
"How does a small increase in a very small component [of CO2] have such
a large apparent effect [On Climate]? The truth is that no one has yet
shown that it does." Don Aitkin
|
