Bryan's Blog 2005/04
Glaciers Indicate Temperature Increase
Back when I started criticising Michael Crichton, I commented on glacial length from what I freely admitted was a state of ignorance.
However, for some real expert opinion we can cite Oerlemans, 2005, in Science who states that:
Mass-balance modeling for a large number of glaciers has shown that a 25% increase in annual precipitation is typically needed to compensate for the mass loss due to a uniform 1 K warming. These results, combined with evidence that precipitation anomalies normally have smaller spatial and temporal scales than those of temperature anomalies, indicate that glacier fluctuations over decades to centuries on a continental scale are primarily driven by temperature.
Which means that the situation is not simple on the Greenland scale. But as usual, the issue with Crichton is he picks his data to tell his tale. The same Oerlemans article uses glacial records from 169 glaciers world-wide (although he admits a bias to European alpine observations) to construct a global temperature proxy based only on glacial length. The reconstruction is split into regions to avoid the European bias.
He finds that:
Moderate global warming started in the middle of the 19th century. The reconstructed warming in the first half of the 20th century is 0.5 kelvin. This warming was notably coherent over the globe.
Although looking at the figures, the coherency recently is certainly not global, with significant advances in many places. However, I'd be very intrigued to see what would happen if we looked at the precipitation records for the areas of significant advances (e.g. New Zealand). It's also difficult to argue that NZ is "continental scale" (neither indeed is Greenland, despite the size it gets on Mercator projection maps).
What I found particularly impressive is the figure with the reconstructed temperature series! I've snipped a bit of it out so you can get the flavour, although you should look at the original article for the real thing and the axis information:
 |
(I've deliberately degrated the image by removing the axis information and the line legend, so as to meet my own fairuse criteria).
by Bryan Lawrence : 2005/04/29 : Categories climate (permalink)
Copyright, Blogs and Fair Use
One of the things I want to do with my blog is often cite research articles, and I often want to include a figure or text from those articles. There is obviously then an issue of copyright that I need to address.
There are no real guidelines1 on what to do, but I plan to do the following unless I find that I'm doing wrong:
I will always cite the original article.
I will never quote a substantial part of any article.
When reusing figures from any article protected by copyright I'll degrade2 them in some way, e.g. lower the resolution or remove legend and/or axis information) to encourage readers of my blog to go to the original article if they want to refer to it or use it themselves.
I hope that in this way I will remain within the fair use criteria ... as usual with these things I wont really know if this is fair use unless someone objects and claims it isn't. It does seem fair to me.
(Updated May 24th and again July 1st, 2005)
2005/04/29 (permalink)
US Likely to cut Earth Observation
Science is reporting that the U.S. are seriously considering cutting back on their global observation programme. It would appear that NASA and NOAA are in what I would call and inverse turf war: neither wants to take responsibility for "non-operational" earth observation missions.
Missions at risk include ones covering:
global precipitation
ocean vector winds
land cover
optical properties of aerosols
seal level altimetry.
by Bryan Lawrence : 2005/04/29 : Categories environment (permalink)
XML Databases
Ronald Bourret has a nice article in xml.com on xml databases, with lots of examples of real live systems using native xml databases, and some of the power of using xpath type queries on semi-unstructured documents (where you know something about some elements of the structure).
by Bryan Lawrence : 2005/04/28 : Categories xml (permalink)
Sparklines
Joe Gregorio in his bitworking blog has some python code for producing sparklines (RFC2397 inline images as opposed to URI referenced images). He makes the point that it doesn't work on IE browsers. Regrettably it doesn't on my kde3.3 konqueror 3.3.2 browser either.
by Bryan Lawrence : 2005/04/28 : Categories python (permalink)
Managing Action Items
43 folders (via dirtsimple) has a nice discussion of how to manage one's action items, particularly in the context of categorising things into "next actions". I liked the following key points about action item lists (paraphrased):
Actions need to be atomic (if not, put them on a project to-do list)
Make them physical actions (not "think about", but "write notes" etc)
Make sure that any dependencies are resolved
If it's not something one is committed to (i.e. learn about something), then put on a different sort of list (e.g. on-hold, learning list, etc).
Make them well defined (begin with a physical verb, e.g. email, call, recode, visit etc).
Make sure the items are complete enough that you know what needs doing and why tomorrow, or next week, when you actually get to doing it.
by Bryan Lawrence : 2005/04/28 : Categories badc management (permalink)
Sleepless in Oxfordshire
Like all new parents, I'm not getting enough sleep, but I'm back at work, sort of. Anyone wanting gratuitous baby photos can look here ...
2005/04/26 (permalink)
Fine Excuse for Digital Silence
This blog will go silent for a couple of weeks now, as our first child arrived on Tuesday night, and will be home from the hospital shortly. Everyone tells me I'll be too tired to do anything. I believe them.
 |
2005/04/14 (permalink)
To trac or not to trac?
... that's the question!
There are a host of blog entries about trac out there: (1, 2, 3, 4 ... and more).
The bottom line appears that everyone loves trac, but installing it is a nightmare. It runs on subversion, which is a good thing, but a raft of packages are needed ...
Right now we are desperate to find a better way of managing our disparate activities and coding for the NDG ... and this looks good. If anyone has a better idea, I'd like to hear it!
by Bryan Lawrence : 2005/04/11 : Categories management ndg badc (permalink)
Satellite Temperature Trends
There has been some contention over exactly what the satellite data tells us about temperature trends. In particular, analysis of data from the MSU instruments aboard the NOAA satellites has been interpretted in a number of ways.
Nathan Gillet gave us a seminar today where he reported work done on repeating the Fu et al 2004 analysis (in a model). Nathan's work (coauthored with Ben Santer and Andrew Weaver) showed that the Fu analysis gave a good estimate of actual trends (comparing pseudo obs in the model with actual model values). That being so, the methodology should work in the real atmosphere, which means that the satellite obs are consistent with tropospheric warming and stratospheric cooling associated with greenhouse gas induced climate change.
So, he's supporting the Fu et al analysis. The bottom line of that analysis is that if you take just the T2 channel of the MSU, the "tropopsheric" channel, you don't see much trend. However, T2 extends1 into the stratosphere, and so it's "contaminated" by the stratospheric cooling. However, Fu and et al (verified by Gillet et al) have shown that you can untangle the stratospheric signal, and when you do that, the satellites show clear tropospheric warming signals. Here, is the key result from the Fu et al paper:
 |
What you see is that in case a) the analyses (two different methods) from satellite don't give as large a cooling trend as the surface observations (i.e. this is the result that the greenhouse skeptics keep reporting). However, if one corrects for the stratospheric influence on the T2 channel, one gets the results in b). That is, the satellite data supports the surface analyses (albeit weakly in the SH for the UAH analysis).
by Bryan Lawrence : 2005/04/11 : Categories climate (permalink)
Suse 9.3 Looming
My personal computational environment consists of my work laptop, and my home computer. Both run Suse 9.2, and on the whole I'm pretty happy with Suse, I've been upgrading regularly on my laptop every six months (on the Suse release cycle) for years now ... and everytime it's got easier (the first few times I had to do kernel rebuilds and all sorts of hacks to support my laptop).
9.3 is coming now, and I've preordered my copy. I more or less have to upgrade on my laptop because I've broken Yast (no I don't know how, and haven't bothered to find out how), and I want to to be able to do security upgrades automagically ... I need that, because my laptop gets connected everywhere. There seem to be some other good reasons too:
xen seems to be an excellent virtualisation tool, and we need something like that in the BADC. We're about to deploy a development system to support multiple O/S instances, but they're all Linux based, so xen may save us the vmware cost and overheads.
OpenOffice 2.0 (ok, a pre release) ... just last week I had (yet another) a nightmare with a Word Document ... neither myself (running Word-2000 on Cross-Over Office) nor my colleagues (running Word-2003 on their XP systems) could remove a footnote that had spuriously appeared ... until I found the problem using CrossOver office (which reads more and more of the docs I get by email that Word-2000 does not) ... maybe with Suse 9.3 now will be the time for me to change to OO for good ... (I'll keep CrossOver office on until
I can get korganiser to communicate to our exchange server (and lose the Outlook calendaring support I currently use crossover office and outlook for), and
I find something on linux as good as visio ...)
Adobe pdf reader version 7 ...
I have to confess once upon a time I would have been downloading these things and playing with them already ... but I seem to have less and less time for mucking with my system ... (or desire to do so) ... I think this means that the systems are better, and i'm becoming more and more a candidate for a Macintosh.
It's not so obvious that I want to upgrade my home computer. After much mucking around, I've got the multimedia situation where I want it, but it sounds like multimedia support in 9.3 is broken without some package downloads ... and as I say, my mucking with the system threshold is that much higher now :-)
by Bryan Lawrence : 2005/04/10 : Categories computing (permalink)
Arctic Climate Change
I've just found the Arctic change site via Overland and Wang, GRL, 2005. The website has some excellent graphs of climate change indicators for the arctic region. Two of the most interesting (my definition, there are lots of others that you might find interesting, go look ...) are
 |
which is the Tundra area based on raw satellite data and on a classifcation scheme using two alternative input temperature datasets (I have a local copy of the image in case that goes missing).
Also interesting is the sea ice extent:
 |
(Similarly, here is a local copy).
by Bryan Lawrence : 2005/04/10 : Categories climate (permalink)
Glacier Retreat
The NERC quarterly mag1 reports that eighty-seven percent of glaciers in Antarctica have retreated since 1940, based on an analysis of maps of the continental ice-sheet margin.
Mind you, the 244 glaciers analysed are all in the vicinity of the Antarctic peninsula. As the article states, the glacial retreat pattern has been different from that of the ice shelves, which suggests that warming may not be the sole driver of retreat in the Antarctic Peninsula, but that other boundary conditions may be playing a significant role. nothing new in that, but the skeptics always need to be reminded that we don't think it's all about climate change all the time ...
(Update; April 29th: These results are published properly in Cook et.al.,2005).
by Bryan Lawrence : 2005/04/08 : Categories climate (permalink)
Another excellent summary of climate modelling
The Institute of Physics commissioned Alan Thorpe to explain how predictions of future climate change are made using climate models. They did so hoping
that the paper will increase believability in these models and be persuasive that anthropogenic activity is likely to be causing global warming. It aims to convince policy-makers, the general public and the scientific community that the threats posed by global climate change are real.
The paper (pdf) begins:
Like the weather, everyone has a view on climate change but, as will be discussed, not all such views, ... are equally defensible on scientific grounds.
Climate change is a fundamental problem involving basic science including physics ... There is little doubt that a lack of knowledge about how climate change is predicted and the associated uncertainties are amongst the main reasons for ill-informed comment on climate change.
and I think that's the bottom line. People need to understand how it's done, and the real limitations, and lack there of ...
Later on we have
Scientists are appalled that they could be suspected of distorting the evidence to enhance their reputations or funding opportunities. Of course scientific hypotheses and analysis can be refuted by later discoveries but this is not the same as complicity. The fact that everyone experiences weather and climate may explain why nonscientists feel confident in attempting to refute the scientific evidence. The complexity of the climate system and its many interacting and compensating physical processes means that simple arguments that gloss over this complexity have to be approached with a significant degree of scepticism.
Hear, hear!
A common method of arguing starts by identifying a single cause or physical process that either has not been included or has been included in an imperfect way, into climate models. But the climate changes because of a multiplicity of interacting processes and any one process alone cannot be the whole story. ... Climate modellers attempt to include in the models all the processes that are even remotely likely to have a detectable effect, any newly discovered process will quickly find itself incorporated into the models!
As an aside, RealClimate has a nice discussion of one such single process issue that causes a few contrarians grief: the water vapour feedback/forcing issue. Well worth a read.
by Bryan Lawrence : 2005/04/08 : Categories climate crichton (permalink)
Loaded Dice
Suppose a weight is added to a dice, and detailed physical calculation predicts that this will increase the number of 6s at the expense of 1s, but a sceptic refuses to believe the calculation: how many times do we need to throw the dice in order to convince the sceptic?
We all know that if I throw a (fair) dice, I have a one-sixth chance of any given number. If I throw it enough times, the expectation is that the average of all my throws should be
Imagine now that we have a dice loaded so that it is more likely to throw a six (3 times in 12 throws) and less likely to throw a one (1 times in 12), so that my expectation is that
Now, let's ask the question, if I gave you one of those two dice, but didn't tell you anything about the loaded one apart from the fact that
it is loaded with an expectation that if you threw it many times, the average would be 3.9
Could you tell the difference, i.e. which one I had given you?
Well, simple probability suggests the error in your estimate of the expectation would be, for the first dice:
Similarly, allowing for the altered weights, and altered mean,
(not much difference then). For n throws of the dice, the error in my estimate of the mean value of the dice is
So, to distinguish between them, we need to throw the dice enough times that the error is less than 3.9-3.5=0.4, that is we need
throws (using the largest estimate for σ).
What this tells us is that, on average, seventeen throws of the dice should be enough to tell which dice it is (at the one sigma level). Of course, any given sequence of throws may, or may not, conform to the probability. It is entirely possible that one throws a six, every time, with both dice ... all this is saying is that after seventeen throws of the dice, most such sequences will end up with an average value that will indicate which dice is used.
Of course, to be properly sure, we would recast this in terms of a null hypothesis, and depending on whether you can argue that this is a one or two tailed problem you might end up arguing you need twice or four times as many throws to reach 97% confidence ... Even better, we could be more proper and do a proper chi squared test, and if we knew that the loading didn't alter the 2 to 5 values, we could only test on sixes etc ...
... but the argument is much simpler without the statistical test. The key point is that to know whether the dice is loaded, you do have to appeal to statistics!
For the record, I used the following piece of python to do the calculations:
## Loaded Dice
## BNL, April, 2005
import Numeric as N
import math as M
def sigma(x,p):
e=sum(x*p)
s=M.sqrt(sum(p*(x-e)**2))
return e,s
x=N.arrayrange(6.)+1 # dice has sides one thru six.
p1=N.ones(6)/6. # normal dice probabilities
p2=p1.copy() #loaded dice probabilities, I needed to use copy,
p2[0],p2[5]=1./12.,3./12. #else changing part of p2 would modify p1
e1,s1=sigma(x,p1)
e2,s2=sigma(x,p2)
print 'Expectation values for dice one and two are: ',e1,e2
print 'Sigma values for dice one and two are: ',s1,s2
print 'Number of tries required:',4*(int((max(s1,s2)/(e2-e1))**2)+1)
***
highlight file error
***
by Bryan Lawrence : 2005/04/07 : Categories climate (permalink)
Predictability and Crichton
Well, back to that book :-)
On page 284 we find Crichton (or his characters, it's fiction damn it), totally misunderstanding the nature of predictability. He reinforces it absolutely on page 570 in his authors message (where we can't blame it as fiction). He then goes on to make his own prediction for the amount of warming over the next century: 0.812436 degrees C, and then states (my emphasis):
There is no evidence that my guess about the state of the world once hundred years from now is better or worse than anyone elses's ... we can't ... predict it. ... we can only guess ... an informed guess is just a guess.
And there we have it. The last word from Crichton. I think this blog entry on Crichton will be my last on the book per se, although I will take up one more issue from it later (that of over-attribution of weather events to climate change, but that's for anotherday). But, it's a good one to finish with, because these snippets highlight how little he's understood.
So, what is climate predictability. Let's start by what it's not. It is not deterministic predictability. What is deterministic predictability? If I launch a missile at the moon, with a specific velocity, then I know to an enormous amount of accuracy1 where that missile will go. This is an example of a system where the state of the system is (nearly) totally described by deterministic equations and entirely predictable from the initial condition (launch position and velocity).
If we examine our missile example a bit further, we find that our equations consist of equations of motion and coefficients that depend on the planet mass and mass of the moon. You can think of the mass of the planetary bodies as being boundary conditions. Clearly, if I launch my missile in another planetary system, I'll have the same equations, but different coefficients, but you can rely on my predictions of what will happen there because I've tested my system rather accurately here with earth/moon boundary conditions. The key thing here is that the system is totally constrained by the initial conditions (given the boundary conditions) ...
Now let's make the situation just a lot more complicated. Instead of just having the equations of motion involved, let's add a thermodynamic equation, and recast the equations for a fluid medium not a particle. At this point we have a system of equations called the Navier Stokes equations. These are the foundations of everything we know about aerodynamics and hydrodynamics. People design aircraft wings with these things ... but they describe a system that is not always deterministic, that is, it is possible for there to be states where the initial condition does not allow predictability of the system state some time ahead (but sometimes it is, more of this below).
If we do some scale analysis of those equations (i.e. deal with the fact we want to solve these equations on our rotating planet and we have only finite computing), we end up with a system of equations we call the primitive equations. If we add an equation of state for water in various guises, we then have a system where the fundamental equations are as well understood (and as reliable) as our missile equation, but the nature of the things that they can describe has changed significantly. If we add some equations for radiation, clouds, and to simulate the affect of scales not resolved we have a weather prediction model.
The predictability of weather is well described in a met office page on the concept of ensemble prediction. Go read that, and come back ... :-)
The key points to understand from the met office article are, that for the weather system
initial condition predictability is poor beyond a week or two, but
it does support some elements of predictability, and
you can make probabilistic forecasts based on averages which are better when the system is in some states than others.
But weather isn't climate. Climate is about averages. We inevitably want to know about climate at times a long way past that initial condition predictability of the system. In the same way as the planetary boundary conditions affected the missile, they affect weather and climate, and in particular, the boundary conditions dominate over the initial conditions for climate.
Why is that? Well, consider another simple system. A dice. At the risk of pushing my analogies, consider the average value of a series of dice throws to be a proxy for the climate state. Clearly the initial condition of how I throw the dice is pretty irrelevant to what I get on any throw, that's dominated by the physical nature of the dice. A fair dice ought to give me a sequence of throws which average to 3.5. But if I load the dice then I can get a different average value.
Back to the climate. The boundary conditions on climate depend on the timescale you are interested in and things for which you are interested in knowing the climate. For example, if I want to know the global average temperature, then CO2 is a boundary condition (although it may not be a boundary condition if I chose to ask what is the amount of CO2 in the atmosphere on average ... that too is a climate question).
So, I can ask the meaningful question: what will the global average mean temperature be if I double CO2, and it's a boundary condition problem. I can make a prediction. But to evaluate whether my prediction is true, I have to look at an awful lot of cases (throws of the dice), and it'll be a statistical analysis in the end that will validate my prediction (to a certain level of confidence).
This requires me to understand the nature of the errors in my prediction, so I can do the test. That's why projects like http://climateprediction.net are so important.
However, the bottom line here is that no such analysis of the climate can ever give Mr Crichton the level of precision in prediction he wants. It's not a simple engineering problem (like firing a missile), and so on page 284 and at the end of his book, Crichton shows that he just doesn't understand the nature of predictability.
by Bryan Lawrence : 2005/04/07 : Categories climate crichton (permalink)
PDF Heads down hill?
Can this really be true?
It implies that pdf's can be contaminated (tagged) with what I would call a virus: a piece of digital rights management code that is fired up when a pdf is read ... and potentially stops it being read ...
update: ok, so virus was a bit over the top, but you know what I mean
by Bryan Lawrence : 2005/04/07 : Categories computing msxml (permalink)
xmlbase, xinclude and schemas
I found Norman Walsh's excellent discussion of issues with xmlbase, xinclude and validating schemas via Tim Bray.
xmlbase is the standard which supports the anchoring of relative xlinks from a document, e.g. using the Walsh examples, the method that expands to find the full URI of the following image
<imagedata fileref="picture.png"/>
Walsh is making the argument that the schemas are broken, although one of the comments states:
Personally, I don't consider this all that big a deal. Schema-validity is vastly overrated. I routinely add markup to my documents that is not accounted for by the schemas, and as long as you don't blindly throw away all invalid documents, everything pretty much works ... If you really need validity and XInclude, then you need to update your schemas/DTDs to support xml:base everywhere. It's probably a good idea to do that anyway.
The bottom line appears to be that
Your schema needs to support xmlbase
explicity ...
Regardless of all these things, the Walsh blog entry is a nice summary of how it all works ... and links to a good discussion of pipelining from xml documents to bring them together and process them to output. I guess I hadn't appreciated that the schema validation needs to be done after xincludes are completed.