... personal wiki, blog and notes
Bryan's Blog 2009/01
Reading in 2009, 4: Engleby not for me
I have a t-shirt that reads "too many books, too little time".
Every now and then I read a book and think, "with all the books in the world, why did I bother with this one?" Despite the amount of trash I read, this happens relatively rarely, if a book is entertaining, or, interesting, or both, then I'm usually a happy bunny, and I don't set the bar high.
And so to Sebatian Faulks' Engleby. Half way through I seriously considered not finishing it. It starts well enough, but it doesn't take long to suss the plot, and despite the odd passage of quality prose, mostly the mood it builds up is tedium, and a feeling of "when is this thing going to end" instead of the any of the feelings I would rather have had, like "how is this going to end" (frankly I didn't care) or "give me more" (please no more) ... or "how time has flown while I've been reading" (nope). So, sadly, this was a "I wish I never started it book". Having started a book, it has to be really bad for me not to finish it, and this wasn't really bad ...
If you ever make the mistake of reading it (all the way through), you'll realise that tedium may well have been exactly what Faulks wanted you to feel, but being true to your plot while boring your readers is something you can only get away with when you have a big reputation.
So, I'm sure some folk will like it (a quick look at Amazon seems to imply that lots of people liked it!). But not me. I wish I had spent the time reading something else.
(I hope that when I grow up I'll learn to not finish books I'm not enjoying!)
when the paparazzi are ok
I stumbled across this a week or two ago, and have had it sitting tabbed waiting for a response since, because it really got my goat!
The basic thesis of the writer is that it's a bad idea for someone to wander round a poster session at a (scientific) conference, snapping away at the posters using a camera. Leaving aside the issue that I thought I was the first person to think of doing this with a camera phone (obviously not), like I say, it got my goat!
His justification of his position comes down to the fact that he sees "taking" (information) without "giving" (feedback) as not keeping up with the takers part of a two-way process. He's also worried about what he calls "espionage", and data getting discussed before it's peer reviewed.
Firstly, as to the taking without giving: In some communities, presenting is the price of attendance, the feedback is incidental. In all communities only a tiny percentage of attendees ever give feedback. Does not giving feedback mean I can't/shouldn't listen (to a talk)? Can't read (a poster)? Given how much it costs (in time, money, and emissions) to go to a conference shouldn't we make damn sure we get as much as possible. I could never engage with (or often even read) all the posters at many conferences.
As to the "discussion" before peer review. What's the point of putting an idea into the community if you don't want it to be discussed? (Risk of the data being analysed by someone else I hear him respond? They can't publish it without credible provenance, so what's the issue, the idea was out the moment you took it to a conference?)
Finally, in my opinion, the best conferences make sure the posters (and the presentations if you're really lucky) are on a memory stick and/or in a repository, so I can have access later. If they don't do that, how is not better for me to at least take a copy so I can read it later?
Science is about communication. Anything that hinders communication hinders science. Attribution is important, but a camera copy of a paper doesn't make attribution any less likely than publication, in fact, if we extrapolate from the open access experience, it's likely to make attribution more likely.
by Bryan Lawrence : 2009/01/30 : 0 trackbacks : 0 comments (permalink)
bbc goes to rdf
And not only that, they built their domain model first then built an RDF ontology:
We set about converting our programmes domain model into an RDF ontology which we've since published under a Creative Commons License (www.bbc.co.uk/ontologies/programmes/). Which took one person about a week. The trick here isn't the RDF mapping - it's having a well thought through and well expressed domain model. And if you're serious about building web sites that's something you need anyway.
Someone once said to me that RDF wasn't big out there. Well I knew it was, and maybe he will believe me now!
Reading in 2009, 2: Water Supply
And so to "When the Rivers Run Dry" by Fred Pearce. Which is about what it says on the tin ...
Another apocalyptic read (I'm not in an apocalyptic mood, it just happened that I got two birthday presents last year in the same vein). This book reads well, but it's another one that could get you breaking out the whiskey before the sun gets over the yardarm. It's absolutely not a book about global warming! Although global warming gets a few mentions, it's a book primarily about good intentions going bad coupled with bungled engineering and short term thinking. It is scary precisely because it would appear we're stuffed on the water front before we even get to the implications of warming ...
There are some really fascinating bits in this book, the state of the Aral Sea for example, I guess I vaguely knew what had been going on, but the detail presented in this book is scary, not just because of what has happened, but it (the dry up) was planned that way (and despite all the planning, the resulting water for "use" is being frittered away).
Here are a few bits that I noted (for my own nefarious purposes, not because they were necessarily the most important or most interesting ...). All the numbers (except where stated) are from the book, I don't know what the original sources might have been.
Not enough water in the first place
A back of the envelope calculation (p33-35) of water availability goes something like this: we will run out of water unless we only use the water that falls as rain (somewhere), that is from the "fast water cycle". In practise we only care about that which falls on land (60K cubic km per annum). If we neglect that which evaporates, and that which is transpired (hmm, I'll get back to that), that leaves about 40K cubic km of runoff for "consumption. Of that hydrologists reckon it's practical to "capture" 14K (why?). Take out the runoff in inaccessible places (like Siberia), and we're left with 9K, or about 1400 cubic m per annum per person. But earlier on (p22) he's calculated that he himself consumes around 1500-2000 cubic m per annum in terms of water needed to feed and cloth him (as well as that directly consumed which is far less). So the bottom line is that if everyone wants to live like him, then there's a problem.
But we neglected the transpiration earlier on, and surely that's part of the water consumed to feed him? So I'm not so sure about the budget. However, whether or not he's got the budget details right, the actual efficiencies (or lack thereof) of actual hydrological systems that he discusses throughout the book make it clear that we have a major problem, and we're eating into water from the "slow water cycle" (deep acquifers etc, which are slowly, but surely, being drained).
We can feed them, but can we water them?
(p38) The UN FAO says that globally we now grow twice as much food as we did a generation ago, but we abstract three times as much water from rivers and acquifers to do so.
Dam them all
As a kiwi I both appreciate(d) the benefits of hydro power and mourn(ed) the losses from flooding ... but I always thought of dams as being a Good Thing (TM). However, it appears that it's not always that way: A World Commission on Dams (appointed by the World Bank) made some interesting observations in 2000 (p157-159):
Two thirds of all damns built globally for water supply to cities deliver less than planned (a quarter less than half)
A quarter of dams built to irrigate fields irrigated less than a third of the land intended.
Half produced significantly less power than advertised
an interesting number is the number of kw/flooded hectare: ranging from 0.2 to 5 for some examples he gives. I thought about that a bit: an interesting comparison is that a tenth of the area could provide between one and four times the same energy for the best of these (at 5-20 W/sq m - this last number from the synopsis - pdf - of a new book I want to read).
Even dams built to protect against flooding have increased vulnerabilities (because they're generally kept full and "emergency releases" are floods in their own right).
Dams have resulted in at least 80 million rural folks losing homes, lands and livelihoods!
Many have been poorly sited, often on the basis of faulty estimates of climatic flow (even in wealthy countries like the States: consider the poor future for the Colorado, and Lake Powell in particular - p223 and more recently).
and that's without considering silting and wetland removal etc
Water from thin air
On the positive side! Chapter 31 discusses technologies for "generating" water.
The discussion of water budgets above was about precipitated water. Of course at any given time a lot of water is sitting in the atmosphere as water vapour - roughly 98% of the 13K cubic km in the atmosphere at any one time (about six times the amount in the worlds rivers - again, at any one time).
There is a discussion of dew ponds, and artificial dew producers (using cold ocean water to cause condensation in the desert), and fog capture. Inspiring stuff. (He also talks about desalination and cloud seeding, both of which are rather less inspiring! Even if the former is widely deployed and/or necessary in some places, it's too energy intensive to be a "solution" to the global water issue.)
The bottom line
Actually, just like climate change, the water problem is not just a supply side problem, it's a demand problem too. In the final analysis, we need to drop demand, as well as address changing modes of supply. As far as the latter is concerned, there will be no one solution.
If I got one take home thread from this book it would be that there is a dire need for rational politicians and (more) sensible water managment practises, coupled with geographically realistic assessments of crop suitability. (And on the demand side, less cotton production - and consumption.)
Reading in 2009, 3: Miss Smilla
We've just spent a week on holiday in Cornwall, where, apart from the IPCC tomes (mostly unopened) and Obama souvenier edition newspapers (mostly disguarded unread), my holiday reading was a reread of Peter Hoeg's Miss Smilla's Feeling for Snow.
I first read this about a decade ago, and had dug it out for someone else to read because we'd been talking about excellent translations (and translators). There is an interesting story (pdf) about the translation itself - there being substantial differences between the US and UK editions. Anyway, the book came back, and languished on the floor of the car until a couple of weeks ago, when I was stuck for half an hour in the car, and so I started to read it again ...
Of course a decade was plenty long enough for me to have forgotten the entire plot so it was effectively a fresh read.
The first thing to say about this book is that despite the Danish orgins, even in English, the prose is just fabulous! Mostly I don't care for "fabulous" prose ... when I'm reading a novel I just want a direct connect from text to my brain that doesn't have me realising that I'm actually reading at all ... fancy language gets in the way of that (for me). But this book is different. I can't pull a sentence, or paragraph for you, because I somehow managed to read it in my normal way (ie without being conscious of actually reading), but I still have a sense of joy from the process of reading it. It was clearly wonderful in the original Danish, so all Kudos to the translator(s).
The story itself is a pretty good thriller, with an engaging, resourceful lead character (Miss Smilla) who manages to segue in a nearly believable way from one scarcely survivable event to another. I enjoyed the first half more, as the believability/survivability function fell significantly in the second half, but for all that, it was a good read as a thriller. A majorly unbelievable bit was the reason for it all, but since that only became clear right at the denouement (which, as wikipedia puts it, is "unresolved"), it mattered not.
There is more to it than the thriller and the prose, the glimpses of Denmark and Innuit culture and society and their relationships, with each other and Smilla, weave throughout and give it real character.
Miss Smilla's Feeling for Snow is a seriously good read on many levels. If you even half like thrillers, read it!
SD cards to the rescue
Next generation storage (press release pdf):
The next-generation SDXC (eXtended Capacity) memory card specification, pending release in Q1 2009, dramatically improves consumers' digital lifestyles by increasing storage capacity from 32 GB up to 2 TB and increasing SD interface read/write speeds up to 104 MB per second in 2009 with a road map to 300 MB per second. SDXC will provide more portable storage and speed, which are often required to support new features in consumer electronic devices and mobile phones.
Never mind the electronic devices and mobile phones, my data centre will scale to petabytes without issues associated with air conditioning, pwer consumption and physical volume!
It also removes another worry for me. In 2009 we expect to add between 500 TB and 1 PB of new physical storage (on spinning disk). This is a rather large perturbation to our normal growth, and I had been worried about how we would replace it in four years time. If consumer electronics does what it normally does, then in 2012-2013 we'll be replacing a room full of spinning disk with a rack full of SDXC cards ...
The faster bus speeds in the SDXC specification also will benefit SDHC, Embedded SD and SDIO specifications.
and scientific data analysis!
Hat tip (of all places) the online photographer!
Heat not Drought
Just as my night time reading is all about drought (I'll tell you about that another day), I find this fascinating paper in this weeks Science:
Battisti, David S. and Rosamond L. Naylor, Historical Warnings of Future Food Insecurity with Unprecedented Seasonal Heat, Science (2009)
The bottom line is that heat waves may be more important than droughts for some food production. They give the example of a major perturbation on wheat production and consequential world wheat prices arising from the hot, dry 1972 summer in the Ukraine and Russia. They then point out that while that summer ranked in the top ten percent of temperature anomalies between 1900 and 2006 (with temperatures 2-4C above the long term mean), one third of the summers in the observation period were drier!
They then use observational data and simulations from 23 global climate models to show a high probability (>90%) that growing season temperatures in the tropics and subtropics by the end of the 21st century will exceed the most extreme seasonal temperatures recorded from 1900 to 2006. The consequences for food production are extreme!
A couple of choice quotes:
... regional disruptions can easily become global in character. Countries often respond to production and price volatility by restricting trade or pursuing large grain purchases in international markets?both of which can have destabilizing effects on world prices and global food security. In the future, heat stress on crops and livestock will occur in an environment of steadily rising demand for food and animal feed worldwide, making markets more vulnerable to sharp price swings.
... with growing season temperatures in excess of the hottest years on record for many countries, the stress on crops and livestock will become global in character. It will be extremely difficult to balance food deficits in one part of the world with food surpluses in another, unless major adaptation investments are made soon to develop crop varieties that are tolerant to heat and heat-induced water stress and irrigation systems suitable for diverse agroecosystems. The genetics, genomics, breeding, management, and engineering capacity for such adaptation can be developed globally but will be costly and will require political prioritization ...
Well, I haven't been to a major conference for a while, and I received a raft of invitations to give talks at EGU this year.
So, with colleagues, we have a raft of abstracts submitted:
Two degrees of Warming
William asked what ML thought would happen with two degrees. I suspect the reason he asked that is that most of us believe that two degrees is in the pipeline, and pretty much inescapable now. Indeed, I reckon we'll see it (wrt 1960) within a few decades (wrt now).
Ideally folk should go read the book, but this is the gist of the one and two degree chapters - via section titles (and my parenthetic summary):
America's Slumbering Desert (droughts, soil loss etc)
(An aside on the fact that the Day After Tomorrow hasn't and isn't likely to happen)
Africa's Shining Mountain (fairwell glaciers on kilimanjaro, implications for water)
Ghost Rivers of the Sahara (greening the Sahara, yes, no, maybe, floods and droughts).
The Arctic Meltdown Begins (tipping points for ice, permafrost melt, drying)
Danger in the Alps (mountains and villages at risk of destruction as permafrost melts)
Queenslands Frogs Boil (dramatic biodiversity loss, in rainforests and reefs, in Queensland and elsewhere)
Hurricane Warnings in the South Atlantic (are hurricane characteristics changing?)
Sinking Atolls (bye bye Tuvalu, Kiribati etc)
China's Thirsty Cities (water shortages)
Acidic Oceans (real problems for phytoplankton, and thus everything)
Mercury Rise in Europe (i.e. more heat waves)
Mediteranean Sunburn (fires and drought)
The coral and the icecap (sea level rise beyond the IPCC predictions)
Last stand of the polar bear (arctic melting)
Indian summer (food production decline, water issues)
Peru's melting point (glacier melt leading to water shortage)
Sun and Snow in California (water crisis)
Feeding the Eight Billion (ups and downs in food production, net down)
Silent Summer (climate change too quick for ecosystems, mass extinctions)
There is clearly much more. Please read the book, even if you want to disagree with a few of the details! Obviously ML is cherry picking the literature, but there is much more out there, and I don't think it's unrepresentative!
Some of this echoes what I've always said: In the near future, the climate change risks are not about magnitude and human comfort (hot, cold), they're about climate change speed, and it's implications for ecosystems, water, and food.
curation and specification
Chris Rusbridge has a couple of interesting posts (original and followup) about specification and curation. The gist is that he's reporting some conversations, which I might baldly summarise as saying something like:
Any application stores data and renders it for consumption (by a human or another layer of software). In the best of possible worlds, a specification for the data structure AND the application functionality should be enough to ensure that a third party could render the data for consumption at the next level without reference to the application code itself. However, certain real world experience suggests that the specifications are not enough, you need the code as well, because real implementors break their specifications.
There was some discussion about the ability of OOXML and ODT to preserve semantic content preferentially over latex and PDF ... (primarily I think because some of the key semantic content would have been in figures which could be parsed for logical content, and both latex and pdf would have turned those figures into images).
As a consequence, Chris gets to this position:
So ... running code is better than specs as representation information, and Open Source running code is better than proprietary running code. And, even if you migrate on ingest, keep BOTH the rich format and a desiccated format (like PDF/A). It won't cost you much and may win you some silent thanks from your eventual users!
You'll not be surprised to find I have some opinions on this ...
I think in nearly all the cases where the specification is not enough, it's because the specification was a) not designed for interoperabilty, and b) was not the definition of the format and functionality. In these cases we find the spec is an a postiori attempt to document the format (almost certianly the problem with the Microsoft and Postscript examples discussed in the links). In particular, in those cases where we're dealing with preserving information from a format and specification from one vendor, we find both a) and b) violated, nearly all the time. What that says to me is that we should avoid trying to curate information which is in vendor-specific formats in favour of those where there are multiple (preferably open-source) implementations.
Running code will become non-running code in time, and not much time at that. What I hope Chris means, is keep the source code which ran the application. Even then, every software engineer knows that the code is not documentation, and that with sufficiently complex code, NO ONE will understand it. So, code without specfication is a candidate for obsolescence and eventual residence in the canon of "not translated, not understood, not useful" write only read never (WORN) archives.
What do we do at BADC? (In principle!)
Preserve input data. Copy on ingest if we have to, but we prefer (for the data itself) to demand that the supplier reformat into a format which does conform to a) designed for interoperability, and b) where there is a specification which preserves enough of the information content. (Duplicates of TB of data are not viable).
Preserve input documentation. Preserve specifications. Demand PDF (for now). Yes, the images are an issue, but if the images are data, then they ought to be actively preserved as data in their own right.
Ban MS documentation. History suggests that MS documents become WORN in about 6-8 years. Those who know no history are doomed to repeat it ...
So, I would argue that if you are doing curation, you have to address workflow before you get to the point of curation. If you know you want to preserve it, then think about that from the off. If you know you don't care about the future (shame on you), then yeah, ok, use your cool vendor tool ... but don't give the data to someone to curate, because curation is, in the end, about format conversion. If not now, sometime in the future INEVITABLY. If the documentation doesn't exist to do it, it's not curating. Don't kid yourselves.
All that said, much of the initial conversation was in the context of document curation, not data curation. IMHO the reason for much of what i perceive as confusion in their discussion, is not recognising the distinction! In the final analysis, I think that
if your object is to curate documents (i.e. what I would call the library functionality), then preserving PDF/A, latex etc, is perfectly fine - after all, with the spec, you're preserving with the same fidelity that documents have always been preserved.
if your object is to preserve the data, then it's a different ballgame, and folk need to confront the fact that curating data requires changes to the original workflow!
european summer drying
OK, I confess, I'm clearly reading my abstract summaries this morning ...
Briffa, van der Schrier and Jones: Wet and dry summers in Europe since 1750: evidence of increasing drought (International Journal of Climatology, 2009):
Moisture availability across Europe is calculated based on 22 stations that have long instrumental records for precipitation and temperature. The metric used is the self-calibrating Palmer Drought Severity Index which is based on soil moisture content. This quantity is calculated using a simplified water budget model, forced by historic records of precipitation and temperature data, where the latter are used in a simple parameterization for potential evaporation.
The Kew record shows a significant clustering of dry summers in the most recent decade. When all the records are considered together, recent widespread drying is clearly apparent and highly significant in this long-term context. By substituting the 1961-1990 climatological monthly mean temperatures for the actual monthly means in the parameterization for potential evaporation, an estimate is made of the direct effect of temperature on drought. This analysis shows that a major influence on the trend toward drier summer conditions is the observed increase in temperatures. This effect is particularly strong in central Europe.
Reading in 2009, 1: Six Degrees
So clearly in the last few weeks I've not been working and I've had half as many kids to look after ... so in between tears I've been dealing with fears .... the sort that abound if you make it much past the first chapter of Mark Lynas' book: Six Degrees: Our Future on a Hotter Planet.
Inspired by Aaaron Schwartz, albeit recognising reality (although heavily modified by recent events), I've decided to try and blog my years reading. Don't expect it to be too erudite, when I've got time I read pretty eclectically, and often choose mindless crap just to eat up time with as few brain cells as possible involved.
Anyway, back to the book of the day. The basic thesis is that there are six chapters describing the likely outcomes should our planet heat by between one and six degrees as a result of anthropogenic CO2 climate change.
It's a pretty well written book, with what looks to me like a reasonable coverage of the apocalyptic end of the literature. Clearly it's got a journalistic tone, with a fair dose of hyperbole, but he does temper it with a some qualification from time to time. While we all hope the six degree end is pretty unlikely, the possible consequences of even (!) the 2-3 degree changes make scary reading.
It'd be pretty easy, I think, to find the "ifs buts and maybes" in the original literature, and not much of it was "news" to me, but the thing about reading it all in one place was that it brought home to me that if even some of the predictions come home to roost, the world (both geographically and socially) is going to be a pretty different place in just a few decades, let alone a few centuries. Again, maybe that wasn't news, but there's something about having it rammed home all in one volume ...
So it's inspired me in two ways: I'm going to get back to the entire IPCC report and read the bits I don't normally (i.e. WG2 and WG3 stuff), and I'm going to try much harder to avoid business travel (you may well ask about personal travel, but we'll save the answer for another day). As regular readers will know, I've been avoiding business travel this last year anyway, in favour of virtual conferencing. You now know why, Evan having been pretty sick for a long time, and while looking after Evan is no longer an excuse for not travelling, I think given my profession, and given what we now believe about the future, it'd be wrong not to continue to try. Which brings me to my new years resolution: to try and convince my colleagues, especially the senior ones, to try harder to avoid physical meetings - particularly where the meetings are part of a regular sequence.
Update later same day: One of the things Lynas worries about is declining growth of corals: Ka ching!
blimey! solar wind and tropical cyclones
Here are a couple of papers that I'm going to have to find time to read properly:
Prikryl, P., Ru?in, V., and Rybansk\'y, M.: The influence of solar wind on extratropical cyclones ? Part 1: Wilcox effect revisited, Ann. Geophys., 27, 1-30, 2009, and
Prikryl, P., Muldrew, D. B., and Sofko, G. J.: The influence of solar wind on extratropical cyclones ? Part 2: A link mediated by auroral atmospheric gravity waves?, Ann. Geophys., 27, 31-57, 2009.
Some choice excerpts from the abstracts:
A sun-weather correlation, namely the link between solar magnetic sector boundary passage (SBP) by the Earth and upper-level tropospheric vorticity area index (VAI), that was found by Wilcox et al. (1974) and shown to be statistically significant by Hines and Halevy (1977) is revisited. A minimum in the VAI one day after SBP followed by an increase a few days later was observed. Using the ECMWF ERA-40 re-analysis dataset for the original period from 1963 to 1973 and extending it to 2002, we have verified what has become known as the "Wilcox effect" for the Northern as well as the Southern Hemisphere winters.
Cases of mesoscale cloud bands in extratropical cyclones are observed a few hours after atmospheric gravity waves (AGWs) are launched from the auroral ionosphere. It is suggested that the solar-wind-generated auroral AGWs contribute to processes that release instabilities and initiate slantwise convection thus leading to cloud bands and growth of extratropical cyclones.
It is also observed that severe extratropical storms, explosive cyclogenesis and significant sea level pressure deepenings of extratropical storms tend to occur within a few days of the arrival of high-speed solar wind.
Do I believe in this?
Well, I haven't read the papers, but I'm on record as believing that upper boundary affects can reach the troposphere, so it's feasible, particularly in that the basic thesis seems to revolve around small scale waves driving systems across instability boundaries, a non-linear affect that is more than feasible.