Bryan Lawrence : Storing and manipulating environmental big data with JASMIN

Bryan Lawrence

... personal wiki, blog and notes

Storing and manipulating environmental big data with JASMIN

We're pleased that our paper on the first year of JASMIN has been accepted by the IEEE BigData 2013 conference.

I'll put a copy of the paper online soon, but for now, here is the abstract:

Storing and manipulating environmental big data with JASMIN

B.N. Lawrence, V.L. Bennett, J. Churchill, M. Juckes, P. Kershaw, S. Pascoe, M. Pritchard, S. Pepler and A. Stephens

JASMIN is a super-data-cluster designed to provide a high-performance high-volume data analysis environment for the UK environmental science community. Thus far JASMIN has been used primarily by the atmospheric science and earth observation communities, both to support their direct scientific workflow, and the curation of data products in the STFC Centre for Environmental Data Archival (CEDA). Initial JASMIN configuration and first experiences are reported here. Useful improvements in scientific workflow are presented. It is clear from the explosive growth in stored data and use that there was a pent up demand for a suitable big-data analysis environment. This demand is not yet satisfied, in part because JASMIN does not yet have enough compute, the storage is fully allocated, and not all software needs are met. Plans to address these constraints are introduced.

Categories: abstracts jasmin

No comments yet

Comments presently read-only.

This page last modified Wednesday 28 August, 2013
DISCLAIMER: This is a personal blog. Nothing written here reflects an official opinion of my employer or any funding agency.