Data journey

Gridded datasets

The Climatic Research Unit (CRU) has played a key role in creating and updating the CRUTEM gridded dataset of global land surface temperature anomalies. CRU created the first version of the dataset in the 1980s. The latest major update was released in 2012 and is called CRUTEM4. The methodology describing how it was created is explained in recent academic research publications co-authored by the CRU team.

The CRUTEM4 gridded dataset is a representation of the difference between monthly average temperatures and the 1961–1990 average temperature across the land surface of the Earth. The dataset covers the period from January 1850 to the present.

“The gridded data are based on an archive of monthly mean temperatures provided by more than 5,500 weather stations distributed around the world. Each station temperature is converted to an anomaly from the 1961–90 average temperature for that station, and each grid-box value is the mean of all the station anomalies within that grid box.

“As well as the mean anomaly, estimates are made of the uncertainties arising from thermometer accuracy, homogenisation, sampling grid boxes with a finite number of measurements available, large-scale biases such as urbanisation and estimation of regional averages with non-complete global measurement coverage.” – Met Office Hadley Centre website

The image above shows CRUTEM4 gridded dataset findings for the grid square covering England as published by CRU scientists on Google Earth (Osborn and Jones, 2014). The yellow markers in the grid square show which stations’ data are being used to calculate the average. The red lines show the years when the temperature was warmer than the 1961–1990 average, and the blue lines show the years it was cooler.

Open CRUTEM4 in Google Earth and explore further (KML, 92KB)

Collaboration

Whilst CRU did much of the early work on CRUTEM, the current version – CRUTEM4 – is a collaborative effort between the Met Office Hadley Centre and CRU. Each undertake part of the maintenance work required to keep it up-to-date:

“The historical data is compiled by the team at UEA [CRU] who, having compiled a database of temperature records for weather stations around the globe, make that data available to use here at the Met Office. We then use that information at the Met Office to produce the gridded temperature maps and time series, merge that data with the sea-surface temperature dataset [into the combined HadCRUT4 dataset] to produce global temperature estimates and derive the uncertainty estimates in these.”

Today, CRU completes its update of the CRUTEM4 gridded dataset on an annual basis. CRU researchers gather and incorporate additional historical data series from stations around the world. CRU researcher David Lister explains a typical part of the update process:

“My job at this time of the year tends to be to go to these different sources, like the Chinese source and the Canadian source, and just grab the recent updates. Sometimes it’s just literally a few monthly values, sometimes they’ve revamped it because they run stuff through algorithms, through homogenisation software etc. And so they might have suddenly had a big rework and so the series could be significantly changed.”

Once the annual update is completed, CRU passes its new version to scientists at the Met Office Hadley Centre who are responsible for undertaking the monthly updates. The updates for each annual update are carefully documented on the Met Office website.

The Hadley team also use these updated CRUTEM4 data in the generation of HadCRUT4 – a combined land and sea surface temperature gridded dataset.

Visit the Met Office station to find out more

Quality control

During the update any new datasets are processed to match the requirements of the CRUTEM4 dataset. For some stations, the first stage involves converting daily data into monthly average (mean) temperature. For all stations there are then detailed checks to ensure data is accurate and that the correct values are imported. David Lister and his colleague Ian Harris explain some of the quality control processing that might be required:

“If it was a daily file, you could have three different readings for one day, simply because one reading might be what came off the instrument. And then the next reading might be a QC’d [quality controlled] value because they’ve determined through their QC software that this value is wrong. Or there may have been two or three different values come out of that instrument for the day, or whatever.”

“Or one’s been homogenised and one hasn’t.”

“You can get these different values for the same day. And there are flags, obviously, so you have to then take the flag to know which of those values to use.”

Data sharing

In order to improve the transparency of the production of CRUTEM4 since the Unit was hacked in 2010 (see Policy page), the dataset now only contains data that can be shared publicly:

“All the series that go into the gridded product are publicly available on the Hadley Centre website, and that’s the rules of engagement now, publicly available. So if anyone offered us a load of series that were for the scientific good but could not be released, we can’t use them because they have to be publicly available.”

Once CRUTEM4 has been updated it is uploaded into the British Atmospheric Data Centre repository so it can be used by other academic researchers. It is also made publicly available on the Met Office Hadley Centre website, along with station data and additional time-series data generated by the Met Office:

“So we make all of our data available as ASCII as well and just describe the format unambiguously so that people can access it. But we still, I get people contacting me all the time for the datasets that I manage, because they’re having trouble opening them in Excel or something.”

In an attempt to improve accessibility of the dataset for the public, CRU have also made CRUTEM4 available via Google Earth.