R&D

Hadley Centre

The Met Office Hadley Centre for Climate Prediction and Research sits on the third floor of the Met Office. The culture is less corporate than on the lower floors of the Met Office. The absence of any branding within the working environment is particularly noticeable.

The research undertaken by the Hadley Centre includes a range of activities including creating and managing climate datasets such as CRUTEM4, HadCRUT4, and the UK’s national gridded climate datasets. Scientists are also involved in modelling and analysing data including the development of new techniques such as reanalysis. See the notes at the end of the page for more information about CRUTEM4, HadCRUT4 and reanalysis.

The scientific research of the Hadley Centre is clearly aligned with academic principles and practices. Academic journal articles are a primary mode of dissemination for research findings. Communication with the general public, businesses and policy makers also appears to be an important part of scientists’ work.

The activity of the Hadley Centre is clearly focused on science. The role of data in enabling that science to take place is unquestionable. Some scientists have a deep affinity with the data work they are engaged in:

“Fundamentally I think I’ve been a data scientist since before data science was fashionable. Right. What I like to do is to find new information, and to see our picture of the world improving as we get a clearer image of what’s going on.”

“I always find the data analysis side of it really, really interesting. So that’s definitely something that I–, yeah it keeps me going, keeps me interested.”

Science is necessarily engaged in the pursuit of new knowledge, and this can be seen in the way that climate datasets are periodically updated to new versions – for example the publication of HadCRUT3 in 2006 and HadCRUT4 in 2012 – in order to refine and improve the scope, certainty and knowledge about the underlying data that climate scientists work with:

“Well, there’s always room for improvement. In the change from [HadCRUT]3 to 4 we had a large increase in the amount of temperature data going into the datasets. We had refinements to the way that we presented the uncertainty of information to make it more useable by scientists trying to use the data.

“We had, from the sea surface temperature side of things, some updates to understanding of the way that observations made by different platforms differ and how that affects the temperature measurements.

“There’s continuous updates to the kind of methods available, progress in various different related fields and statistics. And we try to gradually, as techniques improve, incorporate them into the datasets. So it’s more of a kind of gradual kind of research process as we gain more information and try to incorporate that into the datasets to improve them as we go.”

Whilst the scientists recognised the value of more and better data, the broader funding environment they work within tends to be more focused on scientific results and technology, rather than development of the underlying data infrastructure:

“There’s a big push on climate modelling because of weather forecasting, and of course global change more recently. And so it’s been, there’s been a heavy emphasis on that, which is important and fine, but I think unfortunately at times the data side of things has suffered.”

ACRE

ACRE – Atmospheric Circulation Reconstructions over the Earth – is a project run out of the Hadley Centre. The Met Office is one of nine core international partners, although there are several hundred organisations from around the world involved in the project. The project makes a significant contribution to the development of climate reanalysis techniques.

Reanalysis is a technique which potentially offers many benefits for improving our understanding of climate change. Whilst data from climate stations such as Weston Park are valuable in order to improve the accuracy of the technique far more data about historical weather conditions is needed to feed into the reanalysis models.

One significant area of activity for ACRE is therefore the recovery of historical weather observation data from around the world. Some of this work involves working collaboratively with data holders and demonstrating the benefits of sharing weather and climate data with climate scientists. Other work involves the discovery and transcription of archived weather observations, for example the shipping logs that volunteers on the Old Weather citizen science project are working on.

Challenging environment

Despite the importance of historical data for reanalysis, funding to support data recovery efforts is difficult to find:

“The old questions of money and personnel become critical. And we really do work on a shoestring, there’s no smoke and mirrors out there in what we do. They all think it’s great. They all want the result, but they’re not really prepared to pay for it.”

“I guess what we suffer from is the thing that I guess we’re not seen as sexy. Especially the data part of it, you know, everybody’s sort of, ‘Oh data’, you know.”

“The frustration we’re having is because when we find there’s a block of data and we’ve got to think how the hell are we going to get this imaged and digitised and into the database, because you’ve got to find people with money.”

Enterprising science

Despite this constrained environment, the ACRE project is able to “chip away” at the problem due to its freedom to innovate. To a large extent the project seems to have escaped many of the corporate constraints of its parent institution:

“I’ve been very lucky to be in a position where I can do this without too much from high saying, you know, what are you doing and why are you doing? You know, I just do it [laughs], and it made it happen. So you know, it’s about freely doing things, or more freely doing things than perhaps most other people in the organisation.”

The culture of ACRE diverged quite significantly from that of its parent organisation and similar institutions. As one scientist described:

“The organisations work a bit like the Royal Navy, we work like Pirates of the Caribbean.”

This enterprising form of science prioritised flexibility, serendipity and freedom over careful planning:

“You’ve just got to sort of be completely flexible, and keep thinking laterally, and seeing where things might pop up.”

“The sort of serendipitous, if that’s the word, nature of this. There wasn’t great planning that ended up with this result. It’s just something that’s evolved, and you know, and it’s starting to have value.”

Organic leadership

With several hundred organisations around the world involved in the project, including a number of citizen science projects such as Old Weather, ACRE functions by adopting a bottom-up, organic form of organisation:

“It’s up to them if they want to be involved. We try not to send out dictums to people to do this or that, but sort of say, you know, link in with the work that we’re doing now, you know, so that we all benefit.”

“I try to keep it on sort of on a bit of a knife edge here. We work in the twilight zone a little bit at times. I think in a lot of ways it’s the informality that has given us strength in different ways that allows it to sort of develop and then to be more flexible.”

Sustainability

Despite this organic nature, ACRE is dependent upon a leader who can “keep all the balls in the air”. In order to keep the project sustainable, it is hoped that other people will come forward to take on more of a leadership role in the future:

“I would like to see... more younger people taking on more of the mantle of it.”

It is also hoped that as institutions such as the Met Office begin to see more value in ACRE, the project will “get more traction and embedded more in to institutions like the Met Office and others”. Although it was perceived that whilst this would be beneficial for the long term sustainability of the work, it was uncertain whether the Met Office, and similar institutions, would be able to maintain the organic nature of the project that was perceived to be one of its key strengths:

“I don’t know what they’d do, I don’t know whether they’d be able to do it in the way that we do it, or I do it, because they would try to formalise it more.”

WOW project

The people behind the Met Office’s Weather Observations Website (WOW) are working to improve the accuracy of weather forecasting, particularly in relation to extreme weather events at the local level. In order to do this they need to collect large volumes of local observation data, however the official observation network does not have the required level of granularity and funding is not available to invest in the development of the infrastructure.

The WOW project was therefore set up to collect weather data from thousands of amateur observers around the country:

“To date we’ve received getting close to 300 million observations from about 6,000 different observing sites.”

Scientific understanding and accuracy are at the heart of the WOW project. Yet, the project exists within a constrained environment and aims to meet the increasing data demands of meteorologists within a context of reduced budgets and demands for increased efficiency:

“At that time there were various clear drivers that more observations were required, the forecast models increasing in resolution, super computers getting more powerful, forecasters are always keen to have access to as much live information as possible to compliment the information they’ve got from models or from existing data sources. And there was also sort of as across most organisations there were pressures to reduce costs and become more efficient.”

Data quality and quantity

WOW recognises that for certain meteorological activities there are benefits in simply having as much data as possible, however this is tempered with a need to maintain data quality.

A note of caution is present in the treatment of the amateur data. Significant effort is made to verify the quality of the data, and to assess it against official data sources, before it can be considered for Met Office use.

There is something of a tension between making the WOW website accessible and easy to use for anyone who wants to upload observations, and incorporating data standards to ensure the data has provenance and a minimum level of quality so that it is useful for the Met Office as well as the general public.

Overall there is a strong desire to ‘get it right’, in terms of using data standards, verifying data quality, and providing technical support.

Cautious development

The WOW project has been relatively cautious in its development. Investigations into its initial feasibility and likely acceptance were lengthy, and consultations were undertaken with various interested communities.

At the moment the project is still in ‘beta’ after three years of development. The data collected by WOW is not yet integrated into forecasting systems, however academic research is currently being conducted to gauge the suitability of the amateur data for forecasting purposes.

Whilst the project has attracted a high number of contributors, there has been little public promotion of the website to date:

“Still to date we haven’t done much in the way of announcing its existence to the world.”

However, despite this caution, interest is developing in the international meteorological community and the Australian meteorological service have recently developed their own WOW platform in an effort to gather their own amateur weather data.

Further information

CRUTEM4

CRUTEM4 is a gridded climate dataset produced by the Hadley Centre. The dataset reports global land surface air temperatures relative to the average temperature during the period 1961–1990. The dataset was initially developed by the Climatic Research Unit (CRU) at the University of East Anglia, and it is now updated by both CRU and the Met Office Hadley Centre.

Read more about the CRUTEM4 dataset on the Met Office website

HadCRUT4

Another gridded climate dataset produced by the Hadley Centre. The dataset reports global surface temperatures relative to the average temperature during the period 1961–1990. It is a combination of the CRUTEM4 land surface air temperature dataset developed by CRU and the Hadley Centre, and the HadSST3 sea-surface temperature (SST) dataset developed by the Hadley Centre.

Read more about HadCRUT4 on the Met Office website

Reanalysis

Reanalysis is a climate modelling and prediction technique. One of the Met Office scientists that we spoke to explained:

"It's an approach to reconstructing the weather back through time. It involves taking a weather forecast model, a modern weather forecast model, which is normally used for projecting for future weather, but what they do is they take that model and they effectively put it back in time, so instead of putting in today or yesterday's data they put in a hundred years ago, or 50 years ago. And then progressively push it through time... And then look in to the future.

“Reconstructing from that past data all the other variables in the atmospheres system, and then stepping through time every six hours. Iterating forward in time so that you actually come up with a reconstruction of the weather, dynamical reconstruction of the weather for all the variables that you can think of basically."