Data journey

Ship log books

More than two thirds of the earth’s surface is covered by oceans. The collection of marine weather observations has been less systematic than over land, particularly before the advent of satellite technology, due to the often hostile and inaccessible nature of these oceans. There are therefore significant gaps in historic weather observations over the sea.

However, crews of naval and merchant ships have long made weather observations at sea using a variety of methods:

“You have observations made by ships, either in very old data from say throwing a bucket over the side of the ship, hauling it up, putting a thermometer into it, and then writing down what the temperature was, or otherwise coming in through water intake into an engine room.”

This data was recorded in ships’ log books and many of these are now stored in archives around the world. This data offers climate scientists a possible window through which to develop their understanding of marine weather conditions over the last century and more.

Before climate scientists can analyse this data it needs to be recovered, and in some cases rescued, from the archives.

Filling the gaps

Data recovery and data rescue activities begin with an information need from climate scientists for historical weather data from specific regions and time periods where climate data is sparse.

“I usually say to people if you’ve got a weather observation, if it’s south of the equator, we’ll take it. Right. If it’s north of the equator before about 1950 – we’ll take it. If it’s north of the equator, outside 1950, and it’s not in North America, Europe – Japan, or the North Atlantic – we’re probably still interested.”

They discuss their needs with historians and archivists such as Clive Wilkinson, who use their skills to locate suitable ship log books in the archives. This can often be a challenging task. In many countries, the archives are simply storehouses with boxes of old records whose contents are unknown. Often these records are kept in poor, often humid, conditions which mean they can decay or be destroyed before they are found.

Finding the data

After speaking with the climate scientists, Clive’s first job is to identify appropriate ships and find out which archives hold the required records.

“I compiled an inventory of every major Royal Navy vessel from 1800 to 1950, where it was, what its movements were, where its logbook is, and I also compiled reports for them, which are available, concerning logbooks generally, where they are in the archives, what they consist of, what some of the problems are in using them, this sort of thing.”

He then works with the local archivists to negotiate the project’s access to the archive and begin locating the shipping logs that need to be recovered – or, if they are in a particularly poor state, rescued.

Where no cataloguing and digitisation has taken place, finding the shipping logs often requires a bit of detective work and an element of serendipity. Clive finds that the most fruitful place to start looking is often in the relatively un-catalogued sections of the archive.

“When it says miscellaneous in my view it’s because it’s got lots of stuff with numbers on and they didn’t know what to do with it.”

Recovering the data

Once suitable records are found, the data recovery process begins with a historian or archivist such as Clive compiling a survey of the archived materials. Clive also uses his vast knowledge of shipping logs to gauge how useful they will be for the climate scientists, and advises them on any problems they are likely to encounter using the data:

“Well, my expertise is finding the data, knowing how it was collected, knowing some of the problems of the–, what I’ll do is I’ll look through the logbooks, and I’ve got a lot of experience, I’ve looked at thousands of them, and I anticipate the problems they’re going to have with the data. Sometimes I say, ‘You’re going to have this problem. This is how you overcome it.’”

Climate scientists then use this survey to select which records are of most use to them. These selected log books are then digitised and uploaded into a central repository.

A preservation process will also be undertaken for records that are in a particularly poor state of repair:

“The archivist looks at them and says, ‘How good condition is this record?’ You know, ‘Can we just photograph it, or do we need to repair it first? Is it going to fall to pieces on us? Does it need curating?’ So they do the curating as necessary, and then we try and photograph it, and that requires a bit of human expertise because the quality of the ink after 150 years is not always great. Okay. But usually we get fairly legible photographs out of it.”

These digitised ship log books are then assessed by a climate scientist, and records are selected and released for transcription by citizen scientists working on projects such as Old Weather.