Skip to content

Converting SYNOP data to BUFR

Introduction

Surface synoptic observations (SYNOP) data are used to report weather observations from surface stations (manned or automated). synop2bufr is a tool to help convert SYNOP to BUFR data. ecCodes is a package to reading and writing GRIB and BUFR formats.

In this session you will learn about converting a SYNOP report into the WMO BUFR format using the above mentioned tools, as well as the relationship between SYNOP reports and BUFR messages.

Preparation

Warning

Ensure that you are logged into your student VM.

Navigate to the exercise-materials/synop2bufr-exercises directory and make sure that the exercises directories are there.

cd ~/exercise-materials/synop2bufr-exercises
ls

Tip

You should be able to see the following directories answers ex_1 ex_2 ex_3 ex_4 ex_5

synop2bufr primer

Below are essential synop2bufr commands and configurations:

transform

The transform function converts a SYNOP message to BUFR:

synop2bufr transform --metadata my_file.csv --output-dir ./my_directory --year message_year --month message_month my_SYNOP.txt

Note that if the metadata, output directory, year and month options are not specified, they will assume their default values:

Option Default
--metadata station_list.csv
--output-dir The current working directory.
--year The current year.
--month The current month.

In the examples, the year and month are not given, so feel free to specify a date yourself or use the default values.

ecCodes primer

ecCodes provides both command line tools and can be embedded in your own applications. Below are some useful command line utilities to work with BUFR data.

bufr_dump

The bufr_dump command is a generic BUFR information tool. It has many options, but the following will be the most applicable to the exercises:

bufr_dump -p my_bufr.bufr4

This will display BUFR content to your screen. If you are interested in the values taken by a variable in particular, use the egrep command:

bufr_dump -p my_bufr.bufr4 | egrep -i temperature

This will display variables related to temperature in your BUFR data. If you want to do this for multiple types of variables, filter the output using a pipe (|):

bufr_dump -p my_bufr.bufr4 | egrep -i 'temperature|wind'

Inspecting SYNOP data and BUFR conversion

Exercise 1

Navigate to the exercise-materials/synop2bufr-exercises/ex_1 directory and inspect the SYNOP message file message.txt:

cd ~/exercise-materials/synop2bufr-exercises/ex_1
more message.txt

Question

How many SYNOP reports are in this file?

Inspect the station list:

more station_list.csv

Question

How many stations are listed in the station list?

Question

Convert message.txt to BUFR format.

Tip

See the synop2bufr primer section.

Note

BUFR files have no set file extension, however it is recommended to use .bufr4.

Inspect the resulting BUFR data using bufr_dump.

Question

Compare the latitude and longitude values to those in the station list.

Tip

See the ecCodes primer section.

Exercise 2

Navigate to the exercise-materials/synop2bufr-exercises/ex_2 directory and inspect the SYNOP message file message.txt:

cd ~/exercise-materials/synop2bufr-exercises/ex_2
more message.txt

Question

How many SYNOP reports are in this file?

Inspect the station list:

more station_list.csv

Question

How many stations are listed in the station list?

Question

Convert message.txt to BUFR format.

Question

Based on the results of the exercises in this and the previous exercise, how would you predict the number of resulting BUFR files based upon the number of SYNOP reports and stations listed in the station metadata file?

Inspect the resulting BUFR data using bufr_dump.

Question

Check each of the output BUFR files contain different WIGOS Station Identifiers (WSI).

Exercise 3

Navigate to the exercise-materials/synop2bufr-exercises/ex_3 directory and inspect the SYNOP message file message.txt:

cd ~/exercise-materials/synop2bufr-exercises/ex_3
more message.txt

This SYNOP message only contains one longer report with more sections.

Inspect the station list:

more station_list.csv

Question

Is it problematic that this file contains more stations than there are reports in the SYNOP message?

Note

The station list file is a source of metadata for synop2bufr to provide the information missing in the alphanumeric SYNOP report and required in the BUFR SYNOP.

Question

Convert message.txt to BUFR format.

Inspect the resulting BUFR data using bufr_dump.

Question

Find the following variables:

  • Air temperature (K) of the report
  • Total cloud cover (%) of the report
  • Total period of sunshine (mins) of the report
  • Wind speed (m/s) of the report

Tip

You may find the last command of the ecCodes primer section useful.

Exercise 4

Navigate to the exercise-materials/synop2bufr-exercises/ex_4 directory and inspect the SYNOP message file message.txt:

cd ~/exercise-materials/synop2bufr-exercises/ex_4
more message_incorrect.txt

Question

What is incorrect about this SYNOP file?

Attempt to convert message_incorrect.txt using station_list.csv

Question

What problem(s) did you encounter with this conversion?

Exercise 5

Navigate to the exercise-materials/synop2bufr-exercises/ex_5 directory and inspect the SYNOP message file message.txt:

cd ~/exercise-materials/synop2bufr-exercises/ex_5
more message.txt

Attempt to convert message.txt to BUFR format using station_list_incorrect.csv

Question

What problem(s) did you encounter with this conversion?
Considering the error presented, justify the number of BUFR files produced.

Conclusion

Congratulations!

In this practical session, you learned:

  • the principles of SYNOP data reporting
  • how to use synop2bufr to convert SYNOP data to BUFR format
  • how to use bufr_dump to inspect the content of BUFR data