|
Specification of the ASCII data file format
-- SCIAMACHY data
The daily data at coordinates of the measurements by the satellite
instrument are delivered in the form of ASCII files, one for each orbit file
with spectral data. Measurements of SCIAMACHY come in principle
as one file per orbit, but it happens quite often that files span only
a part of an orbit, or even the end of one orbit and the beginning of the
next. The data files are delivered via the website as zip-files containing
all orbit files of a given day.
See the main Data format specification page
for some general notes on the data formats.
For a given day, the orbits treated are those orbits that have a start
time during that day; this is the time mentioned in the name of the data
file. The name of the SO2 data files contains the orbit date
YYYYMMDD and the start time HHMMSS , both
taken from the name of the original measurement file:
so2cdYYYYMMDD_HHMMSS.dat
where so2cd stands for "SO2 column density". These files
contain the SO2 slant column data (SCD) and -- if available -- the SO2
vertical column density (VCD) derived with an appropriate air-mass factor,
and the cloud cover fraction.
The ASCII data file can be read easily with, for example, a Fortran
program (example page) for
further processing.
The ASCII file can also be read with many plot programs, such as IDL
(example page) and MatLab.
File header
Each data file has a header with comment lines (starting with the
# mark), giving orbit and analysis information, as well as a
list of the data columns. Also given is for format of each data fields,
and the full data format for reading the data lines.
A typical data file header for data files looks as follows. Some
remarks regarding the entries in the file header are given
below the file header.
# SO2 column density for TEMIS / PROMOTE
# --------------------------------------
#
# Product status : archive data
# Process version : 1.0.3
#
# Instrument : SCIAMACHY
# Orbit date/time : 20070320_120511
# Orbit number : 26416
#
# Analysis date : 2007/08/13
# Cloud cover data: FRESCO (SC-v5)
# AMF & VCD values: yes
#
# Nr plume heights: 3
# Nr data columns : 47
#
# Data columns [format]
#
# --- Ground pixel data
# 1 = measurement date as YYYYMMDD [a8]
# 2 = measurement time as HHMMSS.SSS [1x,a10]
# 3 = pixel id: 0=forward, 3=backscan [i4]
# 4-7 = pixel corner latitudes [4f9.3]
# 8 = pixel center latitude [f9.3]
# 9-12 = pixel corner longitudes [4f9.3]
# 13 = pixel center longitude [f9.3]
# 14 = solar zenith angle (SZA) at TOA [f9.3]
# 15 = viewing zenith angle (VZA) at TOA [f9.3]
# 16 = relative azimuth angle (RAA) at TOA [f9.3]
#
# --- Slant column data
# 17 = SO2 slant column density SCD (in DU) [f9.3]
# with background correction
# 18 = retrieval error on the SCD (in DU) [f9.3]
# 19 = chi^2 of the slant column fit (x 1e-6) [f9.3]
# 20 = slant column value index (SVI) [i4]
# 0 : SO2 SCD <= 1.5 DU
# 1 : SO2 SCD > 1.5 DU, no notification issued
# 2 : SO2 SCD > 1.5 DU, notification issued for state
#
# --- Vertical column data
# 21 = AMF quality index (AQI) [i4]
# = -1 : no AMF calculation selected
# = 0 : successful AMF calculation
# = +1 : no cloud cover data available
# > +1 : error computing AMF
# 22 = AMF profile shape number (1 or 2) [i4]
# --- using plume height #1 = 2.0 km above surface
# 23 = SO2 vertical column density VCD (in DU) [f9.3]
# 24 = Error on VCD from column 18 (in DU) [f9.3]
# 25 = Total air-mass factor (AMFtot) [f9.3]
# 26 = AMF for clear-sky part (AMFclr) [f9.3]
# 27 = AMF for cloudy part (AMFcld) [f9.3]
# --- using plume height #2 = 6.0 km *
# 28 = SO2 vertical column density VCD (in DU) [f9.3]
# 29 = Error on VCD from column 18 (in DU) [f9.3]
# 30 = Total air-mass factor (AMFtot) [f9.3]
# 31 = AMF for clear-sky part (AMFclr) [f9.3]
# 32 = AMF for cloudy part (AMFcld) [f9.3]
# --- using plume height #3 = 14.0 km *
# 33 = SO2 vertical column density VCD (in DU) [f9.3]
# 34 = Error on VCD from column 18 (in DU) [f9.3]
# 35 = Total air-mass factor (AMFtot) [f9.3]
# 36 = AMF for clear-sky part (AMFclr) [f9.3]
# 37 = AMF for cloudy part (AMFcld) [f9.3]
# *) or 1 km above surface if that is higher
#
# --- Cloud and Surface data
# 38 = cloud cover index (CCI) [i4]
# = 0 : no cloud cover data
# = 1 : clear sky mode
# = 2 : normal FRESCO mode
# = 3 : snow/ice FRESCO mode
# = 4 : missing or invalid FRESCO data
# 39 = cloud fraction [f9.3]
# 40 = cloud top pressure (in hPa) [f9.3]
# 41 = cloud top height (in km) [f9.3]
# 42 = cloud top albedo [f9.3]
# 43 = surface pressure (in hPa) [f9.3]
# 44 = surface elevation (in km) [f9.3]
# 45 = surface albedo [f9.3]
#
# --- Additional gound pixel data
# 46 = sciamachy state_index [i4]
# 47 = sciamachy state_id [i4]
#
#
# Full data format: (a8,1x,a10,i4,16f9.3,3i4,15f9.3,i4,7f9.3,2i4)
#
#
Which is followed by two lines spanning all columns and giving a header
to each column. Then follow the data lines themselves. The very end of the
file is marked by:
#
# --- end of file.
The first part of the comments gives some general information on the orbit
and retrieval. The number of data columns in the file depends on the number
of plume heights used for computing the AMF and VCD, hence both these
numbers are given here. The first 22 columns are always present. Then follow
sets times five lines, one set for each plume height. And the last 9 columns
are always present.
If no vertical column densities (VCDs) have been computed, then:
- the
AMF & VCD values comment line in this part
of the file says no ;
- the AMF quality index (AQI) in column 21 is set to -1;
- all data entries concerning the AMF and VCD are given the "no data"
value (-99.0), except where possible the clear-sky AMF (see below).
If cloud cover information is included, as in the example, the source of the
cloud data is mentioned. If no cloud cover information has been included,
then:
- the
Cloud cover data comment line in this part
of the file says none ;
- the cloud cover index (CCI) in column 38 in the above example
is set to 0;
- all data entries concerning the cloud cover, the AMF and VCD are given
the "no data" value (-99.0).
and no AMFs or VCDs can be computed.
Remarks
Remarks regarding some of the entries in the above file header:
- On the right of the list of data columns and below the list is the
format with which the data can be read in the notation used, for
example, in Fortran and IDL.
- Column 3 gives the type of ground pixel.
For SCIAMACHY data there is only a distiction between forward (0) and
backward (3) scan pixels.
- The zenith and azimuth angles (columns 14, 15 and 16) are given at the
top-of-atmosphere (TOA) for the centre of the ground pixel.
- The error value given in column 18 is the error following from the
DOAS slant column retrieval. Its value is based on the RMS of the
fit between the measured and the fitted spectrum. The chi-square in
column 19 is also a measure of the quality of the fit. In fact, these
three quantities are all interlinked:
chi-square = RMS * RMS,
and the relationship between the slant column error and the
RMS is assumed to be linear.
To assist data users in their analysis, both the slant column
error and the chi-square are written to the data file.
- The "slant column value index" (SVI) in column 20 is meant to give the
user a quick look facility on the SO2 slant column value. For more
information, see the chapter on the criteria for the notifications
on exceptional SO2 concentrations.
- If an AMF and VCD have been computed, the "AMF quality index" (AQI)
in column 21 indicates success of the AMF computation, and if
successful the VCD was computed.
The relevant AQI values are listed in the file header; more details
are given on a separate page.
- Column 22 gives the number of the profile used for the AMF computation,
which depends on the SO2 concentration: '1' for low concentrations, '2'
for high concentrations; more details are given on the
Air-mass factor using look-up tables
info page.
- In order to compute the AMF, a height must be given of the SO2 plume.
As this height is not known automatically, a number of assumed heights
are used, in this case three. For each assumed plume hieght, the
data file gives five quantities: the SO2 vertical column, an estimate
of the error on it (based on the error given in column 18), and the
three components of the AMF; more details are given on the
Air-mass factor using look-up tables
info page.
- If cloud cover data is included, the "cloud cover index (CCI)" in
column 38 in the above example indicates success of this inclusion. The
relevant CCI values are listed in the file header; some more details
are given on a separate page.
- Data fields for which no value was (successfully) computed are given
the "no data" value -99.
|
|