sciapy.level2

SCIAMACHY level 2 processing

This module contains functions to process SCIAMACHY level 2 number density data (only NO for now).

sciapy.level2.aacgm2005

AACGM 2005 geomagnetic model at 80 km

sciapy.level2.aacgm2005.gmag_aacgm2005(lat, lon, aacgm_name='AACGM2005_80km_grid.nc')

Fixed 2005 AACGM geomagnetic coordinates at 80 km

Geomagnetic coordinates according to the AACGM model but with fixed parameters for the 2005 epoch.

Parameters:

• lat (array_like) – Geographic latitude(s) in degrees north
• lon (array_like) – Geographic longitude(s) in degrees east
• aacgm_name (str, optional) – Filename of the AACGM grid, relating geographic latitude and longitude to AACGM geomagnetic latitude and longitude. The default is the prepared grid file for 2005 and at 80 km.

Returns:

• aacgmlat (numpy.ndarray or float) – The AACGM 2005 geomagnetic latitude(s)
• aacgmlon (numpy.ndarray or float) – The AACGM 2005 geomagnetic longitude(s)

sciapy.level2.binning

SCIAMACHY level 2 binning

sciapy.level2.binning.bin_lat_timeavg(ds, binvar='latitude', tvar='time', bins=array([-90, -85, -80, -75, -70, -65, -60, -55, -50, -45, -40, -35, -30, -25, -20, -15, -10, -5, 0, 5, 10, 15, 20, 25, 30, 35, 40, 45, 50, 55, 60, 65, 70, 75, 80, 85, 90]), labels=None, area_weighted=True, keep_attrs=True, load=True, save_progress=False)

Latitudinally bin and time average xarray dataset(s)

Time-averages the variables in an xarray.Dataset in the given latitude bins. This should be applied to daily-binned datasets from a groupby object (via .apply()) to yield daily zonal means.

The type of latitudes is selected by passing the appropriate binvar and must be a variable in the data set. Area weighting (cos(latitude)) is also supported.

Parameters:

• ds (xarray.Dataset or xarray.DatasetGroupBy instance) – The dataset (or GroupBy) instance to bin latitudinally.
• binvar (str) – The name of the variable used for binning, default: “latitude”.
• tvar (str) – The name of the time variable of the GroupBy object, default: “time”.
• bins (numpy.ndarray) – The (latitudinal) bin edges, default: np.r_[-90:91:5].
• labels (list or None) – The bin labels, if set to None (the default), the labels are set to the central bin values.
• area_weighted (bool) – Use area weighted averages, default: True.
• keep_attrs (bool) – Keep the global and variable attributes from the data set, default: True.
• load (bool) – Loads the data into memory before binning, speeds it up considerably provided that the it fits into memory, default: True
• save_progress (bool) – Saves the individual binned files to netcdf, to enable recovering from interrupted runs, default: False

Returns:

ds – The binned and time-averaged dataset together with the (unbiased) standard deviations of the variables as <variable>_std and the number of averaged values as <variable>_cnt.

Return type:

xarray.Dataset

sciapy.level2.density

SCIAMACHY level 2 number density retrieval results interface

Interface classes for the level 2 retrieval results from text (ascii) files and netcdf files for further processing.

class sciapy.level2.density.scia_densities(author='unknown', ref_date='2000-01-01', ver=None, data_ver=None)

Bases: object

SCIAMACHY orbital retrieved number densities

Class interface to orbit-wise SCIAMACHY retrieval results. The attributes are based on the text file layout and are tied to the NO retrieval for now.

Parameters:

• ref_date (str, optional) – The reference date on which to base the date calculations on. Default: “2000-01-01”
• ver (str, optional) – Explicit density version, used for exporting the data. Not used if set to None. Default: None
• data_ver (str, optional) – Level 2 data version to use, as “ver” used for exporting. Not used if set to None. Default: None

Attributes:

• version – file version string
• data_version – level 2 data version
• date0 – reference date
• nalt – number of altitudes in the orbit
• nlat – number of latitudes in the orbit
• nlon – number of longitudes in the orbit, if longitudes are available
• orbit – SCIAMACHY/Envisat orbit number
• date – number of days of the orbit counting from the reference date date0
• alts_min
• alts
• alts_max – the altitude bins: minimum, central, and maximum altitude
• lats_min
• lats
• lats_max – the latitude bins: minimum, central, and maximum latitude
• lons – the central longitude of the bins, only used if available
• densities – NO number densities in the bins, (nlat, nalt) array_like
• dens_err_meas – NO number densities measurement uncertainty, (nlat, nalt) array_like
• dens_err_tot – NO number densities total uncertainty, (nlat, nalt) array_like
• dens_tot – total number densities calculated and interpolated NRLMSIS-00 values, (nlat, nalt) array_like
• apriori – prior NO number densities, (nlat, nalt) array_like if available, otherwise None
• akdiag – diagonal element of the averaging kernel matrix at the retrieval grid point. (nlat, nalt) array_like if available otherwise None

read_from_textfile()

read_from_netcdf()

read_from_file()

write_to_textfile()

write_to_netcdf()

Note

The variables are empty when initialized, use one of the read_from_…() methods to fill with actual data.

Methods

read_from_file(filename)	Wrapper to read NO desnities from files
read_from_netcdf(filename[, close])	Read NO densities from netcdf files
read_from_textfile(filename)	Read NO densities from ascii table file
write_to_netcdf(filename[, close])	Write NO densities to netcdf files
write_to_textfile(filename)	Write NO densities to ascii table files

read_from_file(filename)

Wrapper to read NO desnities from files

Simple wrapper to delegate reading the data from either netcdf or ascii files. Poor man’s logic: simply try netcdf first, and if that fails, read as ascii.

Parameters:filename (str) – The filename to read the data from.

read_from_netcdf(filename, close=True)

Read NO densities from netcdf files

This function has no stream, i.e. file object support.

Parameters:

• filename (str) – The filename to read the data from.
• close (bool, optional) – Whether or not to close the file after reading. Setting to False enables reading further data from the same file. Default: True

Returns:

• Nothing if close is True. If close is False, returns either an
• netCDF4.Dataset,
• scipy.io.netcdf.netcdf_file or
• pupynere.netcdf_file instance depending on availability.

read_from_textfile(filename)

Read NO densities from ascii table file

Parameters:filename (str, file object or io.TextIOBase.buffer) – The filename or stream to read the data from. For example to read from stdin in python 3, pass sys.stdin.buffer.

write_to_netcdf(filename, close=True)

Write NO densities to netcdf files

This function has no stream, i.e. file object, support.

Parameters:

• filename (str) – The name of the file to write the data to.
• close (bool, optional) – Whether or not to close the file after writing. Setting to False enables appending further data to the same file. Default: True

Returns:

• Nothing if close is True. If close is False, returns either an
• netCDF4.Dataset,
• scipy.io.netcdf.netcdf_file or
• pupynere.netcdf_file instance depending on availability.

write_to_textfile(filename)

Write NO densities to ascii table files

Parameters:filename (str or file object or io.TextIOBase.buffer) – The filename or stream to write the data to. For writing to stdout in python 3, pass sys.stdout.buffer.

sciapy.level2.density_pp

SCIAMACHY level 2 post-processed number densities interface

Interface classes for the level 2 post-processed retrieval results.

class sciapy.level2.density_pp.scia_densities_pp(ref_date='2000-01-01', ver=None, data_ver=None)

Bases: scia_densities

Post-processed SCIAMACHY number densities

Extends scia_densities with additional post-processing attributes such as (MSIS) temperature and density, local solar time, solar zenith angle, and geomagnetic latitudes and longitudes.

This class only supports writing ascii files but reading to and writing from netcdf.

Attributes:

• temperature – NRLMSISE-00 temperatures
• noem_no – NOEM NO nuimber densities
• vmr – NO vmr using the NRLMSISE-00 total air number densities
• lst – Apparent local solar times
• mst – Mean local solar times
• sza – Solar zenith angles
• utchour – UTC hours into measurement day
• utcdays – Number of days since reference date
• gmlats – IGRF-12 geomagentic latitudes
• gmlons – IGRF-12 geomagentic longitudes
• aacgmgmlats – AACGM geomagentic latitudes
• aacgmgmlons – AACGM geomagentic longitudes

write_to_textfile()

write_to_netcdf()

read_from_netcdf()

to_xarray()

Methods

read_from_file(filename)	Wrapper to read NO desnities from files
read_from_netcdf(filename[, close])	Read post-processed level 2 orbit files
read_from_textfile(filename)	Read NO densities from ascii table file
to_xarray(dateo, orbit)	Convert the data to xarray.Dataset
write_to_netcdf(filename[, close])	Write variables to netcdf files
write_to_textfile(filename)	Write the variables to ascii table files

read_from_netcdf(filename, close=True)

Read post-processed level 2 orbit files

Parameters:filename (str) – The name of the netcdf file.

to_xarray(dateo, orbit)

Convert the data to xarray.Dataset

This is a very simple approach, it dumps the data to a temporary netcdf file and reads that using xarray.open_dataset().

Parameters:

• dateo (float) – The days since the reference date at the equator crossing of the orbit. Used to set the time dimension of the dataset.
• orbit (int) – The SCIAMACHY/Envisat orbit number of the retrieved data.

Returns:

dataset

Return type:

xarray.Dataset

write_to_netcdf(filename, close=True)

Write variables to netcdf files

This function has no stream, i.e. file object, support.

Parameters:filename (str) – The name of the file to write the data to.

write_to_textfile(filename)

Write the variables to ascii table files

Parameters:filename (str or file object or io.TextIOBase.buffer) – The filename or stream to write the data to. For writing to stdout in python 3, pass sys.stdout.buffer.

class sciapy.level2.density_pp.scia_density_day(name='NO', ref_date='2000-01-01', author='unknown')

Bases: object

SCIAMACHY daily number densities combined

Contains a stacked version of the post-processed orbit data for multiple orbits on a day. Used to combine the results.

Parameters:

• name (str) – Name of the retrieved species, default: “NO”. Used to name the netcdf variables appropriately.
• ref_date (str, optional) – The reference date on which to base the date calculations on. Default: “2000-01-01”

Attributes:

• alts – Retrieval grid altitudes
• lats – Retrieval grid latitudes
• no_dens – Retrieved number densities
• no_errs – Measurement uncertainty
• no_etot – Total uncertainty
• no_rstd – Relative measurement uncertainty
• no_akd – Averaging kernel diagonal elements
• no_apri – Prior number density
• temperature – NRLMSISE-00 temperatures
• noem_no – NOEM NO nuimber densities
• vmr – NO vmr using the NRLMSISE-00 total air number densities
• lst – Apparent local solar times
• mst – Mean local solar times
• sza – Solar zenith angles
• utchour – UTC hours into measurement day
• utcdays – Number of days since reference date
• gmlats – IGRF-12 geomagentic latitudes
• gmlons – IGRF-12 geomagentic longitudes
• aacgmgmlats – AACGM geomagentic latitudes
• aacgmgmlons – AACGM geomagentic longitudes

Methods

append(cdata)	Append (stack) the data from one orbit
append_data(date, orbit, equtime, scia_dens)	Append (stack) the data from one orbit
to_xarray()	Convert the combined orbit data to xarray.Dataset
write_to_netcdf(filename)	Write variables to netcdf files

append(cdata)

Append (stack) the data from one orbit

Parameters:cdata (scia_densities_pp instance) – Post-processed level 2 orbital data.

append_data(date, orbit, equtime, scia_dens)

Append (stack) the data from one orbit

Updates the data in place.

Parameters:

• date (float) – Days since ref_date for the time coordinate
• orbit (int) – SCIAMACHY/Envisat orbit number
• equtime (float) – UTC hour into the day at the equator
• scia_dens (scia_densities_pp instance) – The post-processed orbit data set

to_xarray()

Convert the combined orbit data to xarray.Dataset

Exports the data using the same data variables as when writing to netcdf.

Returns:dataset Return type:xarray.Dataset

write_to_netcdf(filename)

Write variables to netcdf files

Parameters:filename (str) – The name of the file to write the data to.

sciapy.level2.igrf

IGRF geomagnetic coordinates

This is a python (mix) version of GMPOLE and GMCOORD from http://www.ngdc.noaa.gov/geomag/geom_util/utilities_home.shtml to transform geodetic to geomagnetic coordinates. It uses the IGRF 2012 model and coefficients [1].

[1]	Thébault et al. 2015, International Geomagnetic Reference Field: the 12th generation. Earth, Planets and Space, 67 (79) http://nora.nerc.ac.uk/id/eprint/511258 https://doi.org/10.1186/s40623-015-0228-9

sciapy.level2.igrf.gmag_igrf(date, lat, lon, alt=0.0, centered_dipole=False, igrf_name='IGRF.tab')

Centered or eccentric dipole geomagnetic coordinates

Parameters:

• date (datetime.datetime)
• lat (float) – Geographic latitude in degrees north
• lon (float) – Geographic longitude in degrees east
• alt (float, optional) – Altitude in km. Default: 0.
• centered_dipole (bool, optional) – Returns the centered dipole geomagnetic coordinates if set to True, returns the eccentric dipole geomagnetic coordinates if set to False. Default: False
• igrf_name (str, optional) – Default: “IGRF.tab”

Returns:

• geomag_latitude (numpy.ndarray or float) – Geomagnetic latitude in eccentric dipole coordinates, centered dipole coordinates if centered_dipole is True.
• geomag_longitude (numpy.ndarray or float) – Geomagnetic longitude in eccentric dipole coordinates, centered dipole coordinates if centered_dipole is True.

sciapy.level2.igrf.gmpole(date, r_e=6371.2, filename='IGRF.tab')

Centered dipole geomagnetic pole coordinates

Parameters:

• date (datetime.datetime or datetime.date)
• r_e (float, optional) – Earth radius to evaluate the dipole’s off-centre shift.
• filename (str, optional) – File containing the IGRF parameters.

Returns:

• (lat_n, phi_n) (tuple of floats) – Geographic latitude and longitude of the centered dipole magnetic north pole.
• (lat_s, phi_s) (tuple of floats) – Geographic latitude and longitude of the centered dipole magnetic south pole.
• (dx, dy, dz) (tuple of floats) – Magnetic variations in Earth-centered Cartesian coordinates for shifting the dipole off-center.
• (dX, dY, dZ) (tuple of floats) – Magnetic variations in centered-dipole Cartesian coordinates for shifting the dipole off-center.
• B_0 (float) – The magnitude of the magnetic field.

sciapy.level2.post_process

SCIAMACHY level 2 data post processing

Main script for SCIAMACHY orbital retrieval post processing and data combining (to netcdf).

sciapy.level2.post_process.combine_orbit_data(orbits, ref_date='2000-01-01', L2_version='v6.2', dens_path=None, spec_base=None, save_nc=False)

Combine post-processed SCIAMACHY retrieved orbit data

Parameters:

• orbits (list) – List of SCIAMACHY/Envisat orbit numbers to process.
• ref_date (str, optional) – Base date to calculate the relative days from, of the format “%Y-%m-%d”. Default: 2000-01-01
• L2_version (str, optional) – SCIAMACHY level 2 data version to process
• dens_path (str, optional) – The path to the level 2 data. If None tries to infer the data directory from the L2 version looking for anything in the current directory that ends in <L2_version>: ./*<L2_version>. Default: None
• spec_base (str, optional) – The root path to the level 1c spectra. Uses the current dir if not set or set to None (default).
• save_nc (bool, optional) – Save the intermediate orbit data sets to netcdf files for debugging.

Returns:

(sdday, sdday_ds) – sdday contains the combined data as a scia_density_day instance, sdday_ds contains the same data as a xarray.Dataset.

Return type:

tuple

sciapy.level2.post_process.get_orbits_from_date(date, mlt=False, path=None, L2_version='v6.2')

Find SCIAMACHY orbits with retrieved data at a date

Parameters:

• date (str) – The date in the format “%Y-%m-%d”.
• mlt (bool, optional) – Look for MLT mode data instead of nominal mode data. Increases the heuristics to find all MLT orbits.
• path (str, optional) – The path to the level 2 data. If None tries to infer the data directory from the L2 version using ./*<L2_version>. Default: None

Returns:

orbits – List of found orbits with retrieved data files

Return type:

list

sciapy.level2.post_process.main()

SCIAMACHY level 2 post processing

sciapy.level2.post_process.process_orbit(orbit, ref_date='2000-01-01', dens_path=None, spec_base=None, use_msis=True)

Post process retrieved SCIAMACHY orbit

Parameters:

• orbit (int) – SCIAMACHY/Envisat orbit number of the results to process.
• ref_date (str, optional) – Base date to calculate the relative days from, of the format “%Y-%m-%d”. Default: 2000-01-01
• dens_path (str, optional) – The path to the level 2 data. Uses the current dir if not set or set to None (default).
• spec_base (str, optional) – The root path to the level 1c spectra. Uses the current dir if not set or set to None (default).

Returns:

(dts0, time0, lst0, lon0, sdd) – dts0 - days since ref_date at equator crossing (float) time0 - utc hour into the day at equator crossing (float) lst0 - apparent local solar time at the equator (float) lon0 - longitude of the equator crossing (float) sdd - scia_density_pp instance of the post-processed data

Return type:

tuple

sciapy.level2.post_process.read_spectra(year, orbit, spec_base=None, skip_upleg=True)

Read and examine SCIAMACHY orbit spectra

Reads the limb spactra and extracts the dates, times, latitudes, longitudes to be used to re-assess the retrieved geolocations.

Parameters:

• year (int) – The measurement year to select the corresponding subdir below spec_base (see below).
• orbit (int) – SCIAMACHY/Envisat orbit number of the spectra.
• spec_base (str, optional) – The root path to the level 1c spectra. Uses the current dir if not set or set to None (default).
• skip_upleg (bool, optional) – Skip upleg limb scans, i.e. night time scans. For NO retrievals, those are not used and should be not used here. Default: True

Return type:

(dts, times, lats, lons, mlsts, alsts, eotcorr)

sciapy.level2.post_process.sddata_set_attrs(sdday_ds, file_version='2.2', ref_date='2000-01-01', rename=True, species='NO')

Customize xarray Dataset variables and attributes

Changes the variable names to match those exported from the scia_density_day class.

Parameters:

• sdday_ds (xarray.Dataset instance) – The combined dataset.
• file_version (string “major.minor”, optional) – The netcdf file datase version, determines some variable names and attributes.
• ref_date (str, optional) – Base date to calculate the relative days from, of the format “%Y-%m-%d”. Default: 2000-01-01
• rename (bool, optional) – Rename the dataset variables to match the scia_density_day exported ones. Default: True
• species (str, optional) – The name of the level 2 species, used to prefix the dataset variables to be named <species>_<variable>. Default: “NO”.

sciapy.level2.post_process.solar_zenith_angle(alt, lat, lon, time)

sciapy.level2.scia_akm

SCIAMACHY level 2 averaging kernel interface

sciapy.level2.scia_akm.read_akm(filename, nalt, nlat)

Read SCIAMACHY level 2 averaging kernels into numpy array

Supports plain ascii (text) tables using numpy.genfromtxt() and netcdf files using xarray.

Parameters:

• filename (str) – Filename of the averaging kernel elements
• nalt (int) – Number of altitude bins of the retrieval
• nlat (int) – Number of latitude bins of the retrieval

Returns:

akm – The averaging kernel matrix elements.

Return type:

numpy.ndarray of shape (nalt, nlat, nalt, nlat)