On this Moodle page we are hosting a number of resources to help in handling datasets and prepare for the training course.
In order to be ready:
Instructions: Clicking on the section name will show / hide the section.
The activity makes use of datasets and tools. It is a good idea to get ready before to be efficient on your mini projects. We recommend you to:
1. Get ready by downloading the data. You will be introduced to these datasets, but you can also bring your own data. Datasets will include principal compounds from:
It is recommended to subscribe to the datasets (where required) before the school and be able to download the products / species of your interest.
2. Get ready with the tools to handle the data.You may still make use of your own software, but we encourage you to download and familiarize with the tools we will use in the school.
A large part of the practical sessions will be available on a JupyterHub instance. JupyterHub is a pre-defined environment that gives learners direct access to the data and Python packages required for following the practicals.
We invite you to browse through the content on the JupyterHub before the course. It is a safe learning environment, where you can make changes to the code and test the same code with e.g. a different dataset.
How to access the JupyterHub:
Once you are logged in, we recommend to get started with the index_ltpy_v01.ipynb, which gives you an overview of the course material available and other useful information.
NOTE: if you log into JupyterHub, a docker image will be created. To have a clean environment for the training course, we will delete all docker images before the course starts.
In addition you may use other existing tools as for instance:
conda install -c stcorp coda harp visan
These tools will also be run inside a Python environment. For this we advise also to make a local installation.
3. Make your local installation of python
To reproduce the course modules on your local setup, the following Python version and Python packages will be required:
Python version: Python3
Python packages include xarray, netCDF4, numpy, matplotlib and cartopy.
The main objectives of EUMETSAT Atmospheric Composition SAF (AC SAF, https://acsaf.org/index.html) is to process, archive, validate and disseminate atmospheric composition products from GOME-2 and IASI instruments onboard EUMETSAT Metop satellites. These products include e.g. total columns of different trace gases such as nitrogen dioxide (NO2), ozone (O3) and sulphur dioxide (SO2), tropospheric column of NO2, CO, UV-radiation, aerosol index as well as ozone profiles. This section describes the products relevant for this course obtained from GOME-2 instrument, while IASI products are described in separate section. Important applications of AC SAF data are e.g. monitoring global air quality or UV-radiation reaching the Earth's surface. In this section you will learn about AC SAF data products, what kind of studies they are useful for, and how you can access the data.
AC SAF provides three different types of datasets of GOME-2 observations: Near Real Time, Offline, and Data Records. The datasets that will be used in this course are:
From this section you can find basic information on trace gases (NO2 and Tropospheric O3 ) and aerosol parameters relevant for the course, and learn which kind of studies you can use the data for.
This section gives a short overview of GOME-2 NRT and Offline trace gas and aerosol data.
This section gives a short overview on GOME-2 Level 3 Data Records.
In this section you will learn how to register as a user to the DLR ATMOS and FMI data services. This is needed in order to download the trace gas (DLR) and aerosol (FMI) data.
In this section you will learn how you can access the data via DLR ATMOS and FMI data services.
You may access to the Sentinel 5P data in diverse ways:
1. With the ESA copernicus scihub
Browse the data and select the orbits / variable you would like to download (user and password: s5pguest). Here is also a description of the products.
2. With the TEMIS KNMI service for NO2 data
Daily monthly data are available through the TEMIS KNMI service.
3. Google Earth Engine to view and use TROPOMI S5P
The main objective of Google Earth Engine (GEE) is to provide easy access to satellite data and a processing platform (cloud) to process the data with your own codes. The datasets are provided as so-called L3 products. This means the satellite data is provided on a fixed latitude-longitude grid. The big advantage of using GEE is that there is no need to download large satellite datasets on your own computer.
In this course we will focus on NO2 and CO TROPOMI S5P data.
Tropospheric NO2 as measured by TROPOMI S5P July-Sept. 2018 and visualised using GEE.
Total column of CO as measured by TROPOMI S5P October 2018 and visualised using GEE.
In this section you can find basic information on Google Earth Engine and TROPOMI S5P data.
In this section you can find basic information on NO2 and CO and learn which kind of studies you can use the data for.
In this section you will learn how to register as a user to Google Earth Engine. This is needed in order to use the satellite data.
The Copernicus Atmosphere Monitoring Service (CAMS, https://atmosphere.copernicus.eu/) provides consistent and quality-controlled information related to air pollution and health, solar energy, greenhouse gases and climate forcing, everywhere in the world. It is implemented by the European Centre for Medium-Range Weather Forecasts (ECMWF) on behalf the European Commissions and is one of six Copernicus data services. The principal CAMS datsets are global forecasts and analyses of reactive gases (O3, CO, NO2, SO2, HCHO), greenhouse gases (CO2, CH4) and aerosol optical depth. In addition to the atmospheric composition products, CAMS also provides data on global fire emissions and inventories of anthropogenic and biogenic emissions. Examples of CAMS charts, for an aerosol forecast and fire activity analysis, are shown below and the latest charts can be viewed at https://atmosphere.copernicus.eu/charts/cams/.
CAMS analyses assimilate a wide range of satellite observations of meteorology and atmospheric composition (including from the Atmospheric Composition SAF and TROPOMI/Sentinel-5p), and initial conditions for the forecasts are taken from the analyses. A full list of the satellite observations can be found at https://atmosphere.copernicus.eu/satellite-observations. In situ observations made at the ground and from aircraft and balloons are also vital to CAMS and are used to regularly evaluate and validate the datasets. Validation reports for the CAMS data can be found at https://atmosphere.copernicus.eu/node/325.
In this section you will learn how to register as a user and download CAMS data. Registration is required to access CAMS datasets.
In this section you will learn how to download CAMS data and which settings to use.
Use this section to find help and support for accessing CAMS data.
Atmospheric aerosols are of great importance to global climate because of their scattering as well as absorbing properties, in turn significantly influencing the Earth's radiation budget. In general, aerosols could offset climate warming by directly scattering the sunlight back to space and by indirectly enhancing cloud albedo, thereby cooling the climate. However, it is also known that absorbing aerosols (such as black carbon and dust) warm the atmosphere because of their absorption of sunlight, which in turn enhances climate warming.
The MODIS satellite sensors, onboard NASA’s Terra and Aqua satellites, provide a long-term climate data record of aerosols since the early 2000s. The specific quantity retrieved by MODIS data pertaining to aerosols is the column-integrated Aerosol Optical Depth (AOD) which is a measure of the total extinction (scattering and absorption effects) due to aerosols in the atmosphere.
Aerosol Optical Depth from Terra MODIS data averaged for June 2017 [source- https://earthobservatory.nasa.gov/global-maps/MODAL2_M_AER_OD]
The MODIS aerosol datasets have been widely used in the climate science and air pollution/air quality research communities worldwide in the last two decades, with extensive validation efforts carried out by the MODIS science team and researchers around the world. The MODIS aerosol data are extensively used in global and regional studies related to aerosol characterization, aerosol radiative effects, aerosol-cloud-precipitation interactions and long-term aerosol trend analysis. In recent years, the MODIS AOD products are increasingly being used in global and regional air quality assessment and monitoring as well, owing to the granularity and accuracy of the AOD data. Another relevant area of research has been application of MODIS aerosol datasets in transboundary air pollution transport studies associated with extreme air pollution episodes, forest fires, biomass burning and desert dust outbreaks.
AOD data from MODIS are available on a daily basis in Level-2 (swath data) and Level-3 (gridded data) as well as on a monthly mean basis (Level-3 data). The Level-2 AOD data are available as high resolution data at 10 km, 3 km and 1 km resolution - suitable for aerosol characterization and air quality studies, whereas the Level-3 data are uniformly gridded to 1 degree x 1 degree spatial resolution - more applicable for aerosol-climate and large-scale aerosol transport studies.
Relevant parameters in MODIS Level-2 Aerosol Optical Depth product files (MOD04 for Terra, MYD04 for Aqua)
We focus on the ladsweb site to download MODIS Level-2 aerosol optical depth (AOD) product.
Install the prerequisite Python packages necessary to handle all data. Packages include hdf, basemap matplotlib, xarray and ncdf packages that don't come preinstalled in anaconda and will need to be installed separately for reading and mapping datasets in different format (MODIS in hdf, other in NCDF).
The main objective of Cloud-Aerosol Lidar and Infrared Pathfinder Satellite Observation (CALIPSO) satellite is to provide the vertical distribution of different types of aerosols (airborne particles) and clouds at global scale based on their optical properties. This course will focus on different types of aerosol and cloud data products (such as aerosol/cloud layer and profile products) available from space-borne Lidar called Cloud-Aerosol Lidar with Orthogonal Polarization (CALIOP) onboard CALIPSO satellite. These data products have various applications in monitoring the Earth’s weather, climate and air-quality. More information about CALIPSO satellite can be found at:
CALIOP measures backscattered light (at 532 nm and 1064 nm wavelengths) from aerosol and cloud layers present in the Earth’s atmosphere. With the help of different algorithms, this information is used to detect aerosol/cloud layers and also to derive optical properties (such as backscattering coefficient, extinction coefficient, depolarization ratio, colour ratio, layer optical depth) as a function of altitude along the CALIPSO orbit track. These properties are then used to discriminate aerosols from clouds, ice/water phase identification and to further classify into aerosol subtypes (clean, polluted dust, smoke, etc.) and cloud sub-types (cirrus, altocumulus, deep convective, etc.).
Vertical Feature Mask product of CALIOP along its track showing vertical distribution of aerosol and cloud layers.
Detailed information about various CALIOP data products and Algorithm Theoretical Basis Documents (ATBD) are available at https://eosweb.larc.nasa.gov/project/calipso/calipso_table.
Basic information on aerosols and clouds
This section will introduce you to aerosols and clouds in the Earth’s atmosphere. You will learn how aerosols and clouds impact our Earth’s weather, climate and air-quality. In addition, it will allow you to understand why and how do we monitor the distribution of aerosols and clouds at global scale. Following articles will help you in understanding the fundamentals of aerosols and clouds:
CALIOP data files are in HDF formats. There are many ways through which one can visualize and analyze CALIOP data, some of them are listed here.
CALIOP data can be accessed through the following sources as described here.
IASI is an infrared Fourier transform spectrometer developed jointly by CNES (the French spatial agency) with support of the scientific community, and by EUMETSAT. IASI is mounted on-board the European polar-orbiting MetOp satellite with the primary objective to improve numerical weather predictions, by measuring tropospheric temperature and humidity with high horizontal resolution and sampling. IASI also contributes greatly to atmospheric composition measurements for climate and chemistry applications, providing observations both day and night. IASI retrieves observations of several trace gases and aerosols, for this course primarily carbon monoxide (CO) will be used. Currently, from IASI observations CO and sulphur dioxide (SO2) are part of AC SAF product family.
A short guide on how to access IASI CO data
