<imgsrc='./img/EU-Copernicus-EUM_3Logos.png'alt='Logo EU Copernicus EUMETSAT'align='right'width='50%'></img>
%% Cell type:markdown id: tags:
<br>
%% Cell type:markdown id: tags:
# LTPy - Learning Tool for Python on Atmospheric Composition Data
%% Cell type:markdown id: tags:
<br>
%% Cell type:markdown id: tags:
**LTPy - Learning tool for Python on Atmospheric Composition Data** is a Python-based training course on Atmospheric Composition Data. The training course covers [10 - DATA ACCESS](#data_access), [20 - DATA DISCOVERY](#data_discovery), [30 - CASE STUDIES](#case_studies) and [40 - EXERCISES](#exercises) of satellite- and model-based data on Atmospheric Composition.
The course is based on [Jupyter notebooks](https://jupyter.org/), which allow for a high-level of interactive learning, as code, text description and visualisation is combined in one place. If you have not worked with `Jupyter Notebooks` before, you can look at the module [01 - Python and Project Jupyter 101](./01_Python_and_Jupyter_101.ipynb) to get a short introduction to Jupyter notebooks and their benefits.
%% Cell type:markdown id: tags:
<hr>
%% Cell type:markdown id: tags:
## Data on Atmospheric Composition
%% Cell type:markdown id: tags:
This course features the following **satellite** data:
*`Metop-A/B GOME-2 Level 2` data
*`Metop-A/B GOME-2 Level 3` reprocessed and regridded data
*`Metop-A/B/C GOME-2 Level 2` data
*`Metop-A/B/C GOME-2 Level 3` reprocessed and regridded data
*`Polar Multi-Sensor Aerosol Optical Properties (PMAp) Level 2` data
*`Metop-A/B IASI Level 2` data
*`Metop-A/B/C IASI Level 2` data
*`Copernicus Sentinel-5P TROPOMI Level 2` data
*`Copernicus Sentinel-3 OLCI Level 1B` data
*`Copernicus Sentinel-3 SLSTR NRT FRP Level 2` data
*`Copernicus Sentinel-3 SLSTR NRT AOD Level 2` data
And the following **model-based** data:
*`Copernicus Atmosphere Monitoring Service (CAMS) Global Reanalysis (EAC4)` data
*`Copernicus Atmosphere Monitoring Service (CAMS) Global Fire Assimilation System (GFAS)` data
*`Coperncus Emergency Management Service (CEMS) Global ECMWF Fire Forecast (GEFF)` data
%% Cell type:markdown id: tags:
<hr>
%% Cell type:markdown id: tags:
## Course material
%% Cell type:markdown id: tags:
The course follows a modular approach and offers modules on:
-[10 - DATA ACCESS](#data_access)
-[20 - DATA DISCOVERY](#data_discovery)
-[30 - CASE STUDIES](#case_studies)
-[40 - EXERCISES](#exercises)
<br>
<divclass="alert alert-block alert-info">
<b><aid='data_access'></a>10 - DATA ACCESS</b>
</div>
*[11 - Atmospheric Composition data overview and acccess](./10_data_access/11_ac_data_access_overview.ipynb)
*[12 - WEkEO Harmonized Data Access API](./10_data_access/12_WEkEO_harmonized_data_access_api.ipynb)
<br>
<divclass="alert alert-block alert-success">
<b><aid='data_discovery'></a>20 - DATA DISCOVERY</b>
</div>
#### *Metop-A/B/C GOME-2 Level 2 and Level 3 data*
#### *Copernicus Atmosphere Monitoring Service (CAMS) data*
*[261 - CAMS Global reanalysis (EAC4) - Organic Matter Aerosol Optical Depth - Load and browse](./20_data_discovery/261_CAMS_EAC4_OMAOD_load_browse.ipynb)
*[262 - CAMS Global Fire Assimilation System (GFAS) - Fire Radiative Power - Load and browse](./20_data_discovery/262_CAMS_GFAS_FRPFIRE_load_browse.ipynb)
#### *Copernicus Emergency Management Service (CEMS) data*
*[271 - CEMS Global ECMWF Fire Forecast - Fire Weather Index - Load and browse](./20_data_discovery/271_CEMS_GEFF_FWI_load_browse.ipynb)
*[272 - CEMS Global ECMWF Fire Forecast - Fire Weather Index - Harmonized Danger Classes](./20_data_discovery/272_CEMS_GEFF_FWI_harmonized_danger_classes.ipynb)
*[273 - CEMS Global ECMWF Fire Forecast - Fire Weather Index - Custom Danger Classes](./20_data_discovery/273_CEMS_GEFF_FWI_custom_danger_classes.ipynb)
**NOTE:** Throughout the course, general functions to `load`, `re-shape`, `process` and `visualize` the datasets are defined. These functions are re-used when applicable. The [functions notebook](./functions.ipynb) gives you an overview of all the functions defined and used for the course.
If a notebook makes use of these functions, they are loaded as **helper functions** at the beginning of the notebook. With `?function_name`, you can load the function's docstring to see what it does and which keyword arguments the function requires.
See the example to load the docstring of the function [visualize_pcolormesh](./functions.ipynb#visualize_pcolormesh):
%% Cell type:code id: tags:
``` python
%run./functions.ipynb
```
%% Cell type:code id: tags:
``` python
?visualize_pcolormesh
```
%% Cell type:markdown id: tags:
<hr>
%% Cell type:markdown id: tags:
## Learning outcomes
%% Cell type:markdown id: tags:
The course is designed for `medium-level users`, who have basic Python knowledge and understanding of Atmospheric composition data.
After the course, you should have:
* an idea about the **different datasets on Atmospheric Composition data**,
* knowledge about the most useful **Python packages** to handle, process and visualise large volumes of Earth Observation data
* an idea about different **data application areas**
%% Cell type:markdown id: tags:
<hr>
%% Cell type:markdown id: tags:
## Access to the `LTPy JupyterHub`
%% Cell type:markdown id: tags:
The course material is made available on a JupyterHub instance, a pre-defined environment that give learners direct access to the data and Python packages required for following the course.
The `JupyterHub` can be accessed as follows:
%% Cell type:markdown id: tags:
* Web address: [https://ltpy.adamplatform.eu](https://ltpy.adamplatform.eu)
* Create an account: [https://login.ltpy.adamplatform.eu/](https://login.ltpy.adamplatform.eu/)
* Log into the `JupyterHub` with your account created.
%% Cell type:markdown id: tags:
<hr>
%% Cell type:markdown id: tags:
## Reproduce LTPy on Atmospheric Compostion data locally
%% Cell type:markdown id: tags:
In case you wish to reproduce the course modules on your local setup, the following Python version and Python packages will be required:
* Python version: **Python3.8**
* Python packages: see [requirements.txt](./requirements.txt)
Python packages can be installed as follows: `pip install -r requirements.txt`.
%% Cell type:markdown id: tags:
The `eodata` folder with all the data required for the training course can be accessed and downloaded from [https://sftp.eumetsat.int](https://sftp.eumetsat.int/login). Find the user name and password in order to be able to login [here](https://gitlab.eumetsat.int/eumetlab/atmosphere/atmosphere/-/blob/master/sftp_login.txt).
%% Cell type:markdown id: tags:
<hr>
%% Cell type:markdown id: tags:
<p><imgsrc='./img/copernicus_logo.png'align='left'alt='Logo EU Copernicus'width='25%'></img></p>
<brclear=left>
<pstyle="text-align:left;">This project is licensed under the <ahref="./LICENSE">MIT License</a><spanstyle="float:right;"><ahref="https://gitlab.eumetsat.int/eumetlab/atmosphere/atmosphere">View on GitLab</a> | <ahref="https://training.eumetsat.int/">EUMETSAT Training</a>
The Copernicus Atmopshere Monitoring Service (CAMS) provides consistent and quality-controlled information related to air pollution and health and greenhouse gases. CAMS data consist of `global forecasts and analyses`, `global reanalyses (EAC4)`, `fire emissions` and `greenhouse gas flux inversions`.
This notebook is an `exercise workbook`, where you can practise the data handling with CAMS Global reanalysis (EAC4) data.
This notebook is an `exercise workbook`, where you can practise the data handling with CAMS Global reanalysis (EAC4) data.
%% Cell type:markdown id: tags:
### How it works
%% Cell type:markdown id: tags:
This exercise workbook consists of two types of exercises:
%% Cell type:markdown id: tags:
#### Coding assignments
Coding assignments ask you to fill an empty code cell with code.
You recognize `coding assignments` as the yellow-coloured boxes.
<divclass="alert alert-block alert-warning">
<b>Coding assignment</b>
</div>
%% Cell type:markdown id: tags:
#### Questions
Questions ask you to reflect on a result and output. `Questions` are provided as green-coloured boxes.
<divclass="alert alert-block alert-success">
<b>Question</b>
</div>
%% Cell type:markdown id: tags:
<br>
%% Cell type:markdown id: tags:
### Outline:
*[1 - Load and browse EAC4 reanalysis data](#load_browse)
*[2 - Bring longitude coordinates onto a [-180,180] grid](#shift)
*[3 - Retrieve the data variable total_colomn_carbon_monoxide as xarray.DataArray](#retrieve)
*[4 - Visualize total column carbon monoxide reanalysis data](#visualize)
### <a id='load_browse'></a>Load and browse `EAC4 reanalysis` data
%% Cell type:markdown id: tags:
CAMS global reanalysis data is available either in `GRIB` or `netCDF`. The data for this example is available in `netCDF`. You can use xarray's function `xr.open_dataset()` to open the netCDF file as a xarray.Dataset.
* time (time) datetime64[ns] 2019-12-25 ... 2019-12-31T21:00:00
Data variables:
omaod550 (time, latitude, longitude) float32 ...
aod550 (time, latitude, longitude) float32 ...
tcco (time, latitude, longitude) float32 ...
Attributes:
Conventions: CF-1.6
history: 2020-09-25 08:11:01 GMT by grib_to_netcdf-2.16.0: /opt/ecmw...
%% Cell type:markdown id: tags:
<divclass="alert alert-block alert-success">
<b>Question:</b><br>
Let us inspect the three dimensions (`latitude`, `longitude` and `time`) of the loaded file:
- What data variables does it contain and what `long_name` do they have?
</div>
%% Cell type:markdown id: tags:
You can retrieve the `long_name` of a data variable as follows: `file.omaod550.long_name` retrieves the long name of the `omaod550` variable.
The file contains three data variables:
*`omaod550`: Organic Matter Aerosol Optical Depth at 550nm
*`aod550`: Total Aerosol Optical Depth at 550nm
*`tcco`: Total column Carbon monoxide
%% Cell type:markdown id: tags:
<br>
%% Cell type:markdown id: tags:
### <a id='shift'></a>Bring longitude coordinates onto a [-180,180] grid
%% Cell type:markdown id: tags:
You can assign new values to coordinates in an xarray.Dataset. You can do so with the `assign_coords()` function from the xarray library, which you can apply onto an xarray.Dataset.
%% Cell type:markdown id: tags:
<divclass="alert alert-block alert-warning">
<b>Coding assignment:</b><br>
Enter below the code which re-assigns the longitude values from a [0,360] grid to a [-180,180] degrees grid.<br>
Inspect the newly assigned `longitude` coordinates afterwards and sort the longitude values in an ascending order.
<br>
**HINT:**<br>
You can use xarray's function `assign_coords()` for this operation.
Let us visualize the dataset. You can use the function [visualize_pcolormesh](../functions.ipynb#visualize_pcolormesh), which makes use of matploblib's function `pcolormesh` and the [Cartopy](https://scitools.org.uk/cartopy/docs/latest/) library.
%% Cell type:markdown id: tags:
<divclass="alert alert-block alert-warning">
<b>Coding assignment:</b><br>
Enter below the code that visualizes the `total column carbon monoxide` xarray.DataArray and make use of the following:
* the function [visualize_pcolormesh](../functions.ipynb#visualize_pcolormesh),