Commit 78c3abfa authored by Julia Wagemann's avatar Julia Wagemann
Browse files

Practical content for 4th ACAM training school

parent 9e17ecc1
%% Cell type:markdown id: tags:
<img src='./img/acam_banner.png' alt='Logo EU Copernicus EUMETSAT' align='centre' width='70%'></img>
%% Cell type:markdown id: tags:
<br>
%% Cell type:markdown id: tags:
# 4th ACAM Training School
%% Cell type:markdown id: tags:
Jointly organized by [EUMETSAT](https://www.eumetsat.int/website/home/index.html), [CAMS-ECMWF](https://atmosphere.copernicus.eu/) and [ICIMOD](https://www.icimod.org/).
%% Cell type:markdown id: tags:
The `4th ACAM Training School` focuses on Atmospheric Chemistry and Aerosols in the Asian Monsoon region using satellite- and model-based data.
The course is a combination of lectures and practical sessions introducing you to different satellite- and model-based data for wildfire monitoring and hightlighting the asian summer monsoon anticyclone. The `practical` sessions have the following outline:
* 22 June | 8-9:30 (UTC): **Practical session: Asian summer monsoon anticyclone + Indonesian fires 2020 workflow**
* 23 June | 8-9:30 (UTC): **Practical session: Indonesian fires 2015 workflow**
* 24 June | 8-9:30 (UTC): **Practical session: Model-based data**
%% Cell type:markdown id: tags:
<br>
%% Cell type:markdown id: tags:
## Featured data
%% Cell type:markdown id: tags:
This course features the following data:
* **Satellite-based data products**
* MLS/Aura Level 3 data: [Carbon Monoxide](./01_asian_monsoon_anticyclone/11_MLS_CO_L3_ACAM.ipynb)
* AC-SAF Metop-A/B GOME-2 Level-2 data: [Formaldehyde (HCHO)](./02_fire_monitoring/11_AC-SAF_Metop-AB_GOME-2_HCHO.ipynb)
* AC-SAF Metop-A/B GOME-2 Level-2 data: [Absorbing Aerosol Index (AAI)](./02_fire_monitoring/12_AC-SAF_Metop-AB_GOME-2_AAI.ipynb) and [Absorbing Aerosol Height (AAH)](./02_fire_monitoring/13_AC-SAF_Metop-B_GOME-2_AAH.ipynb)
* Metop-A/B IASI Level-2 data: [Carbon Monoxide](./02_fire_monitoring/14_Metop-AB_IASI_CO.ipynb)
* Copernicus Sentinel-5P TROPOMI Level 2 data: [Ultraviolet Aerosol Index](./23_Sentinel-5P_TROPOMI_UVAI.ipynb) and [Carbon Monoxide](./22_Sentinel-5P_TROPOMI_CO.ipynb)
* Copernicus Sentinel-3 OLCI Level-1B data: [Red-Green-Blue (RGB) radiances](./21_Sentinel-3_OLCI_RGB.ipynb)
* **Model-based data products**
* Copernicus Atmosphere Monitoring Service (CAMS) | Global atmospheric composition forecasts: [Aerosol Optical Depth](./03_cams/3.1_CAMSDataAccess_ADS_GlobalAtmosphericCompositionForecast.ipynb)
* Copernicus Atmosphere Monitoring Service (CAMS) | Global reanalyses (EAC4): [Aerosol Optical Depth](./03_cams/3.2_CAMSDataAccess_ADS_GlobalAtmosphericCompositionReanalysis.ipynb)
* Copernicus Atmosphere Monitoring Service (CAMS) | Global Fire Assimilation System (GFAS): [Fire Radiative Power](./03_cams/3.3_CAMSDataAccess_FireEmissions.ipynb)
* Copernicus Atmosphere Monitoring Service (CAMS) | Greenhouse gas fluxes: [Posterior land surface upward flux](./3.4_CAMSDataAccess_GHGFluxInversions.ipynb)
%% Cell type:markdown id: tags:
<br>
%% Cell type:markdown id: tags:
## Course material
%% Cell type:markdown id: tags:
The course outline is as follows
* **00 - Introduction to Python and data overview**
* [Introduction to Python and Project Jupyter](./01_introduction_to_python_and_jupyter.ipynb)
* [Overview of data and data access systems](./02_atmospheric_composition_overview.ipynb)
* **01 - Asian Summer Monsoon Anticyclone**
* [1.0 MLS/Aura Carbon Monoxide Level 3](./01_asian_summer_monsoon_anticyclone/11_MLS_CO_L3_ACAM.ipynb)
* **02 - Satellite-based data for wildfire monitoring**
* [2.0 Indonesian fires 2015 workflow](./02_fire_monitoring/10_workflow_indonesia_fires_2015.ipynb)
* [2.1 Indonesian fires 2020 workflow](./02_fire_monitoring/20_workflow_indonesia_fires_2020.ipynb)
* **03 - Copernicus Atmosphere Monitoring Service (CAMS) data**
* [3.0 CAMS data introduction](./03_cams/3.0_CAMSDataAccess_Introduction.ipynb)
* [3.1 CAMS Global Atmospheric Composition Forecast](./03_cams/3.1_CAMSDataAccess_ADS_GlobalAtmospherericCompositionForecast.ipynb)
* [3.2 CAMS Global Atmospheric Composition Reanalysis](./03_cams/3.2_CAMSDataAccess_ADS_GlobalAtmospherericCompositionReanalysis.ipynb)
* [3.3 CAMS Fire Emissions](./03_cams/3.3_CAMSDataAccess_FireEmissions.ipynb)
* [3.4 CAMS Greenhouse Gase fluxes](./03_cams/3.4_CAMSDataAccess_GHGFluxInversions.ipynb)
<br>
**NOTE:** Throughout the course, general functions to `load`, `re-shape`, `process` and `visualize` the datasets are defined. These functions are re-used when applicable. The [functions notebook](./functions.ipynb) gives you an overview of all the functions defined and used for the course.
%% Cell type:markdown id: tags:
<br>
%% Cell type:markdown id: tags:
## Learning outcomes
%% Cell type:markdown id: tags:
The course is designed for `medium-level users`, who have basic Python knowledge and understanding of Fire monitoring data.
After the course, you should have:
* an idea about **different datasets on Fire Monitoring** and **atmospheric composition and the asian monsoon**,
* knowledge about the most useful **Python packages** to handle, process and visualise large volumes of Earth Observation data
* an idea about how the **data can help to detect and monitor fire events** as well as **the asian monsoon**.
%% Cell type:markdown id: tags:
<hr>
%% Cell type:markdown id: tags:
## Access to the `JupyterHub`
%% Cell type:markdown id: tags:
The course material is made available on a JupyterHub instance, a pre-defined environment that gives learners direct access to the data and Python packages required for following the course.
The `JupyterHub` can be accessed as follows:
%% Cell type:markdown id: tags:
* Web address: [https://training.ltpy.adamplatform.eu](https://training.ltpy.adamplatform.eu)
* Create an account: [https://login.ltpy.adamplatform.eu/](https://login.ltpy.adamplatform.eu/)
* Log into the `JupyterHub` with your account created.
%% Cell type:markdown id: tags:
<hr>
%% Cell type:markdown id: tags:
<img src='./img/copernicus_logo.png' alt='Logo EU Copernicus' align='right' width='20%'><br><br><br><br>
<p style="text-align:right;">This project is licensed under the <a href="./LICENSE">MIT License</a> and is developed under a Copernicus contract.
%% Cell type:markdown id: tags:
<img src='./img/acam_logos.png' alt='Logo EU Copernicus EUMETSAT' align='right' width='65%'></img>
%% Cell type:markdown id: tags:
<br>
%% Cell type:markdown id: tags:
<br>
%% Cell type:markdown id: tags:
# Optional: Introduction to Python and Project Jupyter
%% Cell type:markdown id: tags:
## Project Jupyter
%% Cell type:markdown id: tags:
<div class="alert alert-block alert-success" align="center">
<b><i>"Project Jupyter exists to develop open-source software, open-standards, and services for interactive computing across dozens of programming languages."</i></b>
</div>
%% Cell type:markdown id: tags:
<br>
%% Cell type:markdown id: tags:
Project Jupyter offers different tools to facilitate interactive computing, either with a web-based application (`Jupyter Notebooks`), an interactive development environment (`JupyterLab`) or via a `JupyterHub` that brings interactive computing to groups of users.
<br>
%% Cell type:markdown id: tags:
<center><img src='./img/jupyter_environment.png' alt='Logo Jupyter environment' width='60%'></img></center>
%% Cell type:markdown id: tags:
* **Jupyter Notebook** is an open-source web application that allows you to create and share documents that contain live code, equations, visualizations and narrative text.
* **JupyterLab 1.0: Jupyter’s Next-Generation Notebook Interface** <br> JupyterLab is a web-based interactive development environment for Jupyter notebooks, code, and data.
* **JupyterHub** <br>JupyterHub brings the power of notebooks to groups of users. It gives users access to computational environments and resources without burdening the users with installation and maintenance tasks. <br> Users - including students, researchers, and data scientists - can get their work done in their own workspaces on shared resources which can be managed efficiently by system administrators.
%% Cell type:markdown id: tags:
<hr>
%% Cell type:markdown id: tags:
## Why Jupyter Notebooks?
%% Cell type:markdown id: tags:
* Started with Python support, now **support of over 40 programming languages, including Python, R, Julia and more**
* Notebooks can **easily be shared via GitHub, NBViewer, etc.**
* **Code, data and visualizations are combined in one place**
* A great tool for **teaching**
* **JupyterHub allows you to access an environment ready to code**
%% Cell type:markdown id: tags:
## Installation
%% Cell type:markdown id: tags:
### Installing Jupyter using Anaconda
%% Cell type:markdown id: tags:
Anaconda comes with the Jupyter Notebook installed. You just have to download Anaconda and following the installation instructions. Once installed, the jupyter notebook can be started with:
%% Cell type:code id: tags:
``` python
jupyter notebook
```
%% Cell type:markdown id: tags:
### Installing Jupyter with pip
%% Cell type:markdown id: tags:
Experienced Python users may want to install Jupyter using Python's package manager `pip`.
With `Python3` you do:
%% Cell type:code id: tags:
``` python
python3 -m pip install --upgrade pip
python3 -m pip install jupyter
```
%% Cell type:markdown id: tags:
In order to run the notebook, you run the same command as with Anaconda at the Terminal :
%% Cell type:code id: tags:
``` python
jupyter notebook
```
%% Cell type:markdown id: tags:
## Jupyter notebooks UI
%% Cell type:markdown id: tags:
* Notebook dashboard
* Create new notebook
* Notebook editor (UI)
* Menu
* Toolbar
* Notebook area and cells
* Cell types
* Code
* Markdown
* Edit (green) vs. Command mode (blue)
%% Cell type:markdown id: tags:
<br>
<div style='text-align:center;'>
<figure><img src='./img/notebook_ui.png' width='100%'/>
<figcaption><i>Notebook editor User Interface (UI)</i></figcaption>
</figure>
</div>
%% Cell type:markdown id: tags:
## Shortcuts
%% Cell type:markdown id: tags:
Get an overview of the shortcuts by hitting `H` or go to `Help/Keyboard shortcuts`
%% Cell type:markdown id: tags:
#### Most useful shortcuts
%% Cell type:markdown id: tags:
* `Esc` - switch to command mode
* `B` - insert below
* `A` - insert above
* `M` - Change current cell to Markdown
* `Y` - Change current cell to code
* `DD` - Delete cell
* `Enter` - go back to edit mode
* `Esc + F` - Find and replace on your code
* `Shift + Down / Upwards` - Select multiple cells
* `Shift + M` - Merge multiple cells
%% Cell type:markdown id: tags:
## Cell magics
%% Cell type:markdown id: tags:
Magic commands can make your life a lot easier, as you only have one command instead of an entire function or multiple lines of code.<br>
> Go to an [extensive overview of magic commands]()
%% Cell type:markdown id: tags:
### Some of the handy ones
%% Cell type:markdown id: tags:
**Overview of available magic commands**
%% Cell type:code id: tags:
``` python
%lsmagic
```
%% Cell type:markdown id: tags:
**See and set environment variables**
%% Cell type:code id: tags:
``` python
%env
```
%% Cell type:markdown id: tags:
**Install and list libraries**
%% Cell type:code id: tags:
``` python
!pip install numpy
!pip list | grep pandas
```
%% Cell type:markdown id: tags:
**Write cell content to a Python file**
%% Cell type:code id: tags:
``` python
%%writefile hello_world.py
print('Hello World')
```
%% Cell type:markdown id: tags:
**Load a Python file**
%% Cell type:code id: tags:
``` python
%pycat hello_world.py
```
%% Cell type:markdown id: tags:
**Get the time of cell execution**
%% Cell type:code id: tags:
``` python
%%time
tmpList = []
for i in range(100):
tmpList.append(i+i)
print(tmpList)
```
%% Cell type:markdown id: tags:
**Show matplotlib plots inline**
%% Cell type:code id: tags:
``` python
%matplotlib inline
```
%% Cell type:markdown id: tags:
<br>
%% Cell type:markdown id: tags:
## Sharing Jupyter Notebooks
%% Cell type:markdown id: tags:
### Sharing static Jupyter Notebooks
%% Cell type:markdown id: tags:
* [nbviewer](https://nbviewer.jupyter.org/) - A simple way to share Jupyter Notebooks. You can simply paste the GitHub location of your Jupyter notebook there and it is nicely rendered.
* [GitHub](https://github.com/) - GitHub offers an internal rendering of Jupyter Notebooks. There are some limitations and time delays of the proper rendering. Thus, we would suggest to use `nbviewer` to share nicely rendered Jupyter Notebooks.
%% Cell type:markdown id: tags:
### Reproducible Jupyter Notebooks
%% Cell type:markdown id: tags:
<img src="./img/mybinder_logo.png" align="left" width="30%"></img>
[Binder](https://mybinder.org/) allows you to open notebooks hosted on a Git repo in an executable environment, making the code immediately reproducible by anyone, anywhere.
Binder builds a Docker image of the repo where the notebooks are hosted.
%% Cell type:markdown id: tags:
<br>
%% Cell type:markdown id: tags:
## Ressources
%% Cell type:markdown id: tags:
* [Project Jupyter](https://jupyter.org/)
* [JupyterHub](https://jupyterhub.readthedocs.io/en/stable/)
* [JupyterLab](https://jupyterlab.readthedocs.io/en/stable/)
* [nbviewer](https://nbviewer.jupyter.org/)
* [Binder](https://mybinder.org/)
%% Cell type:markdown id: tags:
<br>
%% Cell type:markdown id: tags:
<hr>
%% Cell type:markdown id: tags:
<img src='./img/copernicus_logo.png' alt='Logo EU Copernicus' align='right' width='20%'><br><br><br><br>
<p style="text-align:right;">This project is licensed under the <a href="./LICENSE">MIT License</a> and is developed under a Copernicus contract.
%% Cell type:markdown id: tags:
<img src='./img/acam_logos.png' alt='Logo EU Copernicus EUMETSAT' align='right' width='65%'></img>
%% Cell type:markdown id: tags:
<br>
%% Cell type:markdown id: tags:
# ACAM Training School Data - Overview and data access
%% Cell type:markdown id: tags:
This module gives an overview of the following atmospheric composition data services:
<a id="top"></a>
* **[NASA Goddard Space Flight Center Earth Science Data and Information Services Center (GES-DISC)](#mls_aura_ov)**
* [MLS/Aura Level 3 Monthly Binned Carbon Monoxide](./01_asian_summer_monsoon_anticyclone/11_MLS_CO_L3_ACAM.ipynb)
* **[Tropospheric Emission Monitoring Internet Service (TEMIS)](#temis)**
* [AC SAF Metop-A/B GOME-2 Absorbing Aerosol Index](./02_fire_monitoring/12_AC-SAF_Metop-AB_GOME-2_AAI.ipynb)
* **[EUMETSAT AC SAF - The EUMETSAT Satellite Application Facility on Atmospheric Composition Monitoring](#ac_saf)**
* [AC SAF Metop-A/B GOME-2 Aborbing Aerosol Height (AAH)](./02_fire_monitoring/13_AC-SAF_Metop-B_GOME-2_AAH.ipynb)
* **[Aeris IASI - the IASI Portal](#aeris)**
* [Metop-A/B IASI Carbon Monoxide](./02_fire_monitoring/14_Metop-AB_IASI_CO.ipynb)
* **[CODA - Copernicus Online Data Access](#coda_s3)**
* [Sentinel-3 OLCI Level-1B radiances](./02_fire_monitoring/21_Sentinel-3_OLCI_RGB.ipynb)
* **[Copernicus Sentinel-5P Pre-Operations Data Hub](#sentinel_5p)**
* [Sentinel-5P TROPOMI Carbon Monoxide](./02_fire_monitoring/22_Sentinel-5P_TROPOMI_CO.ipynb)
* [Sentinel-5P TROPOMI Ultraviolet Aerosol Index](./02_fire_monitoring/23_Sentinel-5P_TROPOMI_UVAI.ipynb)
* **[ECMWF Web Archive](#ecmwf)**
* [CAMS Global Fire Assimilation System (GFAS) - Fire Radiative Power](./03_cams/3.4_CAMSDataAccess_FireGHG.ipynb)
%% Cell type:markdown id: tags:
<hr>
%% Cell type:markdown id: tags:
## NASA Goddard Space Flight Center Earth Science Data and Information Services Center (GES-DISC)
%% Cell type:markdown id: tags:
> Data available: [MLS/Aura Level 3 Monthly binned Carbon Monoxide](./01_asian_summer_monsoon_anticyclone/11_MLS_CO_L3_ACAM.ipynb)
%% Cell type:markdown id: tags:
Binned (Level 3) MLS/Aura data is available via the [NASA Goddard Space Flight Center Earth Science Data and Information Services Center (GES-DISC)](https://disc.gsfc.nasa.gov/datasets/ML3MBCO_004/summary?keywords=mls%20aura%20level%203%20carbon%20monoxide).
GES-DISC offers monthly and daily aggregated binned MLS/Aura Carbon Monoxide information. The data is available in `netCDF` format for download.
%% Cell type:markdown id: tags:
<div style='text-align:center;'>
<figure><img src='./img/ges_disc.png' width='100%' />
<figcaption><i>The GES DISC web interface</i></figcaption>
</figure>
</div>
%% Cell type:markdown id: tags:
[^Return to top](#top)
%% Cell type:markdown id: tags:
<hr>
%% Cell type:markdown id: tags:
## <a id='temis'></a> Tropospheric Emission Monitoring Internet Service (TEMIS)
%% Cell type:markdown id: tags:
> Data available: [AC SAF Metop-A/B GOME-2 Absorbing Aerosol Index (AAI) Level 3](./02_fire_monitoring/12_AC-SAF_Metop-AB_GOME-2_AAI.ipynb)
%% Cell type:markdown id: tags:
Gridded (Level 3) Metop-A/B/C GOME-2 Absorbing Aerosol Index (AAI) data are available via the [Tropospheric Emission Monitoring Internet Service (TEMIS)](https://www.temis.nl/airpollution/absaai/). The data of the GOME-2 Absorbing Aerosol Index are provided by KNMI in the framework of the [EUMETSAT Satellite Application Facility on Atmospheric Composition Monitoring (AC SAF)](https://www.acsaf.org).
TEMIS offers monthly and daily aggregated gridded Metop-A/B/C GOME-2 Absorbing Aerosol Index (AAI) information. The data is available in `netCDF` for download.
%% Cell type:markdown id: tags:
<br>
%% Cell type:markdown id: tags:
<div style='text-align:center;'>
<figure><img src='../../img/temis_aai.png' width='60%' />
<figcaption><i>The TEMIS web interface</i></figcaption>
</figure>
</div>
%% Cell type:markdown id: tags:
[^Return to top](#top)
%% Cell type:markdown id: tags:
<hr>
%% Cell type:markdown id: tags:
## <a id="ac_saf"></a>EUMETSAT AC SAF - The EUMETSAT Satellite Application Facility on Atmospheric Composition Monitoring
%% Cell type:markdown id: tags:
> Data available: [AC SAF Metop-A/B GOME-2 Aborbing Aerosol Height (AAH)](./02_fire_monitoring/13_AC-SAF_Metop-B_GOME-2_AAH.ipynb)
%% Cell type:markdown id: tags:
<span style=float:left><img src='./img/ac_saf_logo.png' alt='Logo EU Copernicus EUMETSAT' align='left' width='90%'></img></span>
The [EUMETSAT Satellite Application Facility on Atmospheric Composition Monitoring (EUMETSAT AC SAF)](http://acsaf.org/) is one of eight EUMETSAT Satellite Application Facilities (SAFs). <br>
SAFs generate and disseminate operational EUMETSAT products and services and are an integral part of the distributed EUMETSAT Application Ground Segment.
AC SAF processes data on ozone, other trace gases, aerosols and ultraviolet data, obtained from satellite instrumentation.
%% Cell type:markdown id: tags:
<br>
%% Cell type:markdown id: tags: