Commit 58ccf801 authored by Julia Wagemann's avatar Julia Wagemann
Browse files

EGU SC 2021 added

parent 9c3fdbf6
%% Cell type:markdown id: tags:
<img src='./img/egu21_logo.png' alt='Logo EU Copernicus EUMETSAT' align='left' width='25%'></img><img src='./img/atmos_logos.png' alt='Logo EU Copernicus EUMETSAT' align='right' width='65%'></img></span>
<br>
%% Cell type:markdown id: tags:
<br>
%% Cell type:markdown id: tags:
<br>
%% Cell type:markdown id: tags:
#### EGU Short Course 5.5 - Using Copernicus data for Atmospheric Composition Applications | 28 April + 5 May 2021
%% Cell type:markdown id: tags:
# Discover satellite data for Dust Monitoring
%% Cell type:markdown id: tags:
Jointly organized by [EUMETSAT](https://www.eumetsat.int/website/home/index.html), [CAMS-ECMWF](https://atmosphere.copernicus.eu/) and [ESA](https://www.esa.int), with support from [Copernicus](https://www.copernicus.eu/en).
%% Cell type:markdown id: tags:
This practical session gives you an overview of different satellite data for dust monitoring and provides you hands-on practical example how to retrieve, load, process and visualize the data.
%% Cell type:markdown id: tags:
<br>
%% Cell type:markdown id: tags:
<img src='./img/egu_short_course_logo.jpg' alt='Logo EU Copernicus EUMETSAT' align='center' width='70%'></img>
%% Cell type:markdown id: tags:
<br>
%% Cell type:markdown id: tags:
## Notebook overview and featured data
%% Cell type:markdown id: tags:
### The following satellite datasets for dust monitoring are featured:
* [CAMS Global Near-Real-Time Forecast](https://apps.ecmwf.int/datasets/data/cams-nrealtime/levtype=sfc/)
* [CAMS European Air Quality Forecasts and Analyses](https://ads.atmosphere.copernicus.eu/cdsapp#!/dataset/cams-europe-air-quality-forecasts?tab=overview)
* [Sentinel-5P TROPOMI Level 2 Ultraviolet Aerosol Index](https://sentinels.copernicus.eu/web/sentinel/data-products/-/asset_publisher/fp37fc19FN8F/content/sentinel-5-precursor-level-2-ultraviolet-aerosol-index)
* [AC SAF GOME-2 Level 3 Absorbing Aerosol Index](https://acsaf.org/index.html)
%% Cell type:markdown id: tags:
<br>
%% Cell type:markdown id: tags:
### The following notebooks are available:
#### Data discovery
* [01 - Overview - Atmospheric Composition Data - Overview and Retrieve](./01_overview_atmospheric_composition_data_retrieve.ipynb)
* [02 - CAMS global near-real-time forecast - Dust Aerosol Optical Depth - Load and browse](./02_CAMS_global_forecast_duaod_load_browse.ipynb)
* [03 - CAMS European Air Quality Forecasts and Analyses - Dust concentration - Load and browse](./03_CAMS_European_air_quality_forecast_dust_concentration_load_browse.ipynb)
* [04 - Sentinel-5P TROPOMI Ultraviolet Aerosol Index Level 2 - Load and browse](./04_Sentinel-5P_TROPOMI_AI_L2_load_browse.ipynb)
* [05 - Metop-ABC GOME-2 Absorbing Aerosol Index Level 3 - Load and browse](./05_Metop-ABC_GOME-2_AAI_L3_load_browse.ipynb)
#### Exercises
* [12 - CAMS global near-real-time forecast - Dust Aerosol Optical Depth - Exercise](./12_CAMS_global_forecast_duaod_exercise.ipynb)
* [13 - CAMS European Air Quality Forecasts and Analyses - Dust concentration - Exercise](./13_CAMS_European_air_quality_forecast_dust_concentration_exercise.ipynb)
* [14 - Sentinel-5P TROPOMI Ultraviolet Aerosol Index Level 2 - Exercise](./14_Sentinel-5P_TROPOMI_AI_L2_exercise.ipynb)
* [15 - Metop-ABC GOME-2 Absorbing Aerosol Index Level 3 - Exercise](./15_Metop-ABC_GOME-2_AAI_L3_exercise.ipynb)
%% Cell type:markdown id: tags:
<br>
%% Cell type:markdown id: tags:
**NOTE:** Throughout the course, general functions to `load`, `re-shape`, `process` and `visualize` the datasets are defined. These functions are re-used when applicable. The [functions notebook](./functions.ipynb) gives you an overview of all the functions defined and used for the course.
%% Cell type:markdown id: tags:
<hr>
%% Cell type:markdown id: tags:
## Learning outcomes
%% Cell type:markdown id: tags:
The course is designed for `beginners and medium-level users`, who have basic Python knowledge.
After the course, you should have:
* an idea about **satellite and assimilated data products for dust monitoring**,
* knowledge about the most useful **Python packages** to handle, process and visualise atmospheric composition data
* an idea about how the **data can help to detect and monitor dust events**
%% Cell type:markdown id: tags:
<hr>
%% Cell type:markdown id: tags:
## Access to the training platform
%% Cell type:markdown id: tags:
The course material is made available on a JupyterLab training platform, a pre-defined environment that gives learners direct access to the data and Python packages required for following the course.
The `JupyterLab` can be accessed as follows:
%% Cell type:markdown id: tags:
* Web address: [https://training.ltpy.adamplatform.eu](https://training.ltpy.adamplatform.eu)
* Create an account: [https://login.ltpy.adamplatform.eu](https://login.ltpy.adamplatform.eu)
* Log into the `JupyterLab` with your account created.
%% Cell type:markdown id: tags:
<hr>
%% Cell type:markdown id: tags:
<img src='../../img/copernicus_logo.png' alt='Logo EU Copernicus' align='right' width='20%'><br><br><br><br><br>
<p style="text-align:right;">This project is licensed under the <a href="./LICENSE">MIT License</a> and is developed under a Copernicus contract.
%% Cell type:markdown id: tags:
<img src='./img/egu21_logo.png' alt='Logo EU Copernicus EUMETSAT' align='left' width='25%'></img><img src='./img/atmos_logos.png' alt='Logo EU Copernicus EUMETSAT' align='right' width='65%'></img></span>
<br>
%% Cell type:markdown id: tags:
<br>
%% Cell type:markdown id: tags:
<a href="./00_index.ipynb"><< Index</a><span style="float:right;"><a href="./02_CAMS_global_forecast_duaod_load_browse.ipynb">02 - CAMS global forecast - Load and browse >></a></span>
%% Cell type:markdown id: tags:
<hr>
%% Cell type:markdown id: tags:
# Atmospheric composition data - Overview and data access
%% Cell type:markdown id: tags:
This module gives an overview of the following atmospheric composition data services:
* [ECMWF Data Archive](#ecmwf_archive)
* [Copernicus Atmosphere Data Store (ADS)](#ads)
* [Sentinel-5P Pre-Operations data Hub](#s5p)
* [TEMIS - Tropospheric Emissions Monitoring Internet Service](#temis)
%% Cell type:markdown id: tags:
<hr>
%% Cell type:markdown id: tags:
<br>
%% Cell type:markdown id: tags:
## <a id='ecmwf_archive'></a>ECMWF data archive
%% Cell type:markdown id: tags:
[CAMS global near-real-time forecasts](https://apps.ecmwf.int/datasets/data/cams-nrealtime/levtype=sfc) can be accessed via the `ECMWF data archive`. ECMWF's data archive is called `Meteorological and Archival Retrieval System (MARS)` and provides access to ECMWF Public Datasets.
The archive can be accessed in two ways:
* via the [web interface](https://apps.ecmwf.int/datasets/) and
* via the [ECMWF Web API](https://confluence.ecmwf.int/display/WEBAPI/Access+ECMWF+Public+Datasets).
Subsequently, an example is shown how a you can use the ECMWF Web API to execute a MARS request with Python in order to download data in netCDF.
%% Cell type:markdown id: tags:
<br>
%% Cell type:markdown id: tags:
### 1. Register for an ECMWF user account
%% Cell type:markdown id: tags:
- Self-register at https://apps.ecmwf.int/registration/
- Login at https://apps.ecmwf.int/auth/login
%% Cell type:markdown id: tags:
<br>
%% Cell type:markdown id: tags:
### 2. Install the `ecmwfapi` python library
%% Cell type:markdown id: tags:
`pip install ecmwf-api-client`
%% Cell type:markdown id: tags:
<br>
%% Cell type:markdown id: tags:
### 3. Retrieve your API key
%% Cell type:markdown id: tags:
You can retrieve your API key at https://api.ecmwf.int/v1/key/. Add the `url`, `key` and `email` information, when you define the `ECMWFDataServer` (see below).
%% Cell type:markdown id: tags:
<br>
%% Cell type:markdown id: tags:
### 4. Execute a MARS request and download data as a `netCDF` file
%% Cell type:markdown id: tags:
Below, you see the principle of a `data retrieval` request. You can use the web interface to browse through the datasets. At the end, there is the option to generate the `data retrieval` request for the API.
<br>
<div class="alert alert-block alert-success">
<b>NOTE</b>: <br>
Per default, ECMWF data is stored on a grid with longitudes from 0 to 360 degrees. It can be reprojected to a regular geographic latitude-longitude grid, by setting the keyword argument <code>area</code> and <code>grid</code>. Per default, data is retrieved in <code>GRIB</code>. If you wish to retrieve the data in <code>netCDF</code>, you have to specify it by using the keyword argument <code>format</code>.</div>
<br>
The example below requests `Dust Aerosol Optical Depth at 550 nm` forecast data for 19 February 2021 in `netCDF` format and by setting the `area` keyword, the longitude values are reprojected to a [-180,180] grid.
%% Cell type:code id: tags:
``` python
#!/usr/bin/env python
from ecmwfapi import ECMWFDataServer
server = ECMWFDataServer(url="https://api.ecmwf.int/v1", key="#########################", email="########################")
# Retrieve data in NetCDF format
server.retrieve({
"class": "mc",
"dataset": "cams_nrealtime",
"date": "2021-02-19/to/2021-02-19",
"expver": "0001",
"levtype": "sfc",
"param": "209.210",
"step": "0/12/24/36/48/60/72/84/96/108/120",
"stream": "oper",
"time": "00:00:00",
"type": "fc",
"format": "netcdf",
"area": "90/-180/-90/180",
"grid": "0.4/0.4",
"target": "../eodata/cams/near_real_time/2021/02/19/20210219_dustAOD.nc",
})
```
%% Cell type:markdown id: tags:
<br>
%% Cell type:markdown id: tags:
<hr>
%% Cell type:markdown id: tags:
<br>
%% Cell type:markdown id: tags:
## <a id='ads'></a>Copernicus Atmosphere Data Store (ADS)
%% Cell type:markdown id: tags:
<br>
%% Cell type:markdown id: tags:
<div style='text-align:left;'>
<figure><img src='./img/ads_cams.png' width='50%' />
<figcaption><i>The web interface of the Copernicus Atmosphere Data Store</i></figcaption>
</figure>
</div>
%% Cell type:markdown id: tags:
<br>
%% Cell type:markdown id: tags:
The [CAMS European air quality forecasts](https://ads.atmosphere.copernicus.eu/cdsapp#!/dataset/cams-europe-air-quality-forecasts?tab=overview) are available via the `Copernicus Atmosphere Data Store (ADS)`. Data from the `ADS` can be accessed programmtically via the [cdsapi](https://pypi.org/project/cdsapi/) Python library.
To install the [cdsapi](https://pypi.org/project/cdsapi/) follow this [how-to guide](https://cds.climate.copernicus.eu/api-how-to). It includes the following steps:
%% Cell type:markdown id: tags:
### 1. Install the CDS API key
* [Self-register](https://ads.atmosphere.copernicus.eu/#!/home) at the CDS registration page
* [Login](https://ads.atmosphere.copernicus.eu/user/login) to the ADS portal and go to the [api-how-to page](https://ads.atmosphere.copernicus.eu/api-how-to)
* Copy the CDS API key displayed in a file under `$HOME/.cdsapirc` *(Unix / Linux environment)*. If you do not see a URL or key appear in the black terminal box on the `API How To` page, please refresh your browser tab.
%% Cell type:code id: tags:
``` python
%%writefile /home/jovyan/.cdsapirc
url: #########
key: ####:####################
```
%% Cell type:markdown id: tags:
<br>
%% Cell type:markdown id: tags:
### 2. Install the CDS API client
%% Cell type:markdown id: tags:
`pip install cdsapi`
%% Cell type:markdown id: tags:
<br>
%% Cell type:markdown id: tags:
### 3. Use the CDS API client for data access
%% Cell type:markdown id: tags:
Below, you see the principle of a `data retrieval` request. You can use the [web interface](https://ads.atmosphere.copernicus.eu/cdsapp#!/search?type=dataset) to browse through the datasets. Under `Download data`, at the end, there is the option to generate the `API request`.
<br>
<div class="alert alert-block alert-success">
<b>NOTE</b>: <br>
per default, ECMWF data is stored on a grid with longitudes from 0 to 360 degrees. It can be reprojected to a regular geographic latitude-longitude grid, by setting the keyword argument <code>area</code> and <code>grid</code>. Per default, data is retrieved in <code>GRIB</code>. If you wish to retrieve the data in <code>netCDF</code>, you have to specify it by using the keyword argument <code>format</code>.</div>
<br>
Below, it is the example request for `dust concentration of the European Air Quality forecasts` data from 19-25 February 2021 in `netCDF` format. By specifying the `area` keyword, the data is reprojected to a [-180,180] grid and geographically subsetted to Europe.
%% Cell type:markdown id: tags:
<br>
%% Cell type:code id: tags:
``` python
import cdsapi
```
%% Cell type:code id: tags:
``` python
c = cdsapi.Client()
c.retrieve(
'cams-europe-air-quality-forecasts',
{
'model': 'ensemble',
'date': '2021-02-19/2021-02-25',
'format': 'netcdf',
'variable': 'dust',
'level': '1000',
'type': 'analysis',
'area': [
71.56, -24.32, 30,
45,
],
'time': [
'00:00', '06:00', '12:00',
'18:00',
],
'leadtime_hour': '0',
},
'../eodata/cams/near_real_time/2021/02/19/20210219-25_regional_nrt_dust_concentation.nc')
```
%% Cell type:markdown id: tags:
<br>
%% Cell type:markdown id: tags:
<hr>
%% Cell type:markdown id: tags:
## <a id='s5p'></a> Sentinel-5P Pre-Operations Data Hub
%% Cell type:markdown id: tags:
Sentinel-5P data is available for browsing and downloading via the [Copernicus Open Access Hub](https://scihub.copernicus.eu/). The Copernicus Open Access Hub provides complete, free and open access to Sentinel-1, Sentinel-2, Sentinel-3 and Sentinel-5P data.
%% Cell type:markdown id: tags:
<div style='text-align:center;'>
<figure><img src='./img/open_access_hub.png' alt='Sentinel-5p data products' align='middle' width='50%'/>
<figcaption><i>Interface of the Copernicus Open Access Hub and the Sentinel-5P Pre-Operations Data Hub</i></figcaption>
</figure>
</div>
%% Cell type:markdown id: tags:
<br>
%% Cell type:markdown id: tags:
<hr>
%% Cell type:markdown id: tags:
## <a id='temis'></a> Tropospheric Emission Monitoring Internet Service (TEMIS)
%% Cell type:markdown id: tags:
Gridded (Level 3) Metop-A/B/C GOME-2 Absorbing Aerosol Index (AAI) data are available via the [Tropospheric Emission Monitoring Internet Service (TEMIS)](https://www.temis.nl/airpollution/absaai/). The data of the GOME-2 Absorbing Aerosol Index are provided by KNMI in the framework of the [EUMETSAT Satellite Application Facility on Atmospheric Composition Monitoring (AC SAF)](www.acsaf.org).
TEMIS offers monthly and daily aggregated gridded Metop-A/B/C GOME-2 Absorbing Aerosol Index (AAI) information. The data is available in netCDF for download.
%% Cell type:markdown id: tags:
<br>
%% Cell type:markdown id: tags:
<div style='text-align:center;'>
<figure><img src='../../img/temis_aai.png' width='60%' />
<figcaption><i>The TEMIS web interface</i></figcaption>
</figure>
</div>
%% Cell type:markdown id: tags:
<br>
%% Cell type:markdown id: tags:
<hr>
%% Cell type:markdown id: tags:
## Further information
%% Cell type:markdown id: tags:
* [Copernicus Atmosphere Monitoring Service](https://atmosphere.copernicus.eu/)
* [EUMETSAT AC SAF - The EUMETSAT Application Facility on Atmospheric Composition Monitoring](https://acsaf.org/index.html)
* [TEMIS - Tropospheric Emission Monitoring Internet Service](http://temis.nl/index.php)
* [Sentinel-5P Pre-Operations Data Hub](https://s5phub.copernicus.eu/dhus/#/home)
%% Cell type:markdown id: tags:
<br>
%% Cell type:markdown id: tags:
<a href="./00_index.ipynb"><< Index</a><span style="float:right;"><a href="./02_CAMS_global_forecast_duaod_load_browse.ipynb">02 - CAMS global forecast - Load and browse >></a></span>
%% Cell type:markdown id: tags:
<hr>
%% Cell type:markdown id: tags:
<img src='../../img/copernicus_logo.png' alt='Logo EU Copernicus' align='right' width='20%'><br><br><br><br><br>
<p style="text-align:right;">This project is licensed under the <a href="./LICENSE">MIT License</a> and is developed under a Copernicus contract.
MIT License
Copyright (c) 2021 EUMETSAT
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
This diff is collapsed.
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment