CONTRIBUTING.md

# CONTRIBUTING TO THE DATA TAILOR DEVELOPMENT

The following is a set of guidelines for contributing to the Data Tailor.


## Branching and release workflow
The branching and release workflow is very close to GitFlow.

The following main branches always exist:

* the **main** branch contains released software. Only the repository maintainer can merge to it.
* the **development** branch contains consolidated software developments.
 
When a developer wants to work on a new feature or a major bugfix, he needs to:

* [ ] create an issue on the project GitLab repository with a short descriptive title and giving it a "weight" (complexity): 0: straightforward, and assigning a milestone
* [ ] from the issue GitLab page, create a new branch with "development" as a parent. The branch name will be automatically generated by GitLab from the issue title
* [ ] when working on a issue, put it in a "Doing" state (issue tag)
* [ ] work on the new feature on the new branch, and commit changes to it, quoting the issue
* [ ] once the work is complete, create a merge request to "development", quoting the issue number and resolution
* [ ] update the "[unpublished]" section in CHANGES.md to add a one-line description of the closed issues (for humans) as described [here](https://keepachangelog.com/en/0.3.0/)
* [ ] update/close the issue


To release a version:

"Provider":

* [ ] tag the release candidate from  "development" (e.g. 2.1.0-rc1) (ref. www.semver.org for versioning guidelines)
* [ ] validate the release
* [ ] implement bugfixes on the release branch as needed (and port them to development as well)
* [ ] update CHANGES.md to specify the version and release date. Ref. https://keepachangelog.com/en/0.3.0/
* [ ] once the release is validated, create a merge request from the release branch to "master"

Repository maintainer

* [ ] tag the release


## Coding style

### Python

#### Black
The Data Tailor Python code should conform to [Black](https://pypi.org/project/black) code style as
described [here](https://github.com/psf/black/blob/master/docs/the_black_code_style.md) with the
customisation described in the [pyproject.toml](pyproject.toml) file.

To avoid code-style degradation it is recommended to use [pre-commit](https://pre-commit.com/)
(the configuration is provided in the [.pre-commit-config.yaml](.pre-commit-config.yaml) file).

#### Guideline for functional/validation tests
Please, follow these guidelines for writing functional or validation tests:

- add module documentation in the test python file describing the aim of test, mentioning the
  corresponding issue(s) (if any) and any other useful details
- after the import section, use global variables to specify the path and name of the necessary input
  products and also the customisation chain. If many sub-tests are present in the file and if
  they use different chain/input-products, separate with a white-line the groups of variables
  used in each sub-test
- in each sub-test function add documentation about the aim of sub-test, mentioning any details
  useful to understand the goal and the structure of the sub-test
- separate clearly the internal steps of a test function, e.g using a white-line
- add comment lines in the test function code to improve the understanding of the test structure
- if a test needs a dedicated DT configuration and/or an user environment, please use the
  utilities made available by the `epct.setup_test_env` module
- if there is no specific interest in using `api.customisations` (e.g. to test the use of `dask`
  or any other mechanisms connected to the submission of a customisation) to run a customisation,
  please use `api.run_chain` function

The following are general advices to reduce the execution time of a functional/validation test:

- customisations of small input products are faster than customisations of bigger products. For
  example, it could be a good idea to use an EPS PDU product instead of a full-orbit product
- the use of a Region of Interest (ROI) and/or a filter in a customisation-chain speeds up the
  processing. For example, the following customisation:
  ```yaml
  product: HRSEVIRI
  format: geotiff
  filter: hrseviri_natural_color
  roi: western_europe
  ```

  is far faster than:
  ```yaml
  product: HRSEVIRI
  format: geotiff
  ```
- in general, the conversion to the NetCDF4 format is slower than the conversion to GeoTIFF format,
  so, prefer in your chain the `geotiff` output format to the `netcdf4` format

The following DT functional tests can be used as reference:

- [functional_tests/test_FT_output_sub_dirs.py](./functional_tests/test_FT_output_sub_dirs.py) 
- [functional_tests/test_FT_user_quota.py](./functional_tests/test_FT_user_quota.py), especially
  for tests about DT WebService features


#### Additional formatting guidelines
In addition to the standards outlined above, the following guidelines are encouraged:

- f-string syntax instead of %-formatting

### Javascript

See the dedicated [CONTRIBUTING.md](epct-webui/CONTRIBUTING.md) file.