CONTRIBUTING.md 4.92 KB
Newer Older
1
# CONTRIBUTING TO THE DATA TAILOR DEVELOPMENT
2

3
The following is a set of guidelines for contributing to the Data Tailor.
4
5


6
## Branching and release workflow
7
8
9
10
The branching and release workflow is very close to GitFlow.

The following main branches always exist:

11
* the **main** branch contains released software. Only the repository maintainer can merge to it.
12
13
* the **development** branch contains consolidated software developments.
 
m.bottaccio@bopen.eu's avatar
m.bottaccio@bopen.eu committed
14
When a developer wants to work on a new feature or a major bugfix, he needs to:
15

16
* [ ] create an issue on the project GitLab repository with a short descriptive title and giving it a "weight" (complexity): 0: straightforward, and assigning a milestone
17
* [ ] from the issue GitLab page, create a new branch with "development" as a parent. The branch name will be automatically generated by GitLab from the issue title
18
19
* [ ] when working on a issue, put it in a "Doing" state (issue tag)
* [ ] work on the new feature on the new branch, and commit changes to it, quoting the issue
m.bottaccio@bopen.eu's avatar
m.bottaccio@bopen.eu committed
20
* [ ] once the work is complete, create a merge request to "development", quoting the issue number and resolution
m.bottaccio@bopen.eu's avatar
m.bottaccio@bopen.eu committed
21
* [ ] update the "[unpublished]" section in CHANGES.md to add a one-line description of the closed issues (for humans) as described [here](https://keepachangelog.com/en/0.3.0/)
22
23
* [ ] update/close the issue

m.bottaccio@bopen.eu's avatar
m.bottaccio@bopen.eu committed
24

25
26
To release a version:

m.bottaccio@bopen.eu's avatar
m.bottaccio@bopen.eu committed
27
28
"Provider":

m.bottaccio@bopen.eu's avatar
m.bottaccio@bopen.eu committed
29
* [ ] tag the release candidate from  "development" (e.g. 2.1.0-rc1) (ref. www.semver.org for versioning guidelines)
30
31
32
33
* [ ] validate the release
* [ ] implement bugfixes on the release branch as needed (and port them to development as well)
* [ ] update CHANGES.md to specify the version and release date. Ref. https://keepachangelog.com/en/0.3.0/
* [ ] once the release is validated, create a merge request from the release branch to "master"
m.bottaccio@bopen.eu's avatar
m.bottaccio@bopen.eu committed
34

m.bottaccio@bopen.eu's avatar
m.bottaccio@bopen.eu committed
35
Repository maintainer
m.bottaccio@bopen.eu's avatar
m.bottaccio@bopen.eu committed
36

37
38
39
40
41
42
43
* [ ] tag the release


## Coding style

### Python

44
#### Black
45
The Data Tailor Python code should conform to [Black](https://pypi.org/project/black) code style as
46
described [here](https://github.com/psf/black/blob/master/docs/the_black_code_style.md) with the
47
customisation described in the [pyproject.toml](pyproject.toml) file.
48

49
50
To avoid code-style degradation it is recommended to use [pre-commit](https://pre-commit.com/)
(the configuration is provided in the [.pre-commit-config.yaml](.pre-commit-config.yaml) file).
51

52
53
54
55
56
57
#### Guideline for functional/validation tests
Please, follow these guidelines for writing functional or validation tests:

- add module documentation in the test python file describing the aim of test, mentioning the
  corresponding issue(s) (if any) and any other useful details
- after the import section, use global variables to specify the path and name of the necessary input
58
  products and also the customisation chain. If many sub-tests are present in the file and if
59
60
61
62
63
64
  they use different chain/input-products, separate with a white-line the groups of variables
  used in each sub-test
- in each sub-test function add documentation about the aim of sub-test, mentioning any details
  useful to understand the goal and the structure of the sub-test
- separate clearly the internal steps of a test function, e.g using a white-line
- add comment lines in the test function code to improve the understanding of the test structure
65
- if a test needs a dedicated DT configuration and/or an user environment, please use the
66
67
68
69
70
  utilities made available by the `epct.setup_test_env` module
- if there is no specific interest in using `api.customisations` (e.g. to test the use of `dask`
  or any other mechanisms connected to the submission of a customisation) to run a customisation,
  please use `api.run_chain` function

71
The following are general advices to reduce the execution time of a functional/validation test:
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89

- customisations of small input products are faster than customisations of bigger products. For
  example, it could be a good idea to use an EPS PDU product instead of a full-orbit product
- the use of a Region of Interest (ROI) and/or a filter in a customisation-chain speeds up the
  processing. For example, the following customisation:
  ```yaml
  product: HRSEVIRI
  format: geotiff
  filter: hrseviri_natural_color
  roi: western_europe
  ```

  is far faster than:
  ```yaml
  product: HRSEVIRI
  format: geotiff
  ```
- in general, the conversion to the NetCDF4 format is slower than the conversion to GeoTIFF format,
90
  so, prefer in your chain the `geotiff` output format to the `netcdf4` format
91
92
93

The following DT functional tests can be used as reference:

94
95
96
- [functional_tests/test_FT_output_sub_dirs.py](./functional_tests/test_FT_output_sub_dirs.py) 
- [functional_tests/test_FT_user_quota.py](./functional_tests/test_FT_user_quota.py), especially
  for tests about DT WebService features
97
98


99
100
101
102
103
104
105
#### Additional formatting guidelines
In addition to the standards outlined above, the following guidelines are encouraged:

- f-string syntax instead of %-formatting

### Javascript

106
See the dedicated [CONTRIBUTING.md](epct-webui/CONTRIBUTING.md) file.
107
108