CHANGES.md 26.4 KB
Newer Older
m.bottaccio@bopen.eu's avatar
m.bottaccio@bopen.eu committed
1
2
3
4
5
6
# Changelog
All notable changes to this project are documented in this file.

The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).

m.bottaccio@bopen.eu's avatar
m.bottaccio@bopen.eu committed
7
## [3.0.0]
8

9
10
11
12
13
14
15
### Added
- Support processing GRAS data reprocessed via EPSar tool integration (#1103)
- Enabled support for converting NetCDF4 (Data Centre) to HRSEVIRI L1.5 Image Data HRV channel (#1052)
- Added ROI-extraction feature for EPS-native to EPS-native customisations (#879)
- Show download progress in Web GUI (#656)
- Added content-length header in file download requests lowering memory consumption on downloads (#525, #537)

16
### Changed
Giuseppe Le Voci's avatar
Giuseppe Le Voci committed
17
- Removed old prints from preprocessing and postprocessing modules (#1128)
18
19
20
21
- Moved web service code and documentation to separate repository (#1089)
- Moved plugins to their own repositories (#1004, #903)
- Aligned EPS product IDs with new Data Store nomenclature (#1087)
- Migrate Python version to 3.9 in view of version 3.6 end-of-life (#814, #1063, #1066)
22
- Natively compile PublicDecompWT during msg-gdal-driver windows build (#1007)
23
- Upgraded GDAL version to 3.3.1 (#1007)
Luca Fabbri's avatar
Luca Fabbri committed
24
- Removed `falcon_multipart`, now using native Falcon 3.0 support (#1064)
25
- Cleaned code moving configurable default and constant values and dictionaries out of functions (#483)
26

27
### Fixed
Giuseppe Le Voci's avatar
Giuseppe Le Voci committed
28
- Download via GUI with desktop installation (#1120, #1122)
29
- Fix deletion of customisations with output products saved directly into the user output folder (#1093)
30
- Fix epct-plugin-gis building issue due to new distribution of python-eccodes package (#1083)
31
32
- Customization correctly produces NetCDF4 products from HRSEVIRI HRV when HRV is empty due to darkness (#1053)
- Appropriate error message returned for IASISND02 older than 10-2014 to netcdf4_satellite conversions (#1047)
33

34

35
## [2.15.1]
36
37

### Changed
38
- Fix epct-plugin-gis building issue due to new distribution of python-eccodes package (#1083)
39
40
41
42
43
44
45
46

### Fixed
- Deleting a customisation with no output now does not raise an error (#1081)


## [2.15.0]

### Changed
47
48
49
50
51
52
53
54
- Improved service monitoring through EELK (#1036)
- Windows pipelines ported to shell executor (#1037)
- Improved Docker component logs (#1040)
- Added a configurable option to save results in unique output subdirectories (#1025)
- Http2 now enabled in nginx (#1010)
- Download info are now taken from output subdirectories when presents (#1018)

### Fixed
55
- Quicklook option now does not make the HRV processing fail (#1016)
56
57
58
59
60
- Raising a timeout error for the dtwsadmin user (#1021)
- Increased the allowed maximum number of chars in the URL for downloading multiple files results (#1022)
- Prevent file deletion to break quota computation (#1031)
- Customisations producing fixed output filenames return now consistent results (#1032)
- Artifacts are now not overwritten in deploy stage (#1039)
61
- Prevent docker service logs from growing indefinetely (#1055)
62
63


64
65
66
67
68
69
## [2.14.0]

### Changed
- DTWS uses only not deprecated TLS versions (#987) 

### Fixed
70
- WebUI of desktop installation for Windows OS is now working correctly (#1035)
71
72
73
74
75
- Ensure requests in queue do not stay on inactive status (#1003)  
- Data Tailor installation on Windows OS is now working properly (#994)
- Customisations on HRSEVIRI products in native format do not generate ARC files erroneously (#989)
- GDAL commands are now correctly displayed in the customisation log-file (#984)

76
77
78
79
## [2.13.0]

### Added
- Experimental support added for ASCAT Level 1 SZF, SZO, SZR Climate Data Record Release 2 - Metop products (#980)
m.bottaccio@bopen.eu's avatar
m.bottaccio@bopen.eu committed
80
- Support for MSGCLTH (EO:EUM:DAT:MSG:CTH) product added (#981)
81
82
83
84
85
86
87
88
89
- Specified limitations in support of GRAS, MTP15 products (#982)

### Changed
- Updated tests to download products and retrieve product information from new Data Store API URLs (#1000)
- Improved GUI representation of long lists of items (#975)
- Modified basepath in swagger spec so it does not define version (#990)

### Fixed
- Ensured setting maintenance message does not prevent from loading product type list (#993)
90
- Correct geolocalisation of MSG2-RSS data when native format is geographically re-projected to NetCDF output (#986)
91

92
93
94
95
96
## [2.12.1]

### Changed
- User attempts to delete running customisations issue a Warning (#974)

97
98
99
## [2.12.0]

### Added
100
- Heartbeat mechanism introduced to improve process status monitoring (#912)
101
102

### Changed
103
- Improved error message in the GUI when fair queuing cap is reached (#934)
104
105
- Dependencies installed at install time rather than embedded in epct_plugin_umarf package (#933)
- Dependencies installed at install time rather than embedded in epct_webui package (#932)
106
- Logout text displayed in normal case on GUI (#866)
107
- NetCDF-4 types renamed as "NetCDF4 (Data Centre)" and "NetCDF4 (simplified)" (#904)
108
- Linux WGSI server switched to gunicorn to improve performances (#948)
109
110

### Fixed
111
- UI state display not freezing anymore when killing multiple customisations at once (#927)
112
113
114
- Clean up of delivered zipped files in DTWS done to avoid an unexpected increment of user quota (#937) 
- Cancellation of submitted customisations is now working properly (#926, #940)
- Progress estimation from log-file now reflecting the actual customisation progress status (#919)
115
- Remove potential security issues when submitting basket XML (#945)
116
117
118
119

### Removed
- MTG-IRS products removed from the list of supported products (#840)

Giuseppe Le Voci's avatar
Giuseppe Le Voci committed
120
121
122
## [2.11.0]

### Added
Giuseppe Le Voci's avatar
Giuseppe Le Voci committed
123
- Implemented xRIT channel segment selection (#897)
Giuseppe Le Voci's avatar
Giuseppe Le Voci committed
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
- Allow for GEO products the ROI extraction if the input data specifies or known to be in a known projection (#885)
- Re-projection by chunks for ASCATL1SZO and ASCATL1SZR full orbit products (#825)
- Implementation of new KPI, documenting step-by-step operational procedures (#738, #889)

### Changed
- Optimised pipeline jobs duration and structure (#900)
- Improve error information shown to users (#883)
- Improvements in the way static resources are served (#882)
- Improved DTWS nginx logging with response time (#881)
- Improved REST API logging: more details, traceback and customisation submission tracking (#880)
- Updated product discovery after Data Store API change (#847)
- ROI customisations as internal metric for downstream analytics (#838) 
- Data Tailor version always discoverable (#836)

### Fixed
- Webapp not displaying a proper error when a download operation fails (#873)

141
142
143
144
145
146
147
## [2.10.0]

### Added
 - Add UMARF plugins to installer constructor and pipelines (#823)  
 - Provided descriptions for OpenAPI specs definitions (#808)
 - Deploy native to HRIT conda package to EUMETSAT conda repository (#803)
 - Added capability to register Custom Plugins with pluggable configuration, products and formats (#802)
148
 - Added support for multiple products and formats, for integration EUMETSAT Prototype Satellite Data Cube plugin (#796)  
149
 - Introduced mechanism signaling any ongoing or planned maintenance issues to GUI users (#771)
150
 - HRV to netCDF conversion: Providing calibration data (#442)
Giuseppe Le Voci's avatar
Giuseppe Le Voci committed
151

152
### Changed
153
 - Adapt DTWS dockerfile to execute commands as user (#853)
154
 - Temporarily remove support for MTG IRS L1 as format is being updated (#840)
155
 - Removing also the processing-dir (if any) when a customisation is deleted by user or administrator (#837)
156
157
158
159
160
 - Improved propagation of Data Store API URL within code (#781)
 - Optimised loading of status pane in GUI (#751)
 - Switch DTWS logout button to logout link redirection (#668)
 - Projection and ROI enhancements from GUI (#597)
 - Show feedback for uploading shapefile from GUI (#596)
161
 - Input products downloaded from the Data Store are now automatically removed as the customisation ends (#775)
162

Giuseppe Le Voci's avatar
Giuseppe Le Voci committed
163
164
165
### Removed
 - Removed check_olda_cache option as Data Store input products are now downloaded in temporary processing_dir (#775)

166
### Fixed
167
168
169
 - Properly parse content disposition as products downloaded via GUI on chrome contain start and trail underscore (#869)
 - Ensure netcdf-satellite output files can correctly be downloaded from GUI via DTWS (#865)
 - Fix 50-x service pages configuration (#854)
170
 - Conversion of full-orbit AVHRRL1 to EPS-native fixed in DTWS as ROI tab now enabled with roi_by_time (#832)
171
 - Sanitise inputs casting ROI boundary values to float before saving them in processing_info dictionary (#798)
172
 - Native HRV to HRIT now works correctly (#794)
173
 - Native to netCDF did not preserve metadata correctly (#793)
174
 - Ensure previously active fair queuing user cap mechanism is not applied anymore when turned off (#782)
175
 - HRSEVIRI 1.5 data - ToolsUI, Panoply, & IDV could not find coordinates after converting to netCDF (#759)
176
177
178
179
180
 - Sanitised configuration reading so that temporary configuration files are not copied to user configuration (#748)
 - Fix handling of empty uuids set within Delete customisation requests (#727)
 - Avoid truncated information in GUI Log Pane (#709)
 - Bug fix: ASCATL1SZ0 and ASCAL1SZR longitude coordinates converted to +/-180 degrees to be read by GDAL (#587) 
 - ROI by sensing time not available if feature does not contain roi_by_time (#560)
181

182
183
184
185
186
## [2.9.1]

### Changed
 - Input products downloaded from the Data Store are now automatically removed as the customisation ends (#775)

187

188
189
190
## [2.9.0]

### Added
191
 - Add instructions to launch selected pipeline jobs (#739)
192
193
194
 - Add HRSEVIRI archive product type (#730)
 - Requirement, configuration and test procedure for GEMS monitoring (#680, #690, #718)
 - Test and document customisation timeout functionality (#666, #674)
m.bottaccio@bopen.eu's avatar
m.bottaccio@bopen.eu committed
195
196
 - Add functional tests pipeline job (#660)
 - Add for logs produced for ELK (#640)
197
198
199
200
201
202
203
204
205
206
207
208
209
 - Allow manual deployment to EUMETSAT conda channel and use it in installs (#558)
 
### Changed
 - Reformat creation of dask client (#732)
 - Improve api.ensure_config to avoid rereading configuration when API function calls internally another function (#729)
 - For the operational phase, Docker running as a non-priviledged user (#712)
 - Updated launching scheduler service with Data Tailor plugin (#677)
 - Moved support functions from api.py to a new dedicated python module (#664)
 - Automatized error test EPCT.ERR.TP.01.07 (#595)
 - Long-queued customisations fail when token expires (#549)
 - Split epct_restapi/__init__.py in separate python files to improve code readability and efficiency (#467)

### Fixed
210
 - Timeout scheduler leaking memory (#731)
211
212
213
214
215
216
217
218
219
220
 - Xrit DT plugin missing output files (#726)
 - Add test data fom rect2lpToOpenMTP plugin (#723)
 - UMARF plugin missing environment variable (#722)
 - UMARF plugins tests (#696)
 - Capturing exit status in functional tests Windows pipeline job (#682)
 - Fixed and improved tests for manual killing process (#675)
 - Clean scheduler plugin swap variables (#672)
 - report_quota path fix (#636)
 - ELK stack fixes and testing (#635)
 - Fix bug as some IASISND02 products generated an error during the processing (#616)
221
 - Improved validity check fixing error arising when basket config address used as input product path (#456)
222

Giuseppe Le Voci's avatar
Giuseppe Le Voci committed
223
## [2.8.1]
Giuseppe Le Voci's avatar
Giuseppe Le Voci committed
224

m.bottaccio@bopen.eu's avatar
m.bottaccio@bopen.eu committed
225
### Added
Giuseppe Le Voci's avatar
Giuseppe Le Voci committed
226
 - Added configurable parameter about EUMETSAT Data Store URL netloc (#700)
227
228

### Fixed
229
 - Preventing re-projecting geostationary products to geostationary projections (#688, #684)
Giuseppe Le Voci's avatar
Giuseppe Le Voci committed
230
231
 - Update to outdated deployment instructions for DTWS (#683)
 - Logging quota overflows, despite logs contribute to user quota (#642)
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
 - Report quota path fix (#636)

## [2.8.0]

### Added
 - CI/CD: installation of EUMETSAT package eugene on linux pipelines to not skip relative EPS validation tests (#667)
 - Added customization time info to log for ELK reports (#650)
 - Basket check functional test (#634)
 - Added epct_webui code coverage (#622)
 - Added the possibility to specify the path to general configuration folder via environmental variable (#606)
 - DTWS: use an explicit whitelist with administrator usernames (#593)
 - Automatic build and test of conda constructor package on Linux (#590, #591)
 - Added nginx caching of JS, CSS to DTWS (#571)
 - Integration with GEMS monitoring (#547)
 - Enforced a configurable customisation time-out (#545)
247
 - Allowing users to housekeep their space (#544)
248
249
250
 - Installation without internet on Linux (#540)
 - DTWS: enforced a configurable timeout on long-running jobs (#465)
 - DTWS: implemented the ability to monitor and control working nodes (#431)
251
252
 
### Changed
253
254
 - Optimized execution of long tests in CI pipelines (#665)
 - Making the dask dashboard of the DTWS accessible and informative (#659)
255
 - Applied to Python code the [Black](https://pypi.org/project/black/) coding style and f-string syntax (#654)
256
257
258
 - Restructured DTWS fair queuing logic and handling of exceptions (#632, #633)
 - Rename "output_dir" variable "root_path", remove ref to "test" deployment in epct-restapi/__init__.py (#607, #621)
 - Webapp GUI changes for admin user to manage customisations (#600)
259
 - Move scipy requirement from epct/setup.py to epct_plugin_gis/setup.py (#586)
260
261
262
263
264
265
266
267
268
269
270
271
 - OAS* and OR1* products functional tests are now included in validation procedures and testing (#535)
 - Unit tests coverage for epct-plugin-gis expanded up to 60% (#401, #403, #404, #405, #406, #557)
 
### Fixed
 - Active user detection (#655)
 - ELK reports only showing a selection of info and detailed list of customisation reporting id (#646, #648)
 - Deleting a process not working if generic username used in deployment style desktop (#641)
 - Reliable kill processes in DTWS (#639)
 - Restructure logic of api.report_quota for administrator user (#638)
 
### Removed
 - Removed filter feature from UMARF msgclmk backends definition (#601)
272
273
274
275
276

## [2.7.0]

### Added
 - Automatic build of DT conda constructor package with CI pipelines on Linux (#590, #591)
277
 - New manual job in CI allows creating pdfs for all sphinx documents (#574)
278
279
 - First automated tests for GUI validation (#567)
 - API and REST API report on usage of user quota (#565)
280
 - DTWS: User quota activable and configurable (#564)
Giuseppe Le Voci's avatar
Giuseppe Le Voci committed
281
282
283
 - Added a configurable limit to the max shapefile size allowed to upload (#542) 
 - Monitoring concurrent users on system (#536)
 - Monitoring total customisation time per user (#528)
284
 - Setup a fair-shared queuing (#428)
Giuseppe Le Voci's avatar
Giuseppe Le Voci committed
285
286
 
### Changed
287
 - User quota configuration reviewed and improved (#589)
288
289
290
291
292
 - EDT: inhibit new customisations if a user has already used all his quota (#580)
 - Updated DTWS user info URL (#570)
 - Disabled input product selection when in "epcs" deployment (#552)
 - Input product type(s) automatically discovered using Data Store REST API (#546)
 - Allowed user configuration of compression wrappers (#541)
293
294
295
296
297
298
299
300
301
 - Improvements to ELK stack and reports (#538)
 - Automatized multiple manual tests (#532)
 
### Fixed
 - Removed advertising of unavailable features for ASCATL1SZF product (#583)
 - Configuration of user info in DTWS deployment was lacking (#570)
 
### Removed
 - DT configuration in Docker container now stored in conda environment as well (#599)
Giuseppe Le Voci's avatar
Giuseppe Le Voci committed
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316


## [2.6.0]

### Changed
 - Harmonising GUI with other Big Data Services (background, top bar, white flat logo, favicon, font) (#563)
 - Improved performance, especially for the GUI (#556)
 - Improved display of input product types (#554)
 - GUI: better use of screen width (#553)
 - Improved the display of the basket content (#551)
 - Clickable EUMETSAT logo, redirecting to EUMETSAT webpage (#550)
 - GUI: showing username after the login (#548)

### Fixed
 - Allow any feature relying on log parsing to fail gracefully if the format is unexpected (#477)
317
 - Fix false FAILED status due to logging of GDAL errors about warping tolerance tests (#573)
Giuseppe Le Voci's avatar
Giuseppe Le Voci committed
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
 
### Removed
 - Removed output folder selection in service (#555)


## [2.5.0]
### Added
 - Added X-Frame-Options header from epcs.eumetsat.int (#526)
 - Added specific CI/CD pipeline job for detailed validation tests (#512)
 - GUI, REST API: Use query string parameter to pass duplicate token for customisation POSTs (#472)
 - Prepared test specification with test cases and procedures document for DTWS (#471)
 - Added Test Plans for Data Tailor Web Service (DTWS) and Data Tailor standalone (#469)
 - Enforce a configurable cap on the number of running/queued jobs submitted by a user (#466)
 - Add conditional layer filtering feature for Global L3C AVHRR GHRSST (#418)
 - CD/CI: Available for Windows. Dockerfile included into repository (#409)
 - Add MSGCLMKtoNetCDF UMARF Java plugin compilation and backend (#394)
 - Adding coverage step to pipelines (#390)
 - Add RECT2LPtoOpenMTP UMARF plugin compilation and backend (#387)
 - Add compilation of UMARF plugins to relevant Conda recipe (#387)
 - Now support input SIP when retrieving products from the Data Store (#385)
 - Add check about ROI-extraction without re-projection (#381)
 - Conda packages built now as part of CI/CD pipeline, save as artefacts and/or pushed to a target machine (#379)
 - Added technical note on Data Tailor integration with EUMETCAST (#369)
 - CI/CD: add the test step (#365)
 - Add check-inputs and aggregation-analysis entry-points to backends (#145)
 - Add automatic validation tests for CLI, API (#359)
344
 - Webapp can now be used without any network access
Giuseppe Le Voci's avatar
Giuseppe Le Voci committed
345
 - Add ability to start Data Tailor in server mode without launching web UI (#370)
346

m.bottaccio@bopen.eu's avatar
m.bottaccio@bopen.eu committed
347
### Changed
Giuseppe Le Voci's avatar
Giuseppe Le Voci committed
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
 - Skipping tests MSG_07_02, MSG_07_03 and MSG_08_01 on Windows (#512)
 - Updated readme (#495)
 - Updated Change Records section of the VVTCTP document after Issues 445 and 151 (#493)
 - Updated DTWS ICD (#491)
 - Updated validation test-chains after changes in quick-look algorithm to make the tests functional again (#489)
 - Uniformed download log format to general one (#464)
 - Rationalise code about the quick-look creation and improve quick-look support for HRSEVIRI-HRIT products (#151)
 - UMARF plugins build procedures now include install and uninstall rules (#388)
 - Reactivated ability to retrieve products from the Data Store (#385)
 - Unit tests coverage expanded up to 60% for chain_runner, log_reader, download and plugin epct modules (#384)
 - Unit tests coverage expanded up to 60% for pre-processing, post-processing and cli epct modules (#383)
 - Move user configuration in the conda environment (#343)
 - Update and expand unit tests for epct.api and epct.etc (#378)
 - Use canonical names from Product Navigator to identify products (#371)
 - Move test data into own repository (#360)
 - Mark non-regression tests (#366)
 - Improve resampling of the native longitude coordinates for AVHRRL1 (#253)
 - Use the PublicDecompWT library directly from Git (#363)
 - Automate build process (#361)
 - GUI offline mode (#348)
 - Refine strategy to define ETC_DIR, WORKSPACE_DIR, LOG_DIR (#334)
m.bottaccio@bopen.eu's avatar
m.bottaccio@bopen.eu committed
369
370

### Fixed
Giuseppe Le Voci's avatar
Giuseppe Le Voci committed
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
 - Fixed EPCT.EPS.02.02 test with output quick-look not the same depending on the OS, Windows or Linux (#527)
 - AVHRR red quick-look (#523)
 - Fixing long logs not opening on the last line (#522)
 - Fix process window not open when the panel was opened, but only after the first refresh (#521)
 - Fixed process bar sometimes not at 100% even if the job is completed (#520)
 - Fixed QUEUED status for jobs submitted but not running (#519)
 - Fixing jobs that exceed the current worker capacity being displayed as RUNNING but never executed (#518)
 - GUI: Fixed Layer filtering selection not active on Chrome browser (#517)
 - Fixed tests EPS_11_02 and EPS_11_01 on Windows (#512)
 - Fixed flaky test_SAF_TP_01_04_GLBSST_GRIB2.py (#512)
 - Functional issues noted during OSAT for resolution (#510)
 - Documentation issues noted during OSAT for resolution (#506)
 - Miscellaneous fixes to tests and asserts, in particular fixing quick-look chains on Windows (#504)
 - Fix ROI behaviour when using sensing time (#500)
 - Fix username in JWT assertion missing if the user used a client-key for the authentication (#488)
 - Sanitise logs in REST API to remove abolute paths (#487)
 - Fix products being downloaded as download_path=<name> instead of <name> (#485)
 - Fix GUI output directory so it is selected by default (#485)
 - Fix MSGCLMK test showing error log with invalid "space view perspective ortographic" section in GRIB file (#485)
 - Fix Input products written as INFO in logs (#485)
 - IP of the worker now correctly logged with level "DEBUG" instead of "INFO" (#482)
 - Bug in reading customisation log-files when the username is a string with "_" character (#481)
 - Enforced correct kwargs for req.get_param in Rest API (#463)
 - Fix nodata-color errors in quick-look creation (#413)
 - Fix the bands order errors in quick-look creation (#412)
 - Fix band-filtering for HRSEVIRI-HRIT product (#373)
 - FIX ROI-extraction whit northern polar stereographic projection (#367)
 - Fixes to VVP (#335)
 - Fix unpacking of stretch method during pre-processing (#356)
400
401
 - Do not show a "null bytes" string for current directory in filebrowser

Giuseppe Le Voci's avatar
Giuseppe Le Voci committed
402

403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
## [2.4.0]
### Added
 - It is now possible to specify a ROI through a shapefile
 - Results of a customisation can be retrieved as `xarray` datastructures (API only)
 - Support for SIP as an input and output format
 - Support for compression and decompression using standard compression formats (zip,
   tar.gz, tar.bz2)
 - Support for compression using internal compression formats (HDF5 and NetCDF-4 outputs only)
 - EPCT: GUI and REST API on desktop now can be launched with a single command
 - MSG products: support for MSG HRSEVIRI L1.5 - HRV band
 - MSG products: support for MSG MSGCLMK in GRIB format
 - MSG products: support for MSGAMVE
 - MSG products: support for MSGCLAP
 - MSG products: support for MSGFIRG
 - MSG products: support for MSGMPEG
 - MSG products: support for MSGOCAE
 - EPS products: support for Native-to-Native conversion
 - EPS products: support for GOME2 L1B products
 - EPS products: support for IASISND02 products
 - SAF products: support for GLBSST in GRIB format
423
 - SAF products: support for OAS025 in NetCDF3 and BUFR formats
424
425
 - SAF products: support for OR1ASWC12 in NetCDF3 format
 - SAF products: support for OR1ASW025 in NetCDF3 format
426
427
 - SAF products: support for OR1ERW025 in NetCDF3 and BUFR format
 - SAF products: support for OR1SWW025 in NetCDF3 and BUFR formats
428
429
430
431
432
433
434
435
436
437
438
 - MTG products: restore support for conversion of MTG IRS L1 (Principal Component Score)
   to BUFR
 - REST API: return the paths to output products for completed customisations
 - EPCS: support for OLDA Basket metalink as a way to specify input products in OLDA
 - EPCS: support for user login and SSO
 - EPCS: support for user workspaces
 - EPCS: added distributed architecture, allowing resource elasticity
 - Improved logging, now using a uniform template
 - New netcdf-generator plugin, allowing to use external UMARF software to generate NetCDF4 from EPS

### Changed
439
 - Reaming of the EPCT to EUMETSAT Data Tailor
440
441
442
443
444
445
446
447
448
 - Queueing mechanism, now based on `dask distributed`
 - Improved management of CORS for REST API (especially for desktop installations)
 - EPCS: Only the access token is now used to pass authentication info

### Removed
 - Submitted customisations cannot be paused and resumed anymore.

### Fixed

m.bottaccio@bopen.eu's avatar
m.bottaccio@bopen.eu committed
449

m.bottaccio@bopen.eu's avatar
m.bottaccio@bopen.eu committed
450
## [2.3.0]
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
### Added
 - Customisation framework allows to use customisation plugins without changes to the code
 - New REST-based Web Service Interface
 - GUI is now a web application with improved design
 - Flexible deployment approach allows to select plugins be install
 - Products: added support for EPS ASCATL2SMR
 - Products: added support for EPS ASCATL2SMO 
 - API: added support for asynchronous chain execution
 - API/CLI: added interface to query EPCT configuration and settings
 - Plugins: "epct_plugin_umarf" allows to use the msg15toISCCP plugin from UMARF
   (enables format conversion of MSG HRSEVIRI L1.5 products to ISCCP formats)
 - Plugins: new "epct_plugin_umarf" allows to use the msg15toXrit executable from UMARF
 - Plugins: "epct_plugin_fist_iasil1c" allows to use the IASIL1c_v9A plugin from FIST
   (enables conversion of EPS IASI L1C products to  BUFR).
 - EPCS prototype, with support for authenticated retrieval of information through the
   EUMETSAT API gateway.
 - Automated validation tests for format conversion.

### Changed
 - Restored support for HRSEVIRI_HRIT, HRSEVIRI_HRIT_HRV
 - API, CLI: Entry points removed (choices, load)
 - API, CLI: "chain_run" entry point deprecated in favour of "run_chain".
 - CLI: specification of the product for `run_chain` now only possible in the chain configuration.
 - Architecture: new EPS importer (in the "epct_plugin_gis" plugin)
 - Architecture: updated MSG importer.
 - GDAL dependency completely removed from EPCT core components
 - GDAL-based customisation engine is now a plugin
 - Improved configurability (working directory, base path).
 - Simplified deployment; now all packages have their own conda-based building system.
 
### Removed
 - Conditional layer filtering for GLBSST no more supported
 - ASCII format no more supported.
 - JAVA GUI application no more available
 - Support for deployment on Docker containers no longer available.

### Fixed


m.bottaccio@bopen.eu's avatar
m.bottaccio@bopen.eu committed
490
491
492
493
## [2.2.0-rc6] - 2019-07-10
### Changed
 - Temporarily disable the support for HRSEVIRI_HRIT, HRSEVIRI_HRIT_HRV due to licensing issues
 - Properly attribute licenses (in particular to Icons)
494

m.bottaccio@bopen.eu's avatar
m.bottaccio@bopen.eu committed
495
496
497
498
## [2.2.0-rc4] - 2019-06-06
### Fixed
 - Windows: fix the selection of the Python library by GDAL

Alessandro Amici's avatar
Alessandro Amici committed
499
## [2.2.0-rc3] - 2019-05-21
500
501
502
### Changed
 - Apply Apache v2.0 license to source files

Alessandro Amici's avatar
Alessandro Amici committed
503
504
505
506
## [2.2.0-rc2] - 2019-05-09
### Fixed
 - Use safe YAML parsing

507
## [2.2.0-rc1] - 2019-05-03
m.bottaccio@bopen.eu's avatar
m.bottaccio@bopen.eu committed
508
### Added
509
510
 - Conda installer (windows/linux)
 - Dockerfile to build an epct image
511
 - EPCT user guide
512
513
514
515

### Changed
 - Improved documentation
 - Removed Makefile-based installers
516
 - Improved test data
m.bottaccio@bopen.eu's avatar
m.bottaccio@bopen.eu committed
517

518
519
520
### Fixed
 - MTG IRS BUFR generation now operates on PC file alone (as intended)

m.bottaccio@bopen.eu's avatar
m.bottaccio@bopen.eu committed
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
## [2.1.0] - 2019-02-20
### Added
 - Support for MLST (MSG Land Surface Temperature)
 - Support for MET (MSG Evapotranspiration)
 - Support for METREF (MSG Reference Evapotranspiration)
 - Support for MDfAPAR (MSG Fraction of Adsorbed Photosynthetic Active Radiation)
 - Added the geostationary projection centred on (0,0)
 - GUI: Added tooltip for input and output files
 - Contribution guide (CONTRIBUTING.md)
 - Changelog (CHANGES.md)

### Changed
 - GUI: Improved performance in configuration loading
 - API: process management moved to API instead of the CLI
 - entry points for "save" and "delete" CLI moved from etc to api
 - Improved performance for reprojection of large products

### Fixed
 - Installation: minimum version for 'cfgrib' imposed (fixes crash on BUFR conversion)
540
 - Installation: added dependencies for CentOS installation
m.bottaccio@bopen.eu's avatar
m.bottaccio@bopen.eu committed
541
542
543
544
545
546
547
548
549
 - Fixed crash when naming convention referring to non-existing field
 - Fix erroneously duplicated bands for GOME_PMA_AOP
 - Use standard BUFR missing values instead of dummy ones
 - GUI: allow multiple file selection using SHIFT
 - Add check about existence of the input file(s)
 - Fix a failure in quicklook generation if the quicklook image does not have an associated geotransform/projection
 - Do not require a layer filter for quicklook of MSG HRIT
 - Avoid Key Error if configuration section is missing
 - Fix issues with some NetCDF4 on Windows