update content (WIP)

README.md

@@ -31,6 +31,41 @@ AIFS produces highly skilled forecasts for upper-air variables, surface weather
 tropical cyclone tracks. AIFS-single is run four times daily alongside ECMWF’s physics-based NWP model and forecasts
 are available to the public under ECMWF’s open data policy. (https://www.ecmwf.int/en/forecasts/datasets/open-data)
 
+## Data Details
+
+### Data parameteres
+
+#### New parameters
+
+More detailed information about the new parameters introduced with AIFS Single v1.0 is provided in the table below.
+
+| Short Name | Name | Units | Component Type | Lev.Type |
+|:----------:|:----:|:-----:|:--------------:|:--------:|
+| swl1 | Volumetric soil water layer 1 | $m^3 m^{-3}$ | HRES | sfc |
+| swl2 | Volumetric soil water layer 2 | $m^3 m^{-3}$ | HRES | sfc |
+| stl1 | Soil temperature level 1 | $K$ | HRES | sfc |
+| stl2 | Soil temperature level 2 | $K$ | HRES | sfc |
+| 100u | 100 metre U wind component | $m s^{-1}$ | HRES | sfc |
+| 100v | 100 metre V wind component | $m s^{-1}$ | HRES | sfc |
+| ssrd | Surface short-wave (solar) radiation downwards | $J m^{-2}$ | HRES | sfc |
+| strd | Surface long-wave (thermal) radiation downwards | $J m^{-2}$ | HRES | sfc |
+| tcc  | Total cloud cover | $(0 - 1)$ | HRES | sfc |
+| hcc  | High Cloud Cover  | $(0 - 1)$ | HRES | sfc |
+| mcc  | Medium Cloud Cover | $(0 - 1)$ | HRES | sfc |
+| lc   | Low cloud cover | $(0 - 1)$ | HRES | sfc |
+| ro   | Runoff | $m$ | HRES | sfc |
+| sf   | Snowfall | $m$ of water equivalent | HRES | sfc |
+
+
+#### Changes to existing parameters 
+
+There are no changes to existing parameters already introduced with AIFS Single v0.2.1.
+
+#### Discontinued parameters
+
+No parameters have been discontinued with regards to the previous version of AIFS Single v0.2.1.
+
+
 ## Model Details
 
 ### Model Description
@@ -54,13 +89,22 @@ and direct observational data.
 - **License:** These model weights are published under a Creative Commons Attribution 4.0 International (CC BY 4.0).
 To view a copy of this licence, visit https://creativecommons.org/licenses/by/4.0/
 
+### Model resolution
+
+There are no changes in resolution compared to previous version AIFS Single v0.2.1.
+
+| | Component | Horizontal Resolution [kms] | Vertical Resolution [levels] |
+|---|:---:|:---:|:---:|
+| Atmosphere | AIFS-single v1.0 | ~ 36 |  13 |
+
+
 ### Model Sources
 
 <!-- Provide the basic links for the model. -->
 
-
-- **Repository:** [Anemoi](https://anemoi-docs.readthedocs.io/en/latest/index.html)
-   is an open-source framework for creating machine learning (ML) weather forecasting systems, which ECMWF and a range of national meteorological services across Europe have co-developed.
+- **Repository:** [Anemoi](https://anemoi-docs.readthedocs.io/en/latest/index.html) is an open-source framework for
+  creating machine learning (ML) weather forecasting systems, which ECMWF and a range of national meteorological 
+  services across Europe have co-developed.
 - **Paper:** https://arxiv.org/pdf/2406.01465
 
 ## How to Get Started with the Model
@@ -87,7 +131,7 @@ step-by-step workflow is specified to run the AIFS using the HuggingFace model:
 🚨  **Note** we train AIFS using `flash_attention` (https://github.com/Dao-AILab/flash-attention).
 The use of 'Flash Attention' package also imposes certain requirements in terms of software and hardware. Those can be found under #Installation and Features in https://github.com/Dao-AILab/flash-attention
 
-🚨 **Note** the `aifs_single_v0.2.1.ckpt` checkpoint just contains the model’s weights. 
+🚨 **Note** the `aifs_single_v1.0.ckpt` checkpoint just contains the model’s weights. 
 That file does not contain any information about the optimizer states, lr-scheduler states, etc.
 
 
@@ -110,6 +154,8 @@ The full list of input and output fields is shown below:
 |-------------------------------------------------------------------------------------------------------------------------------------------------------------|------------------------------------------------------------------------------|--------------|
 | Geopotential, horizontal and vertical wind components, specific humidity, temperature                                                                       | Pressure level: 50,100, 150, 200, 250,300, 400, 500, 600,700, 850, 925, 1000 | Both         |
 | Surface pressure, mean sea-level pressure, skin temperature, 2 m temperature, 2 m dewpoint temperature, 10 m horizontal wind components, total column water | Surface                                                                      | Both         |
+| Soil moisture and soil temperature (layers 1 & 2) | Surface | Both |
+| 100m horizontal wind components, solar radiation (Surface short-wave (solar) radiation downwards and Surface long-wave (thermal) radiation downwards), cloud variables (tcc, hcc, mcc, lcc), runoff and snow fall  | Surface | Output | 
 | Total precipitation, convective precipitation                                                                                                               | Surface                                                                      | Output       |
 | Land-sea mask, orography, standard deviation of sub-grid orography, slope of sub-scale orography, insolation, latitude/longitude, time of day/day of year   | Surface                                                                      | Input        |
 
@@ -118,18 +164,29 @@ the forcing variables, like orography, are min-max normalised.
 
 ### Training Procedure
 
+Based on the different experiments we have made - the final training recipe for AIFS Single v1.0 has deviated slightly
+from the one used for AIFS Single v0.2.1 since we found that we could get a well trained model by skipping the ERA5
+rollout and directly doing the rollout on the operational-analysis (extended) dataset. When we say 'extended' we refer 
+to the fact that for AIFS Single v0.2.1 we used just operational-analysis data from 2019 to 2021, while in this new 
+release we have done the fine-tunning from 2016 to 2022. 
+
+The other important change in the fine-tuning stage is that for AIFS Single v0.2.1 after the 6hr model training the
+optimiser was not restarted (ie. rollout was done with the minimal lr of $3 × 10^{-7}$). For this release we have seen
+that restarting the optimiser for the rollout improves the model's performance. For the operational-fine tuning rollout
+stage, the learning rate cycle is restarted, gradually decreasing to the minimum value at the end of rollout.
+
 <!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
 
-- **Pre-training**: It was performed on ERA5 for the years 1979 to 2020 with a cosine learning rate (LR) schedule and a total
-of 260,000 steps. The LR is increased from 0 to \\(10^{-4}\\) during the first 1000 steps, then it is annealed to a minimum
-of \\(3 × 10^{-7}\\).
-- **Fine-tuning I**: The pre-training is then followed by rollout on ERA5 for the years 1979 to 2018, this time with a LR
-of \\(6 × 10^{-7}\\). As in [Lam et al. [2023]](doi: 10.21957/slk503fs2i) we increase the
-rollout every 1000 training steps up to a maximum of 72 h (12 auto-regressive steps).
-- **Fine-tuning II**: Finally, to further improve forecast performance, we fine-tune the model on operational real-time IFS NWP
-analyses. This is done via another round of rollout training, this time using IFS operational analysis data
-from 2019 and 2020
+- **Pre-training**: It was performed on ERA5 for the years 1979 to 2022 with a cosine learning rate (LR) schedule and a
+total of 260,000 steps. The LR is increased from 0 to $10^{-4}$ during the first 1000 steps, then it is annealed to a
+minimum of $3 × 10^{-7}$. The local learning rate used for this stage is $3.125 × 10^{-5}$.
+- **Fine-tuning**: The pre-training is then followed by rollout on operational real-time IFS NWP analyses for the years
+2016 to 2022, this time with a local learning rate of $8 × 10^{−7}$, which is decreased to $3 × 10^{−7}$. Rollout steps
+increase per epoch. In this second stage the warm up period of the optimiser is 100 steps to account for shorter length
+of this stage. Optimizer step are equal to 7900 ( 12 epoch with ~630 steps per epoch).
 
+As in the previous version of aifs-single for fine-tuning and initialisation of the model during inference, IFS fields
+are interpolated from their native O1280 resolution (approximately $0.1°$) down to N320 (approximately $0.25°$). 
 
 #### Training Hyperparameters
 
@@ -164,8 +221,16 @@ and windspeed, and SYNOP observations of 2 m temperature, 10 m wind and 24 h tot
 of the metrics, such as ACC (ccaf), RMSE (rmsef) and forecast activity (standard deviation of forecast anomaly,
 sdaf) can be found in e.g Ben Bouallegue et al. ` [2024].
 
+### AIFS Single v1.0 vs AIFS Single v0.2.1 (2023)
+
+<div style="display: flex; justify-content: center;">
+  <img src="assets/scorecard_single1.0_vs_single0.2.1_2023.png" alt="Scorecard comparing forecast scores of AIFS versus IFS (2022)" style="width: 80%;"/>
+</div>
+
+### AIFS Single v1.0 vs IFS (2024)
+
 <div style="display: flex; justify-content: center;">
-  <img src="assets/aifs_v021_scorecard.png" alt="Scorecard comparing forecast scores of AIFS versus IFS (2022)" style="width: 80%;"/>
+  <img src="assets/scorecard_single1.0_vs_ifs_2024.png" alt="Scorecard comparing forecast scores of AIFS versus IFS (2022)" style="width: 80%;"/>
 </div>