Carlos Gomes commited on
Commit
3cd7cc3
1 Parent(s): a82c831

add model card

Browse files
Files changed (1) hide show
  1. README.md +109 -0
README.md CHANGED
@@ -1,3 +1,112 @@
1
  ---
2
  license: apache-2.0
3
  ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
  license: apache-2.0
3
  ---
4
+
5
+ # Model Card for granite-geospatial-biomass
6
+
7
+ <p align="center" width="100%">
8
+ <img src="https://github.com/ibm-granite/granite-geospatial-biomass/blob/main/biomass-image.jpeg?raw=true" width="600">
9
+ </p>
10
+
11
+ The granite-geospatial-biomass model is a fine-tuned geospatial foundation model for predicting the total above ground biomass (i.e., living and dead plant material on the Earth's surface) using optical satellite imagery.
12
+ Above ground biomass is an important component of the carbon cycle and is crucial for estimating crop yields, monitoring forest timber production, and quantifying the carbon sequestered by nature-based actions.
13
+
14
+ The model predicts above ground biomass from the Harmonized Landsat and Sentinel-2 (HLS) L30 optical satellite imagery and is fine-tuned using training labels from the
15
+ Global Ecosystem Dynamics Investigation (GEDI) L4A. Uniquely, the model has been fine-tuned using HLS and GEDI data collected from 15 biomes across the globe.
16
+ Please see [Model Description](Model Description) below for more details.
17
+
18
+ ## How to Get Started with the Model
19
+
20
+ This model was trained using [Terratorch](https://github.com/IBM/terratorch).
21
+
22
+ We make the weights as well as the configuration file that defines it available.
23
+
24
+ You can use it easily with Terratorch through:
25
+
26
+ ```python
27
+ from terratorch.cli_tools import LightningInferenceModel
28
+
29
+ ckpt_path = hf_hub_download(repo_id="ibm-granite/granite-geospatial-biomass", filename="biomass_model.ckpt")
30
+ config_path = hf_hub_download(repo_id="ibm-granite/granite-geospatial-biomass", filename="config.yaml")
31
+
32
+ model = LightningInferenceModel.from_config(config_path, ckpt_path)
33
+
34
+ inference_results, input_file_names = model.inference_on_dir(<input_directory>)
35
+ ```
36
+
37
+ For more details, check out the [Getting Started Notebook](https://github.com/ibm-granite/granite-geospatial-biomass/blob/main/notebooks/agb_getting_started.ipynb) which guides the user through three experiments:
38
+
39
+ 1. Zero-shot for all biomes
40
+ 2. Zero-shot for a single biome
41
+ 3. Few-shot for a single biome
42
+
43
+ ## Model Description
44
+
45
+ The granite-geospatial-biomass model is a geospatial foundation model that has been fine-tuned using HLS and GEDI data to perform regression.
46
+
47
+ The base foundation model from which the granite-geospatial-biomass model is fine-tuned is similar to that described in this [paper](https://arxiv.org/abs/2310.18660),
48
+ with the exception that the backbone is a Swin-B transformer. We opted for the Swin-B backbone instead of the ViT in the original paper because the Swin-B provides the following advantages:
49
+ - a smaller starting patch size which provides a higher effective resolution
50
+ - windowed attention which provides better computational efficiency
51
+ - hierarchical merging which provides a useful inductive bias
52
+
53
+ The base foundation model was pretrained using SimMIM, a self-supervised learning strategy based on masking large parts of the input (HLS2) data which are then reconstructed by the model. A small decoder composed of a single convolutional layer and a Pixel Shuffle module was added to the Swin-B backbone for the (pretraining) reconstruction task.
54
+
55
+ For fine-tuning, we replaced the small decoder with a UPerNet adapted for pixel-wise regression. We opted for the UPerNet because it provides fusion between transformer blocks, a similar intuition to the Unet which is consistently considered state-of-the-art for regression tasks with earth observation data. As the standard UPerNet implementation using the Swin-B backbone predicts a final feature map 4x smaller than the input, we appended two Pixel Shuffle layers to learn the upscaling. More details on the fine-tuned model can be found in this [paper](https://arxiv.org/abs/waiting-on-url).
56
+
57
+ <!-- this [paper](https://arxiv.org/abs/waiting-on-url). -->
58
+
59
+
60
+ ## Model Releases (along with the branch name where the models are stored):
61
+
62
+ - **tag v1 —** - 28/07/2024
63
+
64
+ - Stay tuned for more models!
65
+
66
+ ### Model Sources
67
+
68
+ - **Repository:** https://github.com/ibm-granite/granite-geospatial-biomass/
69
+ - **Paper (biomass):** https://arxiv.org/abs/waiting-on-url
70
+ - **Paper (foundation model):** https://arxiv.org/abs/2310.18660
71
+
72
+ ### External Blogs
73
+ - https://research.ibm.com/blog/img-geospatial-studio-think
74
+
75
+ ## Training Data
76
+
77
+ The model was trained on a collection of datasets provided by NASA:
78
+ - Harmonized Landsat-Sentinel 2 (HLS) L30: https://lpdaac.usgs.gov/products/hlss30v002/
79
+ - Global Ecosystem Dynamics Investigation (GEDI) L4A: https://doi.org/10.3334/ORNLDAAC/1907
80
+
81
+ For training and testing, the model requires a cloud-free snapshot of an area where all pixels are representative of the spectral bands for that location. The approach we used to create the cloud free images was to acquire HLS data during the leaf-on season for each hemisphere, analyze the timeseries, and select pixels that are not contaminated with clouds. We compute the mean value of each cloud-free pixel during the leaf-on season for each spectral band which is then assembled into a composite image representative for that area. The corresponding GEDI L4A biomass data obtained made during the same leaf-on season are interpolated to the HLS grid (CRS:4326) such that the measured biomass points are aligned with HLS data. GEDI data is spatially and temporaly sparse so pixels with no corresponding GEDI measurement are filled with a no data value.
82
+
83
+
84
+ <!-- TODO: add citation -->
85
+ <!-- ## Citation [optional]
86
+ Kindly cite the following paper, if you intend to use our model or its associated architectures/approaches in your
87
+ work
88
+
89
+ **BibTeX:**
90
+
91
+ ```
92
+ @misc{muszynski2024biomass,
93
+ TBD
94
+ }
95
+ ```
96
+
97
+ **APA:**
98
+
99
+ TBD -->
100
+
101
+ ## Model Card Authors
102
+
103
+ Julian Kuehnert, Levente Klein, Catherine Wanjiru, Carlos Gomes and Campbell Watson
104
+
105
+
106
+ ## IBM Public Repository Disclosure:
107
+
108
+ All content in this repository including code has been provided by IBM under the associated
109
+ open source software license and IBM is under no obligation to provide enhancements,
110
+ updates, or support. IBM developers produced this code as an
111
+ open source project (not as an IBM product), and IBM makes no assertions as to
112
+ the level of quality nor security, and will not be maintaining this code going forward.