update learning rate and readme

README.md

@@ -11,6 +11,11 @@ A neural architecture search algorithm for volumetric (3D) segmentation of the p
 # Model Overview
 This model is trained using the state-of-the-art algorithm [1] of the "Medical Segmentation Decathlon Challenge 2018" with 196 training images, 56 validation images, and 28 testing images.
 
+This model is trained using the neural network model from the neural architecture search algorithm, DiNTS [1].
+
+![image](https://developer.download.nvidia.com/assets/Clara/Images/clara_pt_net_arch_search_segmentation_workflow_4-1.png
+)
+
 ## Data
 The training dataset is Task07_Pancreas.tar from http://medicaldecathlon.com/. And the data list/split can be created with the script `scripts/prepare_datalist.py`.
 
@@ -19,17 +24,66 @@ The training was performed with at least 16GB-memory GPUs.
 
 Actual Model Input: 96 x 96 x 96
 
+### Neural Architecture Search Configuration
+The neural architecture search was performed with the following:
+
+- AMP: True
+- Optimizer: SGD
+- Initial Learning Rate: 0.025
+- Loss: DiceCELoss
+
+### Training Configuration
+The training was performed with the following:
+
+- AMP: True
+- Optimizer: SGD
+- (Initial) Learning Rate: 0.025
+- Loss: DiceCELoss
+- Note: If out-of-memory or program crash occurs while caching the data set, please change the cache\_rate in CacheDataset to a lower value in the range (0, 1).
+
+The segmentation of pancreas region is formulated as the voxel-wise 3-class classification. Each voxel is predicted as either foreground (pancreas body, tumour) or background. And the model is optimized with gradient descent method minimizing soft dice loss and cross-entropy loss between the predicted mask and ground truth segmentation.
+
+### Data Pre-processing and Augmentation
+
+Input: 1 channel CT image with intensity in HU
+
+- Converting to channel first
+- Normalizing and clipping intensities of tissue window to [0,1]
+- Cropping foreground surrounding regions
+- Cropping random fixed sized regions of size [96, 96, 96] with the center being a foreground or background voxel at ratio 1 : 1
+- Randomly rotating volumes
+- Randomly zooming volumes
+- Randomly smoothing volumes with Gaussian kernels
+- Randomly scaling intensity of the volume
+- Randomly shifting intensity of the volume
+- Randomly adding Gaussian noises
+- Randomly flipping volumes
+
+### Sliding-window Inference
+Inference is performed in a sliding window manner with a specified stride.
+
 ## Input and output formats
 Input: 1 channel CT image
 
 Output: 3 channels: Label 2: pancreatic tumor; Label 1: pancreas; Label 0: everything else
 
-## Scores
+## Performance
 This model achieves the following Dice score on the validation data (our own split from the training dataset):
 
-Mean Dice = 0.72
+Mean Dice = 0.62
+
+Training loss over 3200 epochs (the bright curve is smoothed, and the dark one is the actual curve)
+
+![image](https://developer.download.nvidia.com/assets/Clara/Images/clara_pt_net_arch_search_segmentation_train_4-2.png)
+
+Validation mean dice score over 3200 epochs (the bright curve is smoothed, and the dark one is the actual curve)
+
+![image](https://developer.download.nvidia.com/assets/Clara/Images/clara_pt_net_arch_search_segmentation_validation_4-2.png)
 
-## commands example
+### Searched Architecture Visualization
+Users can install Graphviz for visualization of searched architectures (needed in custom/decode_plot.py). The edges between nodes indicate global structure, and numbers next to edges represent different operations in the cell searching space. An example of searched architecture is shown as follows:
+
+## Commands Example
 Create data split (.json file):
 
 ```
@@ -72,6 +126,12 @@ Execute inference:
 python -m monai.bundle run evaluating --meta_file configs/metadata.json --config_file configs/inference.yaml --logging_file configs/logging.conf
 ```
 
+Export checkpoint for TorchScript
+
+```
+python -m monai.bundle ckpt_export network_def --filepath models/model.ts --ckpt_file models/model.pt --meta_file configs/metadata.json --config_file configs/inference.yaml
+```
+
 # Disclaimer
 This is an example, not to be used for diagnostic purposes.
 

configs/metadata.json

@@ -1,7 +1,8 @@
 {
     "schema": "https://github.com/Project-MONAI/MONAI-extra-test-data/releases/download/0.8.1/meta_schema_20220324.json",
-    "version": "0.3.2",
+    "version": "0.3.3",
     "changelog": {
+        "0.3.3": "update learning rate and readme",
         "0.3.2": "update to use monai 1.0.1",
         "0.3.1": "fix license Copyright error",
         "0.3.0": "update license files",

configs/multi_gpu_train.yaml

@@ -6,7 +6,7 @@ network:
   find_unused_parameters: true
   device_ids:
   - "@device"
-optimizer#lr: "$0.0125*dist.get_world_size()"
+optimizer#lr: "$0.025*dist.get_world_size()"
 lr_scheduler#step_size: "$80*dist.get_world_size()"
 train#handlers:
   - _target_: LrScheduleHandler

docs/README.md

@@ -4,6 +4,11 @@ A neural architecture search algorithm for volumetric (3D) segmentation of the p
 # Model Overview
 This model is trained using the state-of-the-art algorithm [1] of the "Medical Segmentation Decathlon Challenge 2018" with 196 training images, 56 validation images, and 28 testing images.
 
+This model is trained using the neural network model from the neural architecture search algorithm, DiNTS [1].
+
+![image](https://developer.download.nvidia.com/assets/Clara/Images/clara_pt_net_arch_search_segmentation_workflow_4-1.png
+)
+
 ## Data
 The training dataset is Task07_Pancreas.tar from http://medicaldecathlon.com/. And the data list/split can be created with the script `scripts/prepare_datalist.py`.
 
@@ -12,17 +17,66 @@ The training was performed with at least 16GB-memory GPUs.
 
 Actual Model Input: 96 x 96 x 96
 
+### Neural Architecture Search Configuration
+The neural architecture search was performed with the following:
+
+- AMP: True
+- Optimizer: SGD
+- Initial Learning Rate: 0.025
+- Loss: DiceCELoss
+
+### Training Configuration
+The training was performed with the following:
+
+- AMP: True
+- Optimizer: SGD
+- (Initial) Learning Rate: 0.025
+- Loss: DiceCELoss
+- Note: If out-of-memory or program crash occurs while caching the data set, please change the cache\_rate in CacheDataset to a lower value in the range (0, 1).
+
+The segmentation of pancreas region is formulated as the voxel-wise 3-class classification. Each voxel is predicted as either foreground (pancreas body, tumour) or background. And the model is optimized with gradient descent method minimizing soft dice loss and cross-entropy loss between the predicted mask and ground truth segmentation.
+
+### Data Pre-processing and Augmentation
+
+Input: 1 channel CT image with intensity in HU
+
+- Converting to channel first
+- Normalizing and clipping intensities of tissue window to [0,1]
+- Cropping foreground surrounding regions
+- Cropping random fixed sized regions of size [96, 96, 96] with the center being a foreground or background voxel at ratio 1 : 1
+- Randomly rotating volumes
+- Randomly zooming volumes
+- Randomly smoothing volumes with Gaussian kernels
+- Randomly scaling intensity of the volume
+- Randomly shifting intensity of the volume
+- Randomly adding Gaussian noises
+- Randomly flipping volumes
+
+### Sliding-window Inference
+Inference is performed in a sliding window manner with a specified stride.
+
 ## Input and output formats
 Input: 1 channel CT image
 
 Output: 3 channels: Label 2: pancreatic tumor; Label 1: pancreas; Label 0: everything else
 
-## Scores
+## Performance
 This model achieves the following Dice score on the validation data (our own split from the training dataset):
 
-Mean Dice = 0.72
+Mean Dice = 0.62
+
+Training loss over 3200 epochs (the bright curve is smoothed, and the dark one is the actual curve)
+
+![image](https://developer.download.nvidia.com/assets/Clara/Images/clara_pt_net_arch_search_segmentation_train_4-2.png)
+
+Validation mean dice score over 3200 epochs (the bright curve is smoothed, and the dark one is the actual curve)
+
+![image](https://developer.download.nvidia.com/assets/Clara/Images/clara_pt_net_arch_search_segmentation_validation_4-2.png)
 
-## commands example
+### Searched Architecture Visualization
+Users can install Graphviz for visualization of searched architectures (needed in custom/decode_plot.py). The edges between nodes indicate global structure, and numbers next to edges represent different operations in the cell searching space. An example of searched architecture is shown as follows:
+
+## Commands Example
 Create data split (.json file):
 
 ```
@@ -65,6 +119,12 @@ Execute inference:
 python -m monai.bundle run evaluating --meta_file configs/metadata.json --config_file configs/inference.yaml --logging_file configs/logging.conf
 ```
 
+Export checkpoint for TorchScript
+
+```
+python -m monai.bundle ckpt_export network_def --filepath models/model.ts --ckpt_file models/model.pt --meta_file configs/metadata.json --config_file configs/inference.yaml
+```
+
 # Disclaimer
 This is an example, not to be used for diagnostic purposes.
 

models/model.pt

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:975201057eb16225a1abfd047cb9b293f6d481dc604468d512710f3543f29066
-size 616210421
+oid sha256:4de79b954bb197c9a75d198c7b1038ad504cf0e370ae8b055c49c134c7b0e883
+size 534788757

models/model.ts

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:526d2bdb4d88f6f55f2d88eb0c79deeeb90b3ced182269e33cdc5da6e46ea5fb
-size 616338455
+oid sha256:d9c47d9b4d1ec457dfa28c9e03c4ab847cdbe6ca825e3a84a245ca8a9caa631f
+size 534978281

models/search_code_18590.pt

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:01e361e9843e2f4e5ff1599da0abac77013ea38cab8fdd6c9286bb6572c9a32d
-size 4335
+oid sha256:21a2a05b173e9a5a80a009ce4baa3dfd8118463e7c26a1ef49cce573a3e662cc
+size 4355