Initial version

README.md

@@ -0,0 +1,72 @@
+---
+tags:
+- monai
+- medical
+library_name: monai
+license: unknown
+---
+
+# 2D Cardiac Valve Landmark Regressor
+
+This network identifies 10 different landmarks in 2D+t MR images of the heart (2 chamber, 3 chamber, and 4 chamber) representing the insertion locations of valve leaflets into the myocardial wall. These coordinates are used in part of the construction of 3D FEM cardiac models suitable for physics simulation of heart functions.
+
+Input images are individual 2D slices from the time series, and the output from the network is a `(2, 10)` set of 2D points in `HW` image coordinate space. The 10 coordinates correspond to the attachment point for these valves:
+
+1. Mitral anterior in 2CH
+2. Mitral posterior in 2CH
+3. Mitral septal in 3CH
+4. Mitral free wall in 3CH
+5. Mitral septal in 4CH
+6. Mitral free wall in 4CH
+7. Aortic septal
+8. Aortic free wall
+9. Tricuspid septal
+10. Tricuspid free wall
+
+Landmarks which do not appear in a particular image are predicted to be `(0, 0)` or close to this location. The mitral valve is expected to appear in all three views. Landmarks are not provided for the pulmonary valve.
+
+Example plot of landmarks on a single frame, see [view_results.ipynb](./view_results.ipynb) for visualising network output:
+
+![Landmark Example Image](./prediction_example.png)
+
+## Training
+
+The training script `train.json` is provided to train the network using a dataset of image pairs containing the MR image and a landmark image. This is done to reuse image-based transforms which do not currently operate on geometry. A number of other transforms are provided in `valve_landmarks.py` to implement Fourier-space dropout, image shifting which preserve landmarks, and smooth-field deformation applied to images and landmarks.
+
+The dataset used for training unfortunately cannot be made public, however the training script can be used with any NPZ file containing the training image stack in key `trainImgs` and landmark image stack in `trainLMImgs`, plus `testImgs` and `testLMImgs` containing validation data. The landmark images are defined as 0 for every non-landmark pixel, with landmark pixels contaning the following values for each landmark type:
+
+* 10: Mitral anterior in 2CH
+* 15: Mitral posterior in 2CH
+* 20: Mitral septal in 3CH
+* 25: Mitral free wall in 3CH
+* 30: Mitral septal in 4CH
+* 35: Mitral free wall in 4CH
+* 100: Aortic septal
+* 150: Aortic free wall
+* 200: Tricuspid septal
+* 250: Tricuspid free wall
+
+The following command will train with the default NPZ filename `./valvelandmarks.npz`:
+
+```sh
+PYTHONPATH=./scripts python -m monai.bundle run training --meta_file configs/metadata.json \
+    --config_file configs/train.json --bundle_root . --dataset_file /path/to/data --output_dir /path/to/outputs
+```
+
+## Inference
+
+The included `inference.json` script will run inference on a directory containing Nifti files whose images have shape `(256, 256, 1, N)` for `N` timesteps. For each image the output in the `output_dir` directory will be a npy file containing a result array of shape `(N, 2, 10)` storing the 10 coordinates for each `N` timesteps. Invoking this script can be done as follows, assuming the current directory is the bundle directory:
+
+```sh
+PYTHONPATH=./scripts python -m monai.bundle run evaluating --meta_file configs/metadata.json \
+    --config_file configs/inference.json --bundle_root . --dataset_dir /path/to/data --output_dir /path/to/outputs
+```
+
+It is important to set the `PYTHONPATH` variable since code in the provided scripts directory is necessary for inference. The provided test Nifti file can be placed in a directory which is then used as the `dataset_dir` value. This image was derived from [the AMRG Cardiac Atlas dataset](http://www.cardiacatlas.org/studies/amrg-cardiac-atlas) (AMRG Cardiac Atlas, Auckland MRI Research Group, Auckland, New Zealand). The results from this inference can be visualised by changing path values in [view_results.ipynb](./view_results.ipynb).
+
+
+### Reference
+
+The work for this model and its application is described in:
+
+`Kerfoot, E, King, CE, Ismail, T, Nordsletten, D & Miller, R 2021, Estimation of Cardiac Valve Annuli Motion with Deep Learning. in E Puyol Anton, M Pop, M Sermesant, V Campello, A Lalande, K Lekadir, A Suinesiaputra, O Camara & A Young (eds), Statistical Atlases and Computational Models of the Heart. MandMs and EMIDEC Challenges - 11th International Workshop, STACOM 2020, Held in Conjunction with MICCAI 2020, Revised Selected Papers. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 12592 LNCS, Springer Science and Business Media Deutschland GmbH, pp. 146-155, 11th International Workshop on Statistical Atlases and Computational Models of the Heart, STACOM 2020 held in Conjunction with MICCAI 2020, Lima, Peru, 4/10/2020. https://doi.org/10.1007/978-3-030-68107-4_15`

configs/inference.json

@@ -0,0 +1,121 @@
+{
+    "imports": [
+        "$import os",
+        "$import glob",
+        "$import scripts"
+    ],
+    "device": "$torch.device('cuda:0' if torch.cuda.is_available() else 'cpu')",
+    "ckpt_path": "$@bundle_root + '/models/model.pt'",
+    "dataset_dir": "/workspace/data",
+    "datalist": "$list(sorted(glob.glob(@dataset_dir + '/*.nii*')))",
+    "output_dir": "./output",
+    "network_def": {
+        "_target_": "scripts.valve_landmarks.PointRegressor",
+        "in_shape": [
+            1,
+            256,
+            256
+        ],
+        "out_shape": [
+            2,
+            10
+        ],
+        "channels": [
+            8,
+            16,
+            32,
+            64,
+            128
+        ],
+        "strides": [
+            2,
+            2,
+            2,
+            2,
+            2
+        ]
+    },
+    "network": "$@network_def.to(@device)",
+    "preprocessing": {
+        "_target_": "Compose",
+        "transforms": [
+            {
+                "_target_": "LoadImage",
+                "image_only": "true"
+            },
+            {
+                "_target_": "EnsureType",
+                "device": "@device"
+            },
+            {
+                "_target_": "Transpose",
+                "indices": [
+                    2,
+                    0,
+                    1,
+                    3
+                ]
+            },
+            {
+                "_target_": "ScaleIntensity"
+            }
+        ]
+    },
+    "dataset": {
+        "_target_": "Dataset",
+        "data": "@datalist",
+        "transform": "@preprocessing"
+    },
+    "dataloader": {
+        "_target_": "DataLoader",
+        "dataset": "@dataset",
+        "batch_size": 1,
+        "shuffle": false,
+        "num_workers": 0
+    },
+    "inferer": {
+        "_target_": "scripts.valve_landmarks.LandmarkInferer",
+        "spatial_dim": 2,
+        "stack_dim": 1
+    },
+    "postprocessing": {
+        "_target_": "Compose",
+        "transforms": [
+            {
+                "_target_": "SqueezeDimd",
+                "keys": "pred",
+                "dim": 0
+            },
+            {
+                "_target_": "scripts.valve_landmarks.NpySaverd",
+                "keys": "pred",
+                "output_dir": "@output_dir",
+                "data_root_dir": "@dataset_dir"
+            }
+        ]
+    },
+    "handlers": [
+        {
+            "_target_": "CheckpointLoader",
+            "_disabled_": "$not os.path.exists(@ckpt_path)",
+            "load_path": "@ckpt_path",
+            "load_dict": {
+                "net": "@network"
+            }
+        }
+    ],
+    "evaluator": {
+        "_target_": "SupervisedEvaluator",
+        "device": "@device",
+        "val_data_loader": "@dataloader",
+        "network": "@network",
+        "inferer": "@inferer",
+        "postprocessing": "@postprocessing",
+        "val_handlers": "@handlers",
+        "decollate": false,
+        "prepare_batch": "$lambda batch, dev,nb: (batch.to(dev),())"
+    },
+    "evaluating": [
+        "$@evaluator.run()"
+    ]
+}

configs/metadata.json

@@ -0,0 +1,83 @@
+{
+    "schema": "https://github.com/Project-MONAI/MONAI-extra-test-data/releases/download/0.8.1/meta_schema_20220729.json",
+    "version": "0.1.0",
+    "changelog": {
+        "0.1.0": "Initial version"
+    },
+    "monai_version": "1.0.0rc1",
+    "pytorch_version": "1.10.2",
+    "numpy_version": "1.21.2",
+    "optional_packages_version": {},
+    "task": "Given long axis MR images of the heart, identify valve insertion points through the full cardiac cycle",
+    "description": "This network is used to find where valves attach to heart to help construct 3D FEM models for computation. The output is an array of 10 2D coordinates.",
+    "authors": "Eric Kerfoot",
+    "copyright": "Copyright (c) Eric Kerfoot",
+    "references": [
+        "Kerfoot, E, King, CE, Ismail, T, Nordsletten, D & Miller, R 2021, Estimation of Cardiac Valve Annuli Motion with Deep Learning. https://doi.org/10.1007/978-3-030-68107-4_15"
+    ],
+    "intended_use": "This is suitable for research purposes only",
+    "image_classes": "Single channel data, intensity scaled to [0, 1]",
+    "data_source": "Non-public dataset comprised of hand-annotated full cycle long axis MR images",
+    "coordinate_values": {
+        "0": 10,
+        "1": 15,
+        "2": 20,
+        "3": 25,
+        "4": 30,
+        "5": 35,
+        "6": 100,
+        "7": 150,
+        "8": 200,
+        "9": 250
+    },
+    "coordinate_meanings": {
+        "0": "mitral anterior 2CH",
+        "1": "mitral posterior 2CH",
+        "2": "mitral septal 3CH",
+        "3": "mitral free wall 3CH",
+        "4": "mitral septal 4CH",
+        "5": "mitral free wall 4CH",
+        "6": "aortic septal",
+        "7": "aortic free wall",
+        "8": "tricuspid septal",
+        "9": "tricuspid free wall"
+    },
+    "network_data_format": {
+        "inputs": {
+            "image": {
+                "type": "image",
+                "format": "magnitude",
+                "modality": "MR",
+                "num_channels": 1,
+                "spatial_shape": [
+                    256,
+                    256
+                ],
+                "dtype": "float32",
+                "value_range": [],
+                "is_patch_data": false,
+                "channel_def": {
+                    "0": "image"
+                }
+            }
+        },
+        "outputs": {
+            "pred": {
+                "type": "tuples",
+                "format": "points",
+                "num_channels": 2,
+                "spatial_shape": [
+                    2,
+                    10
+                ],
+                "dtype": "float32",
+                "value_range": [],
+                "is_patch_data": false,
+                "channel_def": {
+                    "0": "Y Dimension",
+                    "1": "X Dimension"
+                }
+            }
+        }
+    }
+}

configs/train.json

@@ -0,0 +1,270 @@
+{
+    "imports": [
+        "$import datetime",
+        "$import numpy as np",
+        "$import torch",
+        "$import ignite",
+        "$import scripts"
+    ],
+    "device": "$torch.device('cuda:0' if torch.cuda.is_available() else 'cpu')",
+    "ckpt_path": "$@bundle_root + '/models/model.pt'",
+    "dataset_file": "./valvelandmarks.npz",
+    "output_dir": "$datetime.datetime.now().strftime('./results/output_%y%m%d_%H%M%S')",
+    "network_def": {
+        "_target_": "scripts.valve_landmarks.PointRegressor",
+        "in_shape": [
+            1,
+            256,
+            256
+        ],
+        "out_shape": [
+            2,
+            10
+        ],
+        "channels": [
+            8,
+            16,
+            32,
+            64,
+            128
+        ],
+        "strides": [
+            2,
+            2,
+            2,
+            2,
+            2
+        ]
+    },
+    "network": "$@network_def.to(@device)",
+    "im_shape": [
+        1,
+        256,
+        256
+    ],
+    "both_keys": [
+        "image",
+        "label"
+    ],
+    "rand_prob": 0.5,
+    "train_transforms": {
+        "_target_": "Compose",
+        "transforms": [
+            {
+                "_target_": "EnsureTyped",
+                "keys": "@both_keys",
+                "data_type": "numpy",
+                "dtype": "$(np.float32, np.int32)"
+            },
+            {
+                "_target_": "EnsureTyped",
+                "keys": "@both_keys"
+            },
+            {
+                "_target_": "ScaleIntensityd",
+                "keys": "image"
+            },
+            {
+                "_target_": "EnsureChannelFirstd",
+                "keys": "@both_keys"
+            },
+            {
+                "_target_": "RandAxisFlipd",
+                "keys": "@both_keys",
+                "prob": "@rand_prob"
+            },
+            {
+                "_target_": "RandRotate90d",
+                "keys": "@both_keys",
+                "prob": "@rand_prob"
+            },
+            {
+                "_target_": "RandSmoothFieldAdjustIntensityd",
+                "keys": "image",
+                "prob": "@rand_prob",
+                "spatial_size": "@im_shape",
+                "rand_size": [
+                    5,
+                    5
+                ],
+                "gamma": [
+                    0.1,
+                    1
+                ],
+                "mode": "$monai.utils.InterpolateMode.BICUBIC",
+                "align_corners": true
+            },
+            {
+                "_target_": "RandGaussianNoised",
+                "keys": "image",
+                "prob": "@rand_prob",
+                "std": 0.05
+            },
+            {
+                "_target_": "scripts.valve_landmarks.RandFourierDropoutd",
+                "keys": "image",
+                "prob": "@rand_prob"
+            },
+            {
+                "_target_": "scripts.valve_landmarks.RandImageLMDeformd",
+                "prob": "@rand_prob",
+                "spatial_size": [
+                    256,
+                    256
+                ],
+                "rand_size": [
+                    7,
+                    7
+                ],
+                "pad": 2,
+                "field_mode": "$monai.utils.InterpolateMode.BICUBIC",
+                "align_corners": true,
+                "def_range": 0.05
+            },
+            {
+                "_target_": "scripts.valve_landmarks.RandLMShiftd",
+                "keys": "@both_keys",
+                "prob": "@rand_prob",
+                "spatial_size": [
+                    256,
+                    256
+                ],
+                "max_shift": 8
+            },
+            {
+                "_target_": "Lambdad",
+                "keys": "label",
+                "func": "$scripts.valve_landmarks.convert_lm_image_t"
+            }
+        ]
+    },
+    "eval_transforms": {
+        "_target_": "Compose",
+        "transforms": [
+            {
+                "_target_": "EnsureTyped",
+                "keys": "@both_keys",
+                "data_type": "numpy",
+                "dtype": "$(np.float32, np.int32)"
+            },
+            {
+                "_target_": "EnsureTyped",
+                "keys": "@both_keys"
+            },
+            {
+                "_target_": "ScaleIntensityd",
+                "keys": "image"
+            },
+            {
+                "_target_": "EnsureChannelFirstd",
+                "keys": "@both_keys"
+            },
+            {
+                "_target_": "Lambdad",
+                "keys": "label",
+                "func": "$scripts.valve_landmarks.convert_lm_image_t"
+            }
+        ]
+    },
+    "train_dataset": {
+        "_target_": "NPZDictItemDataset",
+        "npzfile": "$@dataset_file",
+        "keys": {
+            "trainImgs": "image",
+            "trainLMImgs": "label"
+        },
+        "transform": "@train_transforms"
+    },
+    "eval_dataset": {
+        "_target_": "NPZDictItemDataset",
+        "npzfile": "$@dataset_file",
+        "keys": {
+            "testImgs": "image",
+            "testLMImgs": "label"
+        },
+        "transform": "@eval_transforms"
+    },
+    "num_iters": 400,
+    "batch_size": 600,
+    "num_epochs": 100,
+    "num_substeps": 3,
+    "sampler": {
+        "_target_": "torch.utils.data.WeightedRandomSampler",
+        "weights": "$torch.ones(len(@train_dataset))",
+        "replacement": true,
+        "num_samples": "$@num_iters*@batch_size"
+    },
+    "train_dataloader": {
+        "_target_": "ThreadDataLoader",
+        "dataset": "@train_dataset",
+        "batch_size": "@batch_size",
+        "repeats": "@num_substeps",
+        "num_workers": 8,
+        "sampler": "@sampler"
+    },
+    "eval_dataloader": {
+        "_target_": "DataLoader",
+        "dataset": "@eval_dataset",
+        "batch_size": "@batch_size",
+        "num_workers": 8
+    },
+    "lossfn": {
+        "_target_": "torch.nn.L1Loss"
+    },
+    "optimizer": {
+        "_target_": "torch.optim.Adam",
+        "params": "$@network.parameters()",
+        "lr": 0.0001
+    },
+    "evaluator": {
+        "_target_": "SupervisedEvaluator",
+        "device": "@device",
+        "val_data_loader": "@eval_dataloader",
+        "network": "@network",
+        "key_val_metric": {
+            "val_mean_dist": {
+                "_target_": "ignite.metrics.MeanPairwiseDistance",
+                "output_transform": "$scripts.valve_landmarks._output_lm_trans"
+            }
+        },
+        "metric_cmp_fn": "$lambda current, prev: prev < 0 or current < prev",
+        "val_handlers": [
+            {
+                "_target_": "StatsHandler",
+                "output_transform": "$lambda x: None"
+            }
+        ]
+    },
+    "handlers": [
+        {
+            "_target_": "ValidationHandler",
+            "validator": "@evaluator",
+            "epoch_level": true,
+            "interval": 1
+        },
+        {
+            "_target_": "CheckpointSaver",
+            "save_dir": "@output_dir",
+            "save_dict": {
+                "net": "@network"
+            },
+            "save_interval": 1,
+            "save_final": true,
+            "epoch_level": true
+        }
+    ],
+    "trainer": {
+        "_target_": "SupervisedTrainer",
+        "max_epochs": "@num_epochs",
+        "device": "@device",
+        "train_data_loader": "@train_dataloader",
+        "network": "@network",
+        "loss_function": "@lossfn",
+        "optimizer": "@optimizer",
+        "key_train_metric": null,
+        "train_handlers": "@handlers"
+    },
+    "training": [
+        "$@trainer.run()"
+    ]
+}

docs/AMRGAtlas_0031.nii.gz

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ddac587d7ced3c7f892113e379256f055895162a5077f741c1bdc79e1a70b558
+size 2190289

docs/AMRGAtlas_0031_key-pred.npy

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:48c44c9b85bb080e33a97b5421fde7037e039e235dbfbf08eff22a8fd2901b89
+size 2688

docs/README.md

@@ -0,0 +1,65 @@
+
+# 2D Cardiac Valve Landmark Regressor
+
+This network identifies 10 different landmarks in 2D+t MR images of the heart (2 chamber, 3 chamber, and 4 chamber) representing the insertion locations of valve leaflets into the myocardial wall. These coordinates are used in part of the construction of 3D FEM cardiac models suitable for physics simulation of heart functions.
+
+Input images are individual 2D slices from the time series, and the output from the network is a `(2, 10)` set of 2D points in `HW` image coordinate space. The 10 coordinates correspond to the attachment point for these valves:
+
+1. Mitral anterior in 2CH
+2. Mitral posterior in 2CH
+3. Mitral septal in 3CH
+4. Mitral free wall in 3CH
+5. Mitral septal in 4CH
+6. Mitral free wall in 4CH
+7. Aortic septal
+8. Aortic free wall
+9. Tricuspid septal
+10. Tricuspid free wall
+
+Landmarks which do not appear in a particular image are predicted to be `(0, 0)` or close to this location. The mitral valve is expected to appear in all three views. Landmarks are not provided for the pulmonary valve.
+
+Example plot of landmarks on a single frame, see [view_results.ipynb](./view_results.ipynb) for visualising network output:
+
+![Landmark Example Image](./prediction_example.png)
+
+## Training
+
+The training script `train.json` is provided to train the network using a dataset of image pairs containing the MR image and a landmark image. This is done to reuse image-based transforms which do not currently operate on geometry. A number of other transforms are provided in `valve_landmarks.py` to implement Fourier-space dropout, image shifting which preserve landmarks, and smooth-field deformation applied to images and landmarks.
+
+The dataset used for training unfortunately cannot be made public, however the training script can be used with any NPZ file containing the training image stack in key `trainImgs` and landmark image stack in `trainLMImgs`, plus `testImgs` and `testLMImgs` containing validation data. The landmark images are defined as 0 for every non-landmark pixel, with landmark pixels contaning the following values for each landmark type:
+
+* 10: Mitral anterior in 2CH
+* 15: Mitral posterior in 2CH
+* 20: Mitral septal in 3CH
+* 25: Mitral free wall in 3CH
+* 30: Mitral septal in 4CH
+* 35: Mitral free wall in 4CH
+* 100: Aortic septal
+* 150: Aortic free wall
+* 200: Tricuspid septal
+* 250: Tricuspid free wall
+
+The following command will train with the default NPZ filename `./valvelandmarks.npz`:
+
+```sh
+PYTHONPATH=./scripts python -m monai.bundle run training --meta_file configs/metadata.json \
+    --config_file configs/train.json --bundle_root . --dataset_file /path/to/data --output_dir /path/to/outputs
+```
+
+## Inference
+
+The included `inference.json` script will run inference on a directory containing Nifti files whose images have shape `(256, 256, 1, N)` for `N` timesteps. For each image the output in the `output_dir` directory will be a npy file containing a result array of shape `(N, 2, 10)` storing the 10 coordinates for each `N` timesteps. Invoking this script can be done as follows, assuming the current directory is the bundle directory:
+
+```sh
+PYTHONPATH=./scripts python -m monai.bundle run evaluating --meta_file configs/metadata.json \
+    --config_file configs/inference.json --bundle_root . --dataset_dir /path/to/data --output_dir /path/to/outputs
+```
+
+It is important to set the `PYTHONPATH` variable since code in the provided scripts directory is necessary for inference. The provided test Nifti file can be placed in a directory which is then used as the `dataset_dir` value. This image was derived from [the AMRG Cardiac Atlas dataset](http://www.cardiacatlas.org/studies/amrg-cardiac-atlas) (AMRG Cardiac Atlas, Auckland MRI Research Group, Auckland, New Zealand). The results from this inference can be visualised by changing path values in [view_results.ipynb](./view_results.ipynb).
+
+
+### Reference
+
+The work for this model and its application is described in:
+
+`Kerfoot, E, King, CE, Ismail, T, Nordsletten, D & Miller, R 2021, Estimation of Cardiac Valve Annuli Motion with Deep Learning. in E Puyol Anton, M Pop, M Sermesant, V Campello, A Lalande, K Lekadir, A Suinesiaputra, O Camara & A Young (eds), Statistical Atlases and Computational Models of the Heart. MandMs and EMIDEC Challenges - 11th International Workshop, STACOM 2020, Held in Conjunction with MICCAI 2020, Revised Selected Papers. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 12592 LNCS, Springer Science and Business Media Deutschland GmbH, pp. 146-155, 11th International Workshop on Statistical Atlases and Computational Models of the Heart, STACOM 2020 held in Conjunction with MICCAI 2020, Lima, Peru, 4/10/2020. https://doi.org/10.1007/978-3-030-68107-4_15`

docs/license.txt

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2022 Eric Kerfoot
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

models/model.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b0187ee65b150c0c693b16e6a65d0b3f659bed163423a1e4a5a6b3bb4fbeb7bf
+size 13349733

scripts/valve_landmarks.py

@@ -0,0 +1,327 @@
+# Copyright (c) 2022 Eric Kerfoot under MIT license, see license.txt
+
+import os
+from typing import Any, Callable, Sequence
+
+import monai
+import monai.transforms as mt
+import numpy as np
+import torch
+import torch.nn as nn
+from monai.data.meta_obj import get_track_meta
+from monai.networks.blocks import ConvDenseBlock, Convolution
+from monai.networks.layers import Flatten, Reshape
+from monai.networks.nets import Regressor
+from monai.networks.utils import meshgrid_ij
+from monai.utils import CommonKeys
+from monai.utils import ImageMetaKey as Key
+from monai.utils import convert_to_numpy, convert_to_tensor
+
+# relates the label in training images to index of landmark point
+LM_INDICES = {
+    10: 0,  # mitral anterior 2CH
+    15: 1,  # mitral posterior 2CH
+    20: 2,  # mitral septal 3CH
+    25: 3,  # mitral free wall 3CH
+    30: 4,  # mitral septal 4CH
+    35: 5,  # mitral free wall 4CH
+    100: 6,  # aortic septal
+    150: 7,  # aortic free wall
+    200: 8,  # tricuspid septal
+    250: 9,  # tricuspid free wall
+}
+
+output_trans = monai.handlers.from_engine(["pred", "label"])
+
+
+def _output_lm_trans(data):
+    pred, label = output_trans(data)
+    return [p.permute(1, 0) for p in pred], [l.permute(1, 0) for l in label]
+
+
+def convert_lm_image_t(lm_image):
+    """Convert a landmark image into a (2,N) tensor of landmark coordinates."""
+    lmarray = torch.zeros((2, len(LM_INDICES)), dtype=torch.float32).to(lm_image.device)
+
+    for _, y, x in np.argwhere(lm_image.cpu().numpy() != 0):
+        im_id = int(lm_image[0, y, x])
+        lm_index = LM_INDICES[im_id]
+
+        lmarray[0, lm_index] = y
+        lmarray[1, lm_index] = x
+
+    return lmarray
+
+
+class ParallelCat(nn.Module):
+    """
+    Apply the same input to each of the given modules and concatenate their results together.
+
+    Args:
+        catmodules: sequence of nn.Module objects to apply inputs to
+        cat_dim: dimension to concatenate along when combining outputs
+    """
+
+    def __init__(self, catmodules: Sequence[nn.Module], cat_dim: int = 1):
+        super().__init__()
+        self.cat_dim = cat_dim
+
+        for i, s in enumerate(catmodules):
+            self.add_module(f"catmodule_{i}", s)
+
+    def forward(self, x):
+        tensors = [s(x) for s in self.children()]
+        return torch.cat(tensors, self.cat_dim)
+
+
+class PointRegressor(Regressor):
+    """Regressor defined as a sequence of dense blocks followed by convolution/linear layers for each landmark."""
+
+    def _get_layer(self, in_channels, out_channels, strides, is_last):
+        dout = out_channels - in_channels
+        dilations = [1, 2, 4]
+        dchannels = [dout // 3, dout // 3, dout // 3 + dout % 3]
+
+        db = ConvDenseBlock(
+            spatial_dims=self.dimensions,
+            in_channels=in_channels,
+            channels=dchannels,
+            dilations=dilations,
+            kernel_size=self.kernel_size,
+            num_res_units=self.num_res_units,
+            act=self.act,
+            norm=self.norm,
+            dropout=self.dropout,
+            bias=self.bias,
+        )
+
+        conv = Convolution(
+            spatial_dims=self.dimensions,
+            in_channels=out_channels,
+            out_channels=out_channels,
+            strides=strides,
+            kernel_size=self.kernel_size,
+            act=self.act,
+            norm=self.norm,
+            dropout=self.dropout,
+            bias=self.bias,
+            conv_only=is_last,
+        )
+
+        return nn.Sequential(db, conv)
+
+    def _get_final_layer(self, in_shape):
+        point_paths = []
+
+        for _ in range(self.out_shape[1]):
+            conv = Convolution(
+                spatial_dims=self.dimensions,
+                in_channels=in_shape[0],
+                out_channels=in_shape[0] * 2,
+                strides=2,
+                kernel_size=self.kernel_size,
+                act=self.act,
+                norm=self.norm,
+                dropout=self.dropout,
+                conv_only=True,
+            )
+            linear = nn.Linear(int(np.product(in_shape)) // 2, self.out_shape[0])
+            point_paths.append(nn.Sequential(conv, Flatten(), linear))
+
+        return torch.nn.Sequential(ParallelCat(point_paths), Reshape(*self.out_shape))
+
+
+class LandmarkInferer(monai.inferers.Inferer):
+    """Applies inference on 2D slices from 3D volumes."""
+
+    def __init__(self, spatial_dim=0, stack_dim=-1):
+        self.spatial_dim = spatial_dim
+        self.stack_dim = stack_dim
+
+    def __call__(self, inputs: torch.Tensor, network: Callable[..., torch.Tensor], *args: Any, **kwargs: Any):
+        if inputs.ndim != 5:
+            raise ValueError(f"Input volume to inferer must have shape BCDHW, input shape is {inputs.shape}")
+
+        results = []
+        input_slices = [slice(None) for _ in range(inputs.ndim)]
+
+        for idx in range(inputs.shape[self.spatial_dim + 2]):
+            input_slices[self.spatial_dim + 2] = idx
+            input_2d = inputs[input_slices]  # BCDHW -> BCHW by iterating over one of DHW
+
+            result = network(input_2d, *args, **kwargs)
+            results.append(result)
+
+        result = torch.stack(results, self.stack_dim)
+        return result
+
+
+class NpySaverd(mt.MapTransform):
+    """Saves tensors/arrays to Numpy npy files."""
+
+    def __init__(self, keys, output_dir, data_root_dir):
+        super().__init__(keys)
+        self.output_dir = output_dir
+        self.data_root_dir = data_root_dir
+        self.folder_layout = monai.data.FolderLayout(
+            self.output_dir, extension=".npy", data_root_dir=self.data_root_dir
+        )
+
+    def __call__(self, d):
+        if not os.path.exists(self.output_dir):
+            os.makedirs(self.output_dir, exist_ok=True)
+
+        for key in self.key_iterator(d):
+            orig_filename = d[key].meta[Key.FILENAME_OR_OBJ]
+            if isinstance(orig_filename, (list, tuple)):
+                orig_filename = orig_filename[0]
+
+            out_filename = self.folder_layout.filename(orig_filename, key=key)
+
+            np.save(out_filename, convert_to_numpy(d[key]))
+
+        return d
+
+
+class FourierDropout(mt.Transform, mt.Fourier):
+    """
+    Apply dropout in Fourier space to corrupt images. This works by zeroing out pixels with greater probability the
+    farther from the centre they are. All pixels closer than `min_dist` to the center are preserved, all beyond
+    `max_dist` become 0. Distances from the centre to an edge in a given dimension are defined as 1.0.
+
+    Args:
+        min_dist: minimum distance to apply dropout, must be >0, smaller values will cause greater corruption
+        max_dist: maximal distance to apply dropout, must be greater than `min_dist`, all pixels beyond become 0
+    """
+
+    def __init__(self, min_dist: float = 0.1, max_dist: float = 0.9):
+        super().__init__()
+        self.min_dist = min_dist
+        self.max_dist = max_dist
+        self.prob_field = None
+        self.field_shape = None
+
+    def _get_prob_field(self, shape):
+        shape = tuple(shape)
+        if shape != self.field_shape:
+            self.field_shape = shape
+            spaces = [torch.linspace(-1, 1, s) for s in shape[1:]]
+            grids = meshgrid_ij(*spaces)
+            # middle is 0, mid edges 1, corners sqrt(2)
+            self.prob_field = torch.stack(grids).pow_(2).sum(axis=0).sqrt_()
+
+        return self.prob_field
+
+    def __call__(self, im):
+        probfield = self._get_prob_field(im.shape).to(im.device)
+
+        # rand range from min_dist to max_dist
+        dropout = torch.rand_like(im).mul_(self.max_dist - self.min_dist).add_(self.min_dist)
+        # keep pixel if dropout value is greater than distance from center, so less likely farther from center
+        dropout = dropout.ge_(probfield)
+
+        result = self.shift_fourier(im, im.ndim - 1)
+        result.mul_(dropout)
+        result = self.inv_shift_fourier(result, im.ndim - 1)
+
+        return convert_to_tensor(result, track_meta=get_track_meta())
+
+
+class RandFourierDropout(mt.RandomizableTransform):
+    def __init__(self, min_dist=0.1, max_dist=0.9, prob=0.1):
+        mt.RandomizableTransform.__init__(self, prob)
+        self.dropper = FourierDropout(min_dist, max_dist)
+
+    def __call__(self, im, randomize: bool = True):
+        if randomize:
+            self.randomize(None)
+
+        if self._do_transform:
+            im = self.dropper(im)
+        else:
+            im = convert_to_tensor(im, track_meta=get_track_meta())
+
+        return im
+
+
+class RandFourierDropoutd(mt.RandomizableTransform, mt.MapTransform):
+    def __init__(self, keys, min_dist=0.1, max_dist=0.9, prob=0.1):
+        mt.RandomizableTransform.__init__(self, prob)
+        mt.MapTransform.__init__(self, keys)
+        self.dropper = FourierDropout(min_dist, max_dist)
+
+    def __call__(self, data, randomize: bool = True):
+        d = dict(data)
+
+        if randomize:
+            self.randomize(None)
+
+        for key in self.key_iterator(d):
+            if self._do_transform:
+                d[key] = self.dropper(d[key])
+            else:
+                d[key] = convert_to_tensor(d[key], track_meta=get_track_meta())
+
+        return d
+
+
+class RandImageLMDeformd(mt.RandSmoothDeform):
+    """Apply smooth random deformation to the image and landmark locations."""
+
+    def __call__(self, d):
+        d = dict(d)
+        old_label = d[CommonKeys.LABEL]
+        new_label = torch.zeros_like(old_label)
+
+        d[CommonKeys.IMAGE] = super().__call__(d[CommonKeys.IMAGE])
+
+        if self._do_transform:
+            field = self.sfield()
+            labels = np.argwhere(d[CommonKeys.LABEL][0].cpu().numpy() > 0)
+
+            # moving the landmarks this way prevents losing some to
+            # interpolation errors if deformation were applied the landmark image
+            for y, x in labels:
+                dy = int(field[0, y, x] * new_label.shape[1] / 2)
+                dx = int(field[1, y, x] * new_label.shape[2] / 2)
+
+                new_label[:, y - dy, x - dx] = old_label[:, y, x]
+
+            d[CommonKeys.LABEL] = new_label
+
+        return d
+
+
+class RandLMShiftd(mt.RandomizableTransform, mt.MapTransform):
+    """Randomly shift the image and landmark image in either direction in integer amounts."""
+
+    def __init__(self, keys, spatial_size, max_shift=0, prob=0.1):
+        mt.RandomizableTransform.__init__(self, prob=prob)
+        mt.MapTransform.__init__(self, keys=keys)
+
+        self.spatial_size = tuple(spatial_size)
+        self.max_shift = max_shift
+        self.padder = mt.BorderPad(self.max_shift)
+        self.unpadder = mt.CenterSpatialCrop(self.spatial_size)
+        self.shift = (0,) * len(self.spatial_size)
+        self.roll_dims = list(range(1, len(self.spatial_size) + 1))
+
+    def randomize(self, data):
+        super().randomize(None)
+        if self._do_transform:
+            rs = torch.randint(-self.max_shift, self.max_shift, (len(self.spatial_size),), dtype=torch.int32)
+            self.shift = tuple(rs.tolist())
+
+    def __call__(self, d, randomize: bool = True):
+        d = dict(d)
+
+        if randomize:
+            self.randomize(None)
+
+        if self._do_transform:
+            for key in self.key_iterator(d):
+                imp = self.padder(d[key])
+                ims = torch.roll(imp, self.shift, self.roll_dims)  # prevents interpolation of landmark image
+                d[key] = self.unpadder(ims)
+
+        return d