title: Diabetic Retinopathy Detection App
emoji: 🐢
colorFrom: blue
colorTo: purple
sdk: gradio
sdk_version: 4.22.0
app_file: app.py
license: mit
Diabetic Retinopathy Detection: Utilizing Multiprocessing for Processing Large Datasets and Transfer Learning to Fine-Tune Deep Learning Models
Efficiently process large datasets & develop advanced model pipelines for diabetic retinopathy detection. Streamlining diagnosis.
TL;DR:
In this project, large datasets are efficiently handled by downloading, extracting, and preparing them for analysis. Utilizing PyTorch Lightning, a robust system for diabetic retinopathy detection is developed, categorizing images into distinct disease stages. The model pipeline is enriched with various pretrained backbone models, with progress tracked using TensorBoard. Furthermore, a user-friendly web app is created to showcase the model's capabilities. The approach pursued aims to streamline both data processing and model development, facilitating accurate and accessible diabetic retinopathy diagnosis.
Getting Started
Introduction: Diabetic retinopathy (DR) remains a significant global health concern, with early detection playing a critical role in preventing vision loss. For those eager to contribute to this vital area of research, a comprehensive project studio is readily available. This studio has already tackled many essential tasks involved in DR detection, providing researchers and enthusiasts with a ready-to-use platform for experimentation.
Get Started with the Project Studio: Researchers and enthusiasts alike can access the necessary tools and resources by duplicating this project studio. This streamlined solution offers an immediate starting point for experimentation on the Diabetic Retinopathy Dataset.
What the Studio Offers:
- Efficient Handling of Large Datasets: The studio automates the management of large datasets, including downloading, extracting, and data preparation.
- Advanced Model Development: Utilizing PyTorch Lightning, the studio facilitates the development of a sophisticated system for DR detection, categorizing images into different disease stages.
- Integration of Pretrained Backbone Models: Various pretrained backbone models are integrated into the pipeline, allowing for experimentation with different architectures.
- Progress Tracking with TensorBoard: Researchers can monitor progress seamlessly with TensorBoard integration, tracking metrics and visualizing model performance.
- User-Friendly Web Application: A user-friendly web application is provided for showcasing model capabilities and sharing findings effortlessly.
Here's a more structured and standardized version of the steps in a blog format:
Downloading and Preprocessing Diabetic Retinopathy Dataset:
Note: You can skip this entire step, as this studio already has it done for you.
In this step, we'll walk through the process of downloading and preprocessing the Diabetic Retinopathy Detection dataset. This dataset is commonly used for developing algorithms to identify diabetic retinopathy in eye images.
Prerequisites
Before we begin, ensure you have the following prerequisites:
- Kaggle API key (Get one here)
kagglelibrary installed (pip install kaggle)
Note: Before proceeding with the steps below, make sure to change your current directory to dr-detection and install the required dependencies by running the following commands:
cd dr-detection
pip install -r requirements.txt
Step 1: Download the Dataset
There are two ways to download the dataset:
First Way: Downloading as a Complete Zip File
kaggle competitions download -c diabetic-retinopathy-detection
# Extract
unzip diabetic-retinopathy-detection.zip -d data/diabetic-retinopathy-detection
rm diabetic-retinopathy-detection.zip
Second Way: Downloading as Parts
./scripts/download-dr-dataset.sh
# Merge and extract the parts
./scripts/merge_and_extract.sh
Step 2: Preprocess Images
Once the dataset is downloaded, preprocess the images to crop and resize them.
Example of cropping and resizing
python scripts/crop_and_resize.py --src data/diabetic-retinopathy-dataset/train data/diabetic-retinopathy-dataset/resized/train
python scripts/crop_and_resize.py --src data/diabetic-retinopathy-dataset/test data/diabetic-retinopathy-dataset/resized/test
Step 3: Split Data and Save to CSV
Finally, split the data into train and validation sets and save them to CSV files.
python scripts/split_dataset.py
Training Model and Monitoring Progress with TensorBoard
In the previous section, we covered how to set up your dataset and configure your training pipeline using a Config class. Now, let's dive into training your model and monitoring its progress using TensorBoard.
Exploring Data Transformations and Augmentations
If you're looking for examples of data transformations and augmentations, you can explore the provided notebook.ipynb file. This notebook contains various examples of data preprocessing techniques, such as resizing, cropping, rotation, and more.
To open and explore the notebook:
- Navigate to the directory containing the
notebook.ipynbfile. - Open the notebook and run the cells to see different transformation and augmentation examples.
Training the Model
To train your model, you can use the provided train.py script. Make sure you have set up your environment correctly and installed all dependencies as mentioned earlier. Here's how you can run the training pipeline:
- Open your terminal or command prompt.
- Navigate to the directory containing the
train.pyscript. - Run the following command:
python train.py
This command will execute the training script and start training your model based on the parameters specified in your Config class.
Monitoring Training Progress with TensorBoard
TensorBoard is a powerful tool for visualizing and monitoring the training process. You can use it to track metrics such as loss, accuracy, and learning rate over time, as well as visualize model graphs and embeddings.
To load TensorBoard logs and monitor your training progress:
- Ensure you have TensorBoard installed. You can install it via pip:
pip install tensorboard
- Once your model starts training, TensorBoard logs will be generated in the specified directory (e.g.,
"logs/"). You can launch TensorBoard using the following command:
tensorboard --logdir=logs/
This command will start a TensorBoard server locally, allowing you to view your training metrics and visualizations in your web browser.
Gradio - Diabetic Retinopathy Detection App
Overview
Welcome to Diabetic Retinopathy Detection App! This app utilizes deep learning models to detect diabetic retinopathy in retinal images. Diabetic retinopathy is a common complication of diabetes and early detection is crucial for effective treatment.
Try It Out
Use the interactive interface below to upload retinal images and get predictions on diabetic retinopathy severity.
Open Diabetic Retinopathy Detection App
How to Use
- Click on the "Open Diabetic Retinopathy Detection App" button above.
- Upload a retinal image by clicking on the "Upload Image" button.
- Once the image is uploaded, the model will process it and provide predictions on the severity of diabetic retinopathy.
- Interpret the results provided by the model.