|
--- |
|
title: EVAv1 S12 |
|
emoji: 🏢 |
|
colorFrom: blue |
|
colorTo: blue |
|
sdk: gradio |
|
sdk_version: 3.39.0 |
|
app_file: app.py |
|
pinned: false |
|
license: mit |
|
--- |
|
|
|
|
|
# S12 |
|
|
|
# CIFAR10 Image Classification with PyTorch Lightning |
|
|
|
This project implements an image classifier trained on the CIFAR10 dataset using PyTorch Lightning. The project aims to showcase the use of ResNet architecture, data augmentation, custom dataset classes, and learning rate schedulers. |
|
|
|
## Project Structure |
|
|
|
The project is structured as follows: |
|
|
|
1. Data loading and preprocessing |
|
2. Dataset statistics calculation |
|
3. Data Augmentation |
|
3. Model creation |
|
4. Training and evaluation |
|
|
|
### Data Loading and Preprocessing |
|
|
|
The data for this project is the CIFAR10 dataset, which is loaded using PyTorch's built-in datasets. To ensure that our model generalizes well, we apply several data augmentations to our training set including normalization, padding, random cropping, and horizontal flipping. |
|
|
|
### Dataset Statistics Calculation |
|
|
|
Before we start training our model, we calculate per-channel mean and standard deviation for our dataset. These statistics are used to normalize our data, which helps make our training process more stable. |
|
|
|
``` |
|
Dataset Mean - [0.49139968 0.48215841 0.44653091] |
|
Dataset Std - [0.24703223 0.24348513 0.26158784] |
|
``` |
|
|
|
### Dataset Augmentation |
|
```python |
|
def get_transforms(means, stds): |
|
train_transforms = A.Compose( |
|
[ |
|
A.Normalize(mean=means, std=stds, always_apply=True), |
|
A.RandomCrop(height=32, width=32, pad=4, always_apply=True), |
|
A.HorizontalFlip(), |
|
A.Cutout (fill_value=means), |
|
ToTensorV2(), |
|
] |
|
) |
|
|
|
test_transforms = A.Compose( |
|
[ |
|
A.Normalize(mean=means, std=stds, always_apply=True), |
|
ToTensorV2(), |
|
] |
|
) |
|
|
|
return(train_transforms, test_transforms) |
|
``` |
|
 |
|
|
|
|
|
### Model Creation |
|
|
|
The model we use for this project is a Custom ResNet, a type of convolutional neural network known for its high performance on image classification tasks. |
|
|
|
``` |
|
| Name | Type | Params |
|
--------------------------------------------------- |
|
0 | criterion | CrossEntropyLoss | 0 |
|
1 | accuracy | MulticlassAccuracy | 0 |
|
2 | prep_layer | Sequential | 1.9 K |
|
3 | layer_one | Sequential | 74.0 K |
|
4 | res_block1 | ResBlock | 295 K |
|
5 | layer_two | Sequential | 295 K |
|
6 | layer_three | Sequential | 1.2 M |
|
7 | res_block2 | ResBlock | 4.7 M |
|
8 | max_pool | MaxPool2d | 0 |
|
9 | fc | Linear | 5.1 K |
|
--------------------------------------------------- |
|
6.6 M Trainable params |
|
0 Non-trainable params |
|
6.6 M Total params |
|
26.292 Total estimated model params size (MB) |
|
``` |
|
|
|
#### ResNet Architecture and Residual Blocks |
|
|
|
The defining feature of the ResNet architecture is its use of residual blocks and skip connections. Each residual block consists of a series of convolutional layers followed by a skip connection that adds the input of the block to its output. These connections allow the model to learn identity functions, making it easier for the network to learn complex patterns. This characteristic is particularly beneficial in deeper networks, as it helps to alleviate the problem of vanishing gradients. |
|
|
|
### Training and Evaluation |
|
|
|
To train our model, we use the Adam optimizer with a OneCycle learning rate scheduler. |
|
|
|
``` |
|
Epoch 23: 100% |
|
196/196 [00:27<00:00, 7.11it/s, v_num=0, val_loss=0.639, val_acc=0.776, train_loss=0.686, train_acc=0.762] |
|
|
|
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━┓ |
|
┃ Test metric ┃ DataLoader 0 ┃ |
|
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━┩ |
|
│ test_acc │ 0.8758000135421753 │ |
|
│ test_loss │ 0.39947837591171265 │ |
|
|
|
``` |
|
|
|
#### OneCycle Learning Rate Scheduler |
|
|
|
The OneCycle learning rate scheduler varies the learning rate between a minimum and maximum value according to a certain policy. This dynamic learning rate can help improve the performance of our model. We train our model for a total of 24 epochs. |
|
|
|
### Learning Rate Finder |
|
|
|
```python |
|
def LR_Finder(model, criterion, optimizer, trainloader): |
|
|
|
lr_finder = LRFinder(model, optimizer, criterion, device="cuda") |
|
lr_finder.range_test(trainloader, end_lr=10, num_iter=200, step_mode='exp') |
|
max_lr = lr_finder.plot(suggest_lr=True, skip_start=0, skip_end=0) |
|
lr_finder.reset() |
|
|
|
return(max_lr[1]) |
|
``` |
|
|
|
 |
|
|
|
|
|
## Dependencies |
|
|
|
This project requires the following dependencies: |
|
|
|
- torch |
|
- torchvision |
|
- numpy |
|
- albumentations |
|
- matplotlib |
|
- torchsummary |
|
|
|
|
|
## Usage |
|
|
|
To run this project, you can clone the repository and run the main script: |
|
|
|
```bash |
|
git clone https://github.com/Delve-ERAV1/S12.git |
|
cd S12 |
|
gradio app.py |
|
``` |
|
|
|
### Upload New Image |
|
 |
|
|
|
### View Misclassified Images |
|
|
|
 |
|
|
|
## Results |
|
|
|
 |
|
|
|
## References |
|
|
|
Deep Residual Learning for Image Recognition Kaiming He et al |
|
Super-Convergence: Very Fast Training of Neural Networks Using Large Learning Rates Leslie N. Smith |
