|
--- |
|
license: apache-2.0 |
|
datasets: |
|
- imagenet-1k |
|
pipeline_tag: image-classification |
|
tags: |
|
- onnx |
|
- ENOT-AutoDL |
|
--- |
|
|
|
# ENOT-AutoDL pruning benchmark on ImageNet-1k |
|
|
|
This repository contains models accelerated with [ENOT-AutoDL](https://pypi.org/project/enot-autodl/) framework. |
|
Models from [Torchvision](https://pytorch.org/vision/stable/models.html) are used as a baseline. |
|
Evaluation code is also based on Torchvision references. |
|
|
|
## ResNet-50 |
|
|
|
| Model | Latency (MMACs) | Accuracy (%) | |
|
|---------------------------|:---------------:|:-------------:| |
|
| **ResNet-50 Torchvision** | 4144.85 | 76.14 | |
|
| **ResNet-50 ENOT (x2)** | 2057.61 (x2.01) | 75.48 (-0.66) | |
|
| **ResNet-50 ENOT (x4)** | 867.94 (x4.77) | 73.58 (-2.57) | |
|
|
|
## ViT-B/32 |
|
|
|
| Model | Latency (MMACs) | Accuracy (%) | |
|
|--------------------------|:---------------:|:-------------:| |
|
| **ViT-B/32 Torchvision** | 4413.99 | 75.91 | |
|
| **ViT-B/32 ENOT (x4.8)** | 911.80 (x4.84) | 75.68 (-0.23) | |
|
| **ViT-B/32 ENOT (x9)** | 490.78 (x8.99) | 73.72 (-2.19) | |
|
|
|
## MobileNetV2 |
|
|
|
| Model | Latency (MMACs) | Accuracy (%) | |
|
|-----------------------------|:---------------:|:-------------:| |
|
| **MobileNetV2 Torchvision** | 334.23 | 71.88 | |
|
| **MobileNetV2 ENOT (x1.6)** | 209.24 (x1.6) | 71.38 (-0.5) | |
|
| **MobileNetV2 ENOT (x2.1)** | 156.80 (x2.13) | 69.90 (-1.98) | |
|
|
|
# Validation |
|
|
|
To validate results, follow this steps: |
|
|
|
1. Install all required packages: |
|
```bash |
|
pip install -r requrements.txt |
|
``` |
|
1. Calculate model latency: |
|
```bash |
|
python measure_mac.py --model-ckpt path/to/model.pth |
|
``` |
|
1. Measure accuracy of ONNX model: |
|
```bash |
|
python test.py --data-path path/to/imagenet --model-onnx path/to/model.onnx --batch-size 1 |
|
``` |
|
1. Measure accuracy of PyTorch (.pth) model: |
|
```bash |
|
python test.py --data-path path/to/imagenet --model-ckpt path/to/model.pth |
|
``` |
|
|
|
If you want to book a demo, please contact us: enot@enot.ai . |
|
|