metadata
license: apache-2.0
datasets:
- imagenet-1k
library_name: torchvision
pipeline_tag: image-classification
tags:
- onnx
- ENOT-AutoDL
ENOT-AutoDL pruning benchmark on ImageNet-1k
This repository contains models accelerated with ENOT-AutoDL framework. Models from Torchvision are used as a baseline. Evaluation code is also based on Torchvision references.
ResNet-50
| Model | Latency (MMACs) | Accuracy (%) |
|---|---|---|
| ResNet-50 Torchvision | 4144.85 | 76.14 |
| ResNet-50 ENOT (x2) | 2057.61 (x2.01) | 75.48 (-0.66) |
| ResNet-50 ENOT (x4) | 867.94 (x4.77) | 73.58 (-2.57) |
ViT-B/32
| Model | Latency (MMACs) | Accuracy (%) |
|---|---|---|
| ViT-B/32 Torchvision | 4413.99 | 75.91 |
| ViT-B/32 ENOT (x4.8) | 911.80 (x4.84) | 75.68 (-0.23) |
| ViT-B/32 ENOT (x9) | 490.78 (x8.99) | 73.72 (-2.19) |
MobileNetV2
| Model | Latency (MMACs) | Accuracy (%) |
|---|---|---|
| MobileNetV2 Torchvision | 334.23 | 71.88 |
| MobileNetV2 ENOT (x1.6) | 209.24 (x1.6) | 71.38 (-0.5) |
| MobileNetV2 ENOT (x2.1) | 156.80 (x2.13) | 69.90 (-1.98) |
Validation
To validate results, follow this steps:
- Install all required packages:
pip install -r requrements.txt - Calculate model latency:
python measure_mac.py --model-ckpt path/to/model.pth - Measure accuracy of ONNX model:
python test.py --data-path path/to/imagenet --model-onnx path/to/model.onnx --batch-size 1 - Measure accuracy of PyTorch (.pth) model:
python test.py --data-path path/to/imagenet --model-ckpt path/to/model.pth
If you want to book a demo, please contact us: enot@enot.ai .