Trick or ResNet Treat
•
3
Ross Wightman
rwightman
AI & ML interests
Computer vision, transfer learning, semi/self supervised learning, robotics.
Recent Activity
New activity
about 9 hours ago
laion/CLIP-ViT-B-32-xlm-roberta-base-laion5B-s13B-b90k
updated
a collection
about 20 hours ago
All the ImageNets
posted
an
update
1 day ago
Articles
Organizations
Posts
10
Post
581
Want to validate some hparams or figure out what
I had this sitting on my drive and forgot where I pulled it together from. It's 100 classes of imagenet, 50k train and 10k val images (from ImageNet-1k train set), and 5k test images (from ImageNet-1k val set). 7.4GB instead of > 100GB for the full ImageNet-1k. This ver is not reduced resolution like some other 'mini' versions. Super easy to use with timm train/val scripts, checkout the dataset card.
I often check fine-tuning with even smaller datasets like:
* timm/resisc45
* timm/oxford-iiit-pet
But those are a bit small to train any modest size model w/o starting from pretrained weights.
timm model to use before commiting to download or training with a large dataset? Try mini-imagenet:
timm/mini-imagenet I had this sitting on my drive and forgot where I pulled it together from. It's 100 classes of imagenet, 50k train and 10k val images (from ImageNet-1k train set), and 5k test images (from ImageNet-1k val set). 7.4GB instead of > 100GB for the full ImageNet-1k. This ver is not reduced resolution like some other 'mini' versions. Super easy to use with timm train/val scripts, checkout the dataset card.
I often check fine-tuning with even smaller datasets like:
* timm/resisc45
* timm/oxford-iiit-pet
But those are a bit small to train any modest size model w/o starting from pretrained weights.
Post
1528
New MobileNetV4 weights were uploaded a few days ago -- more ImageNet-12k training at 384x384 for the speedy 'Conv Medium' models.
There are 3 weight variants here for those who like to tinker. On my hold-out eval they are ordered as below, not that different, but the Adopt 180 epochs closer to AdamW 250 than to AdamW 180.
* AdamW for 250 epochs - timm/mobilenetv4_conv_medium.e250_r384_in12k
* Adopt for 180 epochs - timm/mobilenetv4_conv_medium.e180_ad_r384_in12k
* AdamW for 180 epochs - timm/mobilenetv4_conv_medium.e180_r384_in12k
This was by request as a user reported impressive results using the 'Conv Large' ImagNet-12k pretrains as object detection backbones. ImageNet-1k fine-tunes are pending, the weights do behave differently with the 180 vs 250 epochs and the Adopt vs AdamW optimizer.
There are 3 weight variants here for those who like to tinker. On my hold-out eval they are ordered as below, not that different, but the Adopt 180 epochs closer to AdamW 250 than to AdamW 180.
* AdamW for 250 epochs - timm/mobilenetv4_conv_medium.e250_r384_in12k
* Adopt for 180 epochs - timm/mobilenetv4_conv_medium.e180_ad_r384_in12k
* AdamW for 180 epochs - timm/mobilenetv4_conv_medium.e180_r384_in12k
This was by request as a user reported impressive results using the 'Conv Large' ImagNet-12k pretrains as object detection backbones. ImageNet-1k fine-tunes are pending, the weights do behave differently with the 180 vs 250 epochs and the Adopt vs AdamW optimizer.
Collections
1
Fine-tune experiments for various `timm` models on the INaturalist 2021 Challenge dataset (https://github.com/visipedia/inat_comp/tree/master/2021)
-
timm/eva02_large_patch14_clip_336.merged2b_ft_inat21
Image Classification • Updated • 46 • 11 -
timm/vit_large_patch14_clip_336.datacompxl_ft_augreg_inat21
Image Classification • Updated • 10 • 1 -
timm/vit_large_patch14_clip_336.laion2b_ft_augreg_inat21
Image Classification • Updated • 13 -
timm/vit_large_patch14_clip_336.datacompxl_ft_inat21
Image Classification • Updated • 12 • 1
