Hugging Face's logo

conservationxlabs
/
miewid-msv2

Feature Extraction
Transformers
Safetensors
miewid
custom_code
Model card Files Community
miewid-msv2 / README.md
lashao's picture
lashao
Update README.md
74201a3 verified
preview code
|
raw
history blame contribute delete
19.1 kB
metadata
library_name: transformers
tags: []

Model Details

Model Description

MiewID-msv2 is a feature extractor trained for re-identification using contrastive learning on a large, high-quality dataset of 54 wildlife species - terrestrial and aquatic - including fins, flukes, flanks, faces.

  • Model Type: Wildlife re-identification feature backbone
  • Model Stats:
    • Params (M): 51.11
    • GMACs: 24.38
    • Activations (M): 91.11
    • Image size: 440 x 440

Model Sources [optional]

Usage

Intended use is re-identification of individuals from different species by matching against a database of grount-truth samples. Model features can also be used for species classification by retrieval.

Embedding Extraction 

import numpy as np
from PIL import Image
import torch
import torchvision.transforms as transforms
from transformers import AutoModel

model_tag = f"conservationxlabs/miewid-msv2"
model = AutoModel.from_pretrained(model_tag, trust_remote_code=True)

def generate_random_image(height=440, width=440, channels=3):
    random_image = np.random.randint(0, 256, (height, width, channels), dtype=np.uint8)
    return Image.fromarray(random_image)

random_image = generate_random_image()

preprocess = transforms.Compose([
    transforms.Resize((440, 440)),
    transforms.ToTensor(),
    transforms.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225]),
])

input_tensor = preprocess(random_image)
input_batch = input_tensor.unsqueeze(0) 

with torch.no_grad():
    output = model(input_batch)

print(output)
print(output.shape)

More Examples

View more usage examples at https://github.com/WildMeOrg/wbia-plugin-miew-id/tree/main/wbia_miew_id/examples

Training Details

Training Data

The dataset used for these experiments was a combination of data from Wildbook platforms (multiple users), Happywhale Kaggle competitions multi-species dataset and multiple publicly available datasets. A small subset of data from Wildbook platforms is available at https://lila.science/datasets.

Example images

image/jpg

Evaluation Results

The multi-species model shows improvements over models trained on single species. Moreover the model shows strong generalization ability for majority of the species when trained and evaluated in a leave-one-out fashion.

Detailed results

Source group map rank-1 rank-5 rank-10 rank-20 viewpoints n_test_annots n_test_names n_train_annots n_train_names
Wildbook amur_tiger 91.8 100.0 100.0 100.0 100.0 ['left', 'right'] 233 47 691 75
Wildbook beluga_whale 61.36 71.85 81.63 85.51 89.28 ['up'] 849 228 2767 354
Happywhale blue_whale 36.98 38.68 55.11 61.52 69.04 ['unknown'] 998 392 1185 339
Wildbook bottlenose_dolphin 85.69 93.99 96.18 96.85 97.52 ['right', 'left'] 4911 838 11607 1062
Happywhale brydes_whale 69.96 80.95 92.86 95.24 100.0 ['unknown'] 42 9 81 15
Lomas Capuchin capuchin 34.8 47.97 68.58 76.69 85.14 ['front'] 296 35 1708 29
Wildbook cheetah 57.37 71.15 84.92 87.87 92.13 ['left', 'right'] 624 139 1522 144
Wildbook chimpanzee 94.7 100.0 100.0 100.0 100.0 ['unknown'] 52 8 138 9
C-Tai chimpanzee_ctai 51.67 69.64 83.68 89.18 94.12 ['front'] 527 63 2680 44
C-Zoo chimpanzee_czoo 77.66 86.67 93.33 97.5 99.17 ['front'] 240 24 1258 17
Happywhale commersons_dolphin 0.0 0.0 0.0 0.0 0.0 ['unknown'] 10 5 7 2
Happywhale cuviers_beaked_whale 52.89 55.71 75.71 82.86 90.0 ['unknown'] 70 28 89 21
DogFaceNet dog 82.96 87.38 94.78 96.74 97.99 ['unknown'] 1838 544 5750 1088
Happywhale dusky_dolphin 85.82 83.54 93.04 94.3 94.94 ['unknown'] 158 68 154 41
Wildbook eurasianlynx 58.8 67.82 83.78 88.83 92.02 ['left', 'right'] 726 199 2251 285
Happywhale fin_whale 67.65 76.92 87.18 90.11 93.04 ['unknown'] 273 71 533 55
Happywhale frasiers_dolphin 0.0 0.0 0.0 0.0 0.0 -1 -1 -1 -1
Wildbook giraffe 99.18 99.08 99.08 100.0 100.0 ['right', 'front', 'left', 'back'] 423 162 2540 426
Wildbook horse_wild+face 0.0 0.0 0.0 0.0 0.0 ['front'] 17 7 174 53
Wildbook horse_wild_tunisian+face 79.59 98.53 99.51 100.0 100.0 ['unknown'] 204 39 999 37
Wildbook hyena 64.17 79.32 89.22 93.14 96.41 ['right', 'left'] 1231 301 1154 201
Wildbook hyperoodon_ampullatus 91.14 96.11 98.12 98.34 98.84 ['left', 'right'] 1930 354 24952 759
Wildbook jaguar 67.45 81.72 91.04 92.91 94.4 ['left', 'right'] 540 124 980 103
PrimFace japanese_monkey 89.05 93.85 98.46 98.46 100.0 ['unknown'] 65 12 219 14
Happywhale killer_whale 62.82 77.13 85.48 88.81 91.82 ['unknown'] 599 145 1379 177
Wildbook leopard 65.03 82.64 90.23 92.83 94.46 ['right', 'left'] 595 108 4261 381
Wildbook lion 87.12 93.86 100.0 100.0 100.0 ['front', 'unknown', 'frontright', 'frontleft', 'upfront', 'right', 'upleft', 'left', 'upright', 'backleft', 'back'] 1364 242 3981 241
Happywhale long_finned_pilot_whale 99.06 98.11 100.0 100.0 100.0 ['unknown'] 53 21 33 10
Wildbook lynx_pardinus 48.22 57.68 73.44 77.18 84.96 ['left', 'right'] 467 142 1376 259
MacaqueFaces macaque_face 89.87 95.88 97.94 99.12 100.0 ['front'] 340 34 3480 25
Happywhale melon_headed_whale 90.8 93.42 96.05 96.71 98.68 ['unknown'] 152 55 180 43
Wildbook nyala 64.64 70.65 90.05 94.03 97.01 ['left', 'right'] 418 99 1303 149
Happywhale pantropic_spotted_dolphin 78.15 85.19 90.74 94.44 96.3 ['unknown'] 54 11 55 8
Happywhale pygmy_killer_whale 89.07 84.62 100.0 100.0 100.0 ['unknown'] 26 10 32 5
PrimFace rhesus_monkey 78.12 89.13 97.28 98.91 100.0 ['unknown'] 184 28 478 24
Happywhale rough_toothed_dolphin 0.0 0.0 0.0 0.0 0.0 ['unknown'] 6 3 7 2
SealID seal 37.93 59.51 80.0 86.1 91.71 ['unknown'] 410 41 1448 36
Happywhale sei_whale 67.86 71.55 87.07 93.1 95.69 ['unknown'] 116 35 106 29
Wildbook short_fin_pilot_whale+fin_dorsal 92.49 92.31 98.08 100.0 100.0 ['right', 'left'] 272 64 1290 137
Happywhale short_finned_pilot_whale 94.39 94.96 95.8 95.8 97.48 ['unknown'] 119 57 67 17
Wildbook snow_leopard 62.43 82.45 88.91 92.15 94.53 ['left', 'right'] 927 144 2307 99
Happywhale spinner_dolphin 99.88 100.0 100.0 100.0 100.0 ['left', 'right'] 335 97 1562 251
Happywhale spotted_dolphin 84.92 83.67 89.8 95.92 97.96 ['unknown'] 49 22 128 28
Wildbook turtle_green 100.0 100.0 100.0 100.0 100.0 ['left', 'right', 'up', 'front'] 543 109 5097 76
Wildbook turtle_green+head 95.1 96.02 100.0 100.0 100.0 ['left', 'right', 'up', 'front'] 544 108 5737 76
Wildbook turtle_hawksbill 78.84 81.15 88.37 92.56 95.81 ['up', 'left', 'right', 'front'] 506 106 5397 71
Wildbook turtle_hawksbill+head 86.78 91.74 95.45 96.69 98.35 ['right', 'left', 'up', 'front'] 589 109 7033 71
SeaTurtleID turtle_loggerhead_ext 63.67 75.43 86.61 90.17 92.87 ['unknown', 'left', 'top', 'right'] 815 217 6573 355
SeaTurtleID turtle_loggerhead_ext+head 94.14 94.63 100.0 100.0 100.0 ['left', 'right', 'topright', 'top', 'topleft', 'below', 'front'] 778 209 6361 348
Wildbook whale_fin+fin_dorsal 82.12 85.12 93.87 95.04 95.75 ['left', 'right'] 394 110 1629 186
Wildbook whale_grey 90.97 91.55 95.54 96.18 96.82 ['right', 'left'] 381 103 5662 758
Wildbook whale_humpback+fin_dorsal 80.3 84.07 90.08 91.6 95.42 ['right', 'left'] 509 194 3734 541
Wildbook whale_humpback+fluke 97.11 96.67 100.0 100.0 100.0 ['back'] 150 64 1151 356
Wildbook whale_orca 97.01 98.36 99.18 100.0 100.0 ['right', 'left'] 249 49 2790 435
Wildbook whale_orca+fin_dorsal 95.41 95.87 97.52 97.52 99.17 ['right', 'left'] 623 187 3978 430
Wildbook whale_sperm+fluke 96.14 97.87 98.33 98.48 99.09 ['back'] 658 125 25849 558
Wildbook whaleshark 57.61 72.25 79.96 82.27 87.21 ['left', 'right'] 997 212 5713 376
Wildbook white_shark+fin_dorsal 91.4 94.89 97.87 98.54 99.27 ['left', 'right'] 322 88 1067 162
Happywhale white_sided_dolphin 81.82 76.0 88.0 88.0 100.0 ['unknown'] 25 11 13 4
Wildbook wilddog 73.4 85.11 90.93 92.38 94.07 ['left', 'right'] 3460 848 4217 656
Wildbook zebra_grevys 91.79 96.79 97.57 97.74 98.04 ['right'] 1684 332 8927 362