Upload MiewIdNet

README.md

@@ -0,0 +1,199 @@
+---
+library_name: transformers
+tags: []
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+This is the model card of a 🤗 transformers model that has been pushed on the Hub. This model card has been automatically generated.
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]

config.json

@@ -0,0 +1,25 @@
+{
+  "architectures": [
+    "MiewIdNet"
+  ],
+  "auto_map": {
+    "AutoConfig": "configuration_miewid.MiewIdNetConfig",
+    "AutoModel": "modeling_miewid.MiewIdNet"
+  },
+  "dropout": 0.0,
+  "fc_dim": 512,
+  "k": null,
+  "loss_module": "softmax",
+  "ls_eps": 0.0,
+  "margin": 0.5,
+  "margins": null,
+  "model_name": "efficientnetv2_rw_m",
+  "model_type": "miewid",
+  "n_classes": 10,
+  "pretrained": true,
+  "s": 30.0,
+  "theta_zero": 0.785,
+  "torch_dtype": "float32",
+  "transformers_version": "4.41.2",
+  "use_fc": false
+}

configuration_miewid.py

@@ -0,0 +1,21 @@
+from transformers import PretrainedConfig
+
+
+class MiewIdNetConfig(PretrainedConfig):
+    model_type = "miewid"
+
+    def __init__(self, n_classes=10, model_name='efficientnet_b0', use_fc=False, fc_dim=512, dropout=0.0, loss_module='softmax', s=30.0, margin=0.50, ls_eps=0.0, theta_zero=0.785, pretrained=True, margins=None, k=None, **kwargs):
+        super().__init__(**kwargs)
+        self.n_classes = n_classes
+        self.model_name = model_name
+        self.use_fc = use_fc
+        self.fc_dim = fc_dim
+        self.dropout = dropout
+        self.loss_module = loss_module
+        self.s = s
+        self.margin = margin
+        self.ls_eps = ls_eps
+        self.theta_zero = theta_zero
+        self.pretrained = pretrained
+        self.margins = margins
+        self.k = k

heads.py

@@ -0,0 +1,161 @@
+import torch
+import torch.nn as nn
+import math
+import torch.nn.functional as F
+from torch.nn.parameter import Parameter
+
+class ArcMarginProduct(nn.Module):
+    r"""Implement of large margin arc distance: :
+        Args:
+            in_features: size of each input sample
+            out_features: size of each output sample
+            s: norm of input feature
+            m: margin
+            cos(theta + m)wandb: ERROR Abnormal program exit
+
+        """
+    def __init__(self, in_features, out_features, s=30.0, m=0.50, easy_margin=False, ls_eps=0.0):
+        super(ArcMarginProduct, self).__init__()
+        self.in_features = in_features
+        self.out_features = out_features
+        self.s = s
+        self.m = m
+        self.ls_eps = ls_eps  # label smoothing
+        self.weight = Parameter(torch.FloatTensor(out_features, in_features))
+        nn.init.xavier_uniform_(self.weight)
+
+        self.easy_margin = easy_margin
+        self.cos_m = math.cos(m)
+        self.sin_m = math.sin(m)
+        self.th = math.cos(math.pi - m)
+        self.mm = math.sin(math.pi - m) * m
+
+    def forward(self, input, label):
+        # --------------------------- cos(theta) & phi(theta) ---------------------------
+        cosine = F.linear(F.normalize(input), F.normalize(self.weight))
+        sine = torch.sqrt(1.0 - torch.pow(cosine, 2))
+        phi = cosine * self.cos_m - sine * self.sin_m
+        if self.easy_margin:
+            phi = torch.where(cosine > 0, phi, cosine)
+        else:
+            phi = torch.where(cosine > self.th, phi, cosine - self.mm)
+        # --------------------------- convert label to one-hot ---------------------------
+        # one_hot = torch.zeros(cosine.size(), requires_grad=True, device='cuda')
+        one_hot = torch.zeros(cosine.size(), device='cuda')
+        one_hot.scatter_(1, label.view(-1, 1).long(), 1)
+        if self.ls_eps > 0:
+            one_hot = (1 - self.ls_eps) * one_hot + self.ls_eps / self.out_features
+        # -------------torch.where(out_i = {x_i if condition_i else y_i) -------------
+        output = (one_hot * phi) + ((1.0 - one_hot) * cosine)
+        output *= self.s
+
+        return output
+    
+
+def l2_norm(input, axis = 1):
+    norm = torch.norm(input, 2, axis, True)
+    output = torch.div(input, norm)
+
+    return output
+class ElasticArcFace(nn.Module):
+    def __init__(self, in_features, out_features, s=64.0, m=0.50,std=0.0125,plus=False, k=None):
+        super(ElasticArcFace, self).__init__()
+        self.in_features = in_features
+        self.out_features = out_features
+        self.s = s
+        self.m = m
+        self.kernel = nn.Parameter(torch.FloatTensor(in_features, out_features))
+        nn.init.normal_(self.kernel, std=0.01)
+        self.std=std
+        self.plus=plus
+    def forward(self, embbedings, label):
+        embbedings = l2_norm(embbedings, axis=1)
+        kernel_norm = l2_norm(self.kernel, axis=0)
+        cos_theta = torch.mm(embbedings, kernel_norm)
+        cos_theta = cos_theta.clamp(-1, 1)  # for numerical stability
+        index = torch.where(label != -1)[0]
+        m_hot = torch.zeros(index.size()[0], cos_theta.size()[1], device=cos_theta.device)
+        margin = torch.normal(mean=self.m, std=self.std, size=label[index, None].size(), device=cos_theta.device) # Fast converge .clamp(self.m-self.std, self.m+self.std)
+        if self.plus:
+            with torch.no_grad():
+                distmat = cos_theta[index, label.view(-1)].detach().clone()
+                _, idicate_cosie = torch.sort(distmat, dim=0, descending=True)
+                margin, _ = torch.sort(margin, dim=0)
+            m_hot.scatter_(1, label[index, None], margin[idicate_cosie])
+        else:
+            m_hot.scatter_(1, label[index, None], margin)
+        cos_theta.acos_()
+        cos_theta[index] += m_hot
+        cos_theta.cos_().mul_(self.s)
+        return cos_theta
+    
+########## Subcenter Arcface with dynamic margin ##########
+
+class ArcMarginProduct_subcenter(nn.Module):
+    def __init__(self, in_features, out_features, k=3):
+        super().__init__()
+        self.weight = nn.Parameter(torch.FloatTensor(out_features*k, in_features))
+        self.reset_parameters()
+        self.k = k
+        self.out_features = out_features
+        
+    def reset_parameters(self):
+        stdv = 1. / math.sqrt(self.weight.size(1))
+        self.weight.data.uniform_(-stdv, stdv)
+        
+    def forward(self, features):
+        cosine_all = F.linear(F.normalize(features), F.normalize(self.weight))
+        cosine_all = cosine_all.view(-1, self.out_features, self.k)
+        cosine, _ = torch.max(cosine_all, dim=2)
+        return cosine
+    
+class ArcFaceLossAdaptiveMargin(nn.modules.Module):
+    def __init__(self, margins, out_dim, s):
+        super().__init__()
+#         self.crit = nn.CrossEntropyLoss()
+        self.s = s
+        self.register_buffer('margins', torch.tensor(margins))
+        self.out_dim = out_dim
+            
+    def forward(self, logits, labels):
+        #ms = []
+        #ms = self.margins[labels.cpu().numpy()]
+        ms = self.margins[labels]
+        cos_m = torch.cos(ms) #torch.from_numpy(np.cos(ms)).float().cuda()
+        sin_m = torch.sin(ms) #torch.from_numpy(np.sin(ms)).float().cuda()
+        th = torch.cos(math.pi - ms)#torch.from_numpy(np.cos(math.pi - ms)).float().cuda()
+        mm = torch.sin(math.pi - ms) * ms#torch.from_numpy(np.sin(math.pi - ms) * ms).float().cuda()
+        labels = F.one_hot(labels, self.out_dim).float()
+        cosine = logits
+        sine = torch.sqrt(1.0 - cosine * cosine)
+        phi = cosine * cos_m.view(-1,1) - sine * sin_m.view(-1,1)
+        phi = torch.where(cosine > th.view(-1,1), phi, cosine - mm.view(-1,1))
+        output = (labels * phi) + ((1.0 - labels) * cosine)
+        output *= self.s
+        return output
+    
+class ArcFaceSubCenterDynamic(nn.Module):
+    def __init__(
+        self,
+        embedding_dim,
+        output_classes,
+        margins,
+        s,
+        k=2,
+    ):
+        super().__init__()
+
+        self.embedding_dim = embedding_dim
+        self.output_classes = output_classes
+        self.margins = margins
+        self.s = s
+        self.wmetric_classify = ArcMarginProduct_subcenter(self.embedding_dim, self.output_classes, k=k)
+        
+        self.warcface_margin = ArcFaceLossAdaptiveMargin(margins=self.margins, 
+                                                         out_dim=self.output_classes, 
+                                                         s=self.s)
+
+    def forward(self, features, labels):
+        logits = self.wmetric_classify(features.float())
+        logits = self.warcface_margin(logits, labels)
+        return logits

model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:34dcc7b6c57694ad0ab7cceb09c8a17cfe7eb7cd44650bd5b58005527fb48a51
+size 205809924

modeling_miewid.py

@@ -0,0 +1,159 @@
+import torch
+import torch.nn as nn
+import torch.nn.functional as F
+
+import timm
+
+from transformers import PreTrainedModel
+
+from .heads import ArcMarginProduct, ElasticArcFace, ArcFaceSubCenterDynamic
+from .configuration_miewid import MiewIdNetConfig
+
+def weights_init_kaiming(m):
+    classname = m.__class__.__name__
+    if classname.find('Linear') != -1:
+        nn.init.kaiming_normal_(m.weight, a=0, mode='fan_out')
+        nn.init.constant_(m.bias, 0.0)
+    elif classname.find('Conv') != -1:
+        nn.init.kaiming_normal_(m.weight, a=0, mode='fan_in')
+        if m.bias is not None:
+            nn.init.constant_(m.bias, 0.0)
+    elif classname.find('BatchNorm') != -1:
+        if m.affine:
+            nn.init.constant_(m.weight, 1.0)
+            nn.init.constant_(m.bias, 0.0)
+
+
+def weights_init_classifier(m):
+    classname = m.__class__.__name__
+    if classname.find('Linear') != -1:
+        nn.init.normal_(m.weight, std=0.001)
+        if m.bias:
+            nn.init.constant_(m.bias, 0.0)
+
+class GeM(nn.Module):
+    def __init__(self, p=3, eps=1e-6):
+        super(GeM, self).__init__()
+        self.p = nn.Parameter(torch.ones(1)*p)
+        self.eps = eps
+
+    def forward(self, x):
+        return self.gem(x, p=self.p, eps=self.eps)
+        
+    def gem(self, x, p=3, eps=1e-6):
+        return F.avg_pool2d(x.clamp(min=eps).pow(p), (x.size(-2), x.size(-1))).pow(1./p)
+        
+    def __repr__(self):
+        return self.__class__.__name__ + \
+                '(' + 'p=' + '{:.4f}'.format(self.p.data.tolist()[0]) + \
+                ', ' + 'eps=' + str(self.eps) + ')'
+
+class MiewIdNet(PreTrainedModel):
+    config_class = MiewIdNetConfig
+    
+    def __init__(self, config):
+        """
+        """
+        super(MiewIdNet, self).__init__(config)
+        print('Building Model Backbone for {} model'.format(config.model_name))
+        print('config.model_name', config.model_name)
+
+        n_classes=config.n_classes
+        model_name=config.model_name
+        use_fc=False
+        fc_dim=512
+        dropout=0.0
+        loss_module=config.loss_module
+        s=30.0
+        margin=0.50
+        ls_eps=0.0
+        theta_zero=0.785
+        pretrained=True
+        margins=config.k
+        k=config.k
+
+        print('model_name', model_name)
+
+        self.backbone = timm.create_model(model_name, pretrained=pretrained, num_classes=0)
+        final_in_features = 2152#self.backbone.classifier.in_features
+
+        print('final_in_features', final_in_features)
+        
+        # self.backbone.classifier = nn.Identity()
+        self.backbone.global_pool =  GeM()#nn.Identity()
+        
+        # self.pooling =  GeM()
+        self.bn = nn.BatchNorm1d(final_in_features)
+        self.use_fc = use_fc
+        if use_fc:
+            self.dropout = nn.Dropout(p=dropout)
+            self.bn = nn.BatchNorm1d(fc_dim)
+            self.bn.bias.requires_grad_(False)
+            self.fc = nn.Linear(final_in_features, n_classes, bias = False)            
+            self.bn.apply(weights_init_kaiming)
+            self.fc.apply(weights_init_classifier)
+            final_in_features = fc_dim
+
+        self.loss_module = loss_module
+        if loss_module == 'arcface':
+            self.final = ElasticArcFace(final_in_features, n_classes,
+                                          s=s, m=margin)
+        elif loss_module == 'arcface_subcenter_dynamic':
+            if margins is None:
+                margins = [0.3] * n_classes
+            print(final_in_features, n_classes)
+            self.final = ArcFaceSubCenterDynamic(
+                embedding_dim=final_in_features, 
+                output_classes=n_classes, 
+                margins=margins,
+                s=s,
+                k=k )
+        # elif loss_module == 'cosface':
+        #     self.final = AddMarginProduct(final_in_features, n_classes, s=s, m=margin)
+        # elif loss_module == 'adacos':
+        #     self.final = AdaCos(final_in_features, n_classes, m=margin, theta_zero=theta_zero)
+        else:
+            self.final = nn.Linear(final_in_features, n_classes)
+
+    def _init_params(self):
+        nn.init.xavier_normal_(self.fc.weight)
+        nn.init.constant_(self.fc.bias, 0)
+        nn.init.constant_(self.bn.weight, 1)
+        nn.init.constant_(self.bn.bias, 0)
+
+    def forward(self, x, label=None):
+        feature = self.extract_feat(x)
+
+        return feature
+        # if not self.training:
+        #     return feature
+        # else:
+        #     assert label is not None
+        # if self.loss_module in ('arcface', 'arcface_subcenter_dynamic'):
+        #     logits = self.final(feature, label)
+        # else:
+        #     logits = self.final(feature)
+
+        # return logits
+
+    def extract_feat(self, x):
+        batch_size = x.shape[0]
+        x = self.backbone(x).view(batch_size, -1)
+        # x = self.pooling(x).view(batch_size, -1)
+        x = self.bn(x)
+        if self.use_fc:
+            x1 = self.dropout(x)
+            x1 = self.bn(x1)
+            x1 = self.fc(x1)
+    
+        return x
+
+    def extract_logits(self, x, label=None):
+        feature = self.extract_feat(x)
+        assert label is not None
+        if self.loss_module in ('arcface', 'arcface_subcenter_dynamic'):
+            logits = self.final(feature, label)
+        else:
+            logits = self.final(feature)
+        
+        return logits