Push model to huggingface

1_Pooling/config.json

@@ -1,5 +1,5 @@
 {
-  "word_embedding_dimension": 384,
+  "word_embedding_dimension": 768,
   "pooling_mode_cls_token": false,
   "pooling_mode_mean_tokens": true,
   "pooling_mode_max_tokens": false,

README.md

@@ -1,26 +1,16 @@
 ---
-language: en
-license: mit
-library_name: sentence-transformers
+pipeline_tag: sentence-similarity
 tags:
 - sentence-transformers
+- feature-extraction
 - sentence-similarity
-- task-oriented-dialogues
-- dialog-flow
-datasets:
-- Salesforce/dialogstudio
-pipeline_tag: sentence-similarity
-base_model:
-- aws-ai/dse-bert-base
----
+- transformers
 
+---
 
-# Dialog2Flow single target (DSE-base)
-
-This a variation of the **D2F$_{single}$** model introduced in the paper ["Dialog2Flow: Pre-training Soft-Contrastive Action-Driven Sentence Embeddings for Automatic Dialog Flow Extraction"](https://publications.idiap.ch/attachments/papers/2024/Burdisso_EMNLP2024_2024.pdf) published in the EMNLP 2024 main conference.
-This version uses DSE-base as the backbone model which yields to an increase in performance as compared to the vanilla version using BERT-base as the backbone (results reported in Appendix C).
+# sergioburdisso/dialog2flow-single-dse-base
 
-Implementation-wise, this is a [sentence-transformers](https://www.SBERT.net) model: It maps sentences & paragraphs to a 768 dimensional dense vector space and can be used for tasks like clustering or search.
+This is a [sentence-transformers](https://www.SBERT.net) model: It maps sentences & paragraphs to a 768 dimensional dense vector space and can be used for tasks like clustering or semantic search.
 
 <!--- Describe your model here -->
 
@@ -36,7 +26,7 @@ Then you can use the model like this:
 
 ```python
 from sentence_transformers import SentenceTransformer
-sentences = ["your phone please", "okay may i have your telephone number please"]
+sentences = ["This is an example sentence", "Each sentence is converted"]
 
 model = SentenceTransformer('sergioburdisso/dialog2flow-single-dse-base')
 embeddings = model.encode(sentences)
@@ -61,7 +51,7 @@ def mean_pooling(model_output, attention_mask):
 
 
 # Sentences we want sentence embeddings for
-sentences = ['your phone please', 'okay may i have your telephone number please']
+sentences = ['This is an example sentence', 'Each sentence is converted']
 
 # Load model from HuggingFace Hub
 tokenizer = AutoTokenizer.from_pretrained('sergioburdisso/dialog2flow-single-dse-base')
@@ -81,23 +71,21 @@ print("Sentence embeddings:")
 print(sentence_embeddings)
 ```
 
-## Training
-The model was trained with the parameters:
 
-**DataLoader**:
 
-`torch.utils.data.dataloader.DataLoader` of length 363506 with parameters:
-```
-{'batch_size': 64, 'sampler': 'torch.utils.data.sampler.RandomSampler', 'batch_sampler': 'torch.utils.data.sampler.BatchSampler'}
-```
+## Evaluation Results
 
-**Loss**:
+<!--- Describe how your model was evaluated -->
 
-`spretrainer.losses.LabeledContrastiveLoss.LabeledContrastiveLoss` 
+For an automated evaluation of this model, see the *Sentence Embeddings Benchmark*: [https://seb.sbert.net](https://seb.sbert.net?model_name=sergioburdisso/dialog2flow-single-dse-base)
+
+
+## Training
+The model was trained with the parameters:
 
 **DataLoader**:
 
-`torch.utils.data.dataloader.DataLoader` of length 49478 with parameters:
+`torch.utils.data.dataloader.DataLoader` of length 24615 with parameters:
 ```
 {'batch_size': 64, 'sampler': 'torch.utils.data.sampler.RandomSampler', 'batch_sampler': 'torch.utils.data.sampler.BatchSampler'}
 ```
@@ -110,7 +98,7 @@ Parameters of the fit()-Method:
 ```
 {
     "epochs": 15,
-    "evaluation_steps": 164,
+    "evaluation_steps": 246,
     "evaluator": [
         "spretrainer.evaluation.FewShotClassificationEvaluator.FewShotClassificationEvaluator"
     ],
@@ -136,22 +124,4 @@ SentenceTransformer(
 
 ## Citing & Authors
 
-
-```bibtex
-@inproceedings{burdisso-etal-2024-dialog2flow,
-    title = "Dialog2Flow: Pre-training Soft-Contrastive Action-Driven Sentence Embeddings for Automatic Dialog Flow Extraction",
-    author = "Burdisso, Sergio  and
-      Madikeri, Srikanth  and
-      Motlicek, Petr",
-    booktitle = "Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing",
-    month = nov,
-    year = "2024",
-    address = "Miami",
-    publisher = "Association for Computational Linguistics",
-}
-```
-
-## License
-
-Copyright (c) 2024 [Idiap Research Institute](https://www.idiap.ch/).
-MIT License.
+<!--- Describe where people can find more information -->

config.json

@@ -1,5 +1,5 @@
 {
-  "_name_or_path": "sentence-transformers/all-MiniLM-L6-v2",
+  "_name_or_path": "/idiap/temp/sburdisso/repos/jsalt/keya-dialog/outputs/tod_all/dse-bert-base/soft-labels/label_multi-qa-mpnet-base-dot-v1_t0.35/msl64_pm-mean/ch-on_t0.05/lr3e-06_bs64_e15/best_model_metric_0",
   "architectures": [
     "BertModel"
   ],
@@ -8,14 +8,14 @@
   "gradient_checkpointing": false,
   "hidden_act": "gelu",
   "hidden_dropout_prob": 0.1,
-  "hidden_size": 384,
+  "hidden_size": 768,
   "initializer_range": 0.02,
-  "intermediate_size": 1536,
+  "intermediate_size": 3072,
   "layer_norm_eps": 1e-12,
   "max_position_embeddings": 512,
   "model_type": "bert",
   "num_attention_heads": 12,
-  "num_hidden_layers": 6,
+  "num_hidden_layers": 12,
   "pad_token_id": 0,
   "position_embedding_type": "absolute",
   "torch_dtype": "float32",

config_sentence_transformers.json

@@ -1,7 +1,7 @@
 {
   "__version__": {
-    "sentence_transformers": "2.0.0",
-    "transformers": "4.6.1",
-    "pytorch": "1.8.1"
+    "sentence_transformers": "2.2.2",
+    "transformers": "4.30.2",
+    "pytorch": "2.0.1"
   }
 }

model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:1377e9af0ca0b016a9f2aa584d6fc71ab3ea6804fae21ef9fb1416e2944057ac
-size 90864192
+oid sha256:c6ab50d69e51a41345e8595dbd36fe0c4706e89472b59b675b9656ce2620382c
+size 437951328

modules.json

@@ -10,11 +10,5 @@
     "name": "1",
     "path": "1_Pooling",
     "type": "sentence_transformers.models.Pooling"
-  },
-  {
-    "idx": 2,
-    "name": "2",
-    "path": "2_Normalize",
-    "type": "sentence_transformers.models.Normalize"
   }
 ]

sentence_bert_config.json

@@ -1,4 +1,4 @@
 {
-  "max_seq_length": 256,
+  "max_seq_length": 64,
   "do_lower_case": false
 }

tokenizer.json

@@ -2,14 +2,12 @@
   "version": "1.0",
   "truncation": {
     "direction": "Right",
-    "max_length": 128,
+    "max_length": 64,
     "strategy": "LongestFirst",
     "stride": 0
   },
   "padding": {
-    "strategy": {
-      "Fixed": 128
-    },
+    "strategy": "BatchLongest",
     "direction": "Right",
     "pad_to_multiple_of": null,
     "pad_id": 0,

tokenizer_config.json

@@ -43,12 +43,10 @@
   },
   "clean_up_tokenization_spaces": true,
   "cls_token": "[CLS]",
-  "do_basic_tokenize": true,
   "do_lower_case": true,
   "mask_token": "[MASK]",
-  "max_length": 128,
+  "max_length": 64,
   "model_max_length": 512,
-  "never_split": null,
   "pad_to_multiple_of": null,
   "pad_token": "[PAD]",
   "pad_token_type_id": 0,