Finetuned model on SNLI

1_Pooling/config.json

@@ -0,0 +1,10 @@
+{
+  "word_embedding_dimension": 384,
+  "pooling_mode_cls_token": false,
+  "pooling_mode_mean_tokens": true,
+  "pooling_mode_max_tokens": false,
+  "pooling_mode_mean_sqrt_len_tokens": false,
+  "pooling_mode_weightedmean_tokens": false,
+  "pooling_mode_lasttoken": false,
+  "include_prompt": true
+}

README.md

@@ -0,0 +1,475 @@
+---
+base_model: sentence-transformers/all-MiniLM-L12-v2
+library_name: sentence-transformers
+metrics:
+- pearson_cosine
+- spearman_cosine
+- pearson_manhattan
+- spearman_manhattan
+- pearson_euclidean
+- spearman_euclidean
+- pearson_dot
+- spearman_dot
+- pearson_max
+- spearman_max
+pipeline_tag: sentence-similarity
+tags:
+- sentence-transformers
+- sentence-similarity
+- feature-extraction
+- generated_from_trainer
+- dataset_size:100000
+- loss:CosineSimilarityLoss
+widget:
+- source_sentence: The church has granite statues of Jesus and the Apostles adorning
+    its porch .
+  sentences:
+  - There were no statues in the church .
+  - L' Afrique du sud et le reste de l' Afrique sont les mêmes .
+  - Tours on foot are a great way to see LA .
+- source_sentence: Au Centre du réseau routier de la région , Alicante est également
+    une base logique pour les automobilistes et pour les liaisons ferroviaires et
+    ferroviaires .
+  sentences:
+  - Alicante est fréquentée par les automobilistes et les touristes .
+  - Les examinateurs ont passé sept mois à étudier leurs conclusions .
+  - Ferries to the island depart from the central station every 2 hours .
+- source_sentence: Scheduled to reopen in 2002 or 2003 , the Malibu site will house
+    only the Getty holdings in Greek and Roman antiquities , some of which date as
+    far back as 3000 b.c.
+  sentences:
+  - C' est impossible d' avoir des billets pour les enregistrements télévisés .
+  - The Getty holdings were taken hold of thanks to the researchers ' effort .
+  - After the first of may ends the peak season for ferries .
+- source_sentence: Une nouvelle recherche relie ces bactéries parodontale aux maladies
+    cardiaques , au diabète , aux bébés à faible poids de naissance , et à d' autres
+    saletés que vous attendez des bactéries qui se déchaînent dans le sang .
+  sentences:
+  - Le prix des actions de Caterpillar a baissé en 1991 quand ils ont fait grève .
+  - Ils agissent comme chaque année est la même .
+  - La recherche indique qu' il n' y a pas de lien entre les bactéries parodontale
+    et les maladies cardiaques ou le diabète .
+- source_sentence: L' ancien n' est pas une classification juridique qui entraîne
+    une perte automatique de ces droits .
+  sentences:
+  - Some degree of uncertainty is inherent in free-market systems .
+  - Les villes grecques d' Anatolie ont été exclues de l' appartenance à la Confédération
+    Delian .
+  - Ils voulaient plaider pour les personnes âgées .
+model-index:
+- name: SentenceTransformer based on sentence-transformers/all-MiniLM-L12-v2
+  results:
+  - task:
+      type: semantic-similarity
+      name: Semantic Similarity
+    dataset:
+      name: snli dev
+      type: snli-dev
+    metrics:
+    - type: pearson_cosine
+      value: 0.35421287329686374
+      name: Pearson Cosine
+    - type: spearman_cosine
+      value: 0.3592670991851331
+      name: Spearman Cosine
+    - type: pearson_manhattan
+      value: 0.34936411192844985
+      name: Pearson Manhattan
+    - type: spearman_manhattan
+      value: 0.3583327923327215
+      name: Spearman Manhattan
+    - type: pearson_euclidean
+      value: 0.34982920048695176
+      name: Pearson Euclidean
+    - type: spearman_euclidean
+      value: 0.35926709915022625
+      name: Spearman Euclidean
+    - type: pearson_dot
+      value: 0.3542128787197555
+      name: Pearson Dot
+    - type: spearman_dot
+      value: 0.35926727022169175
+      name: Spearman Dot
+    - type: pearson_max
+      value: 0.3542128787197555
+      name: Pearson Max
+    - type: spearman_max
+      value: 0.35926727022169175
+      name: Spearman Max
+---
+
+# SentenceTransformer based on sentence-transformers/all-MiniLM-L12-v2
+
+This is a [sentence-transformers](https://www.SBERT.net) model finetuned from [sentence-transformers/all-MiniLM-L12-v2](https://huggingface.co/sentence-transformers/all-MiniLM-L12-v2). It maps sentences & paragraphs to a 384-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.
+
+## Model Details
+
+### Model Description
+- **Model Type:** Sentence Transformer
+- **Base model:** [sentence-transformers/all-MiniLM-L12-v2](https://huggingface.co/sentence-transformers/all-MiniLM-L12-v2) <!-- at revision 30ce63ae64e71b9199b3d2eae9de99f64a26eedc -->
+- **Maximum Sequence Length:** 128 tokens
+- **Output Dimensionality:** 384 tokens
+- **Similarity Function:** Cosine Similarity
+<!-- - **Training Dataset:** Unknown -->
+<!-- - **Language:** Unknown -->
+<!-- - **License:** Unknown -->
+
+### Model Sources
+
+- **Documentation:** [Sentence Transformers Documentation](https://sbert.net)
+- **Repository:** [Sentence Transformers on GitHub](https://github.com/UKPLab/sentence-transformers)
+- **Hugging Face:** [Sentence Transformers on Hugging Face](https://huggingface.co/models?library=sentence-transformers)
+
+### Full Model Architecture
+
+```
+SentenceTransformer(
+  (0): Transformer({'max_seq_length': 128, 'do_lower_case': False}) with Transformer model: BertModel 
+  (1): Pooling({'word_embedding_dimension': 384, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
+  (2): Normalize()
+)
+```
+
+## Usage
+
+### Direct Usage (Sentence Transformers)
+
+First install the Sentence Transformers library:
+
+```bash
+pip install -U sentence-transformers
+```
+
+Then you can load this model and run inference.
+```python
+from sentence_transformers import SentenceTransformer
+
+# Download from the 🤗 Hub
+model = SentenceTransformer("Nessrine9/finetuned-snli-MiniLM-L12-v2-100k-en-fr")
+# Run inference
+sentences = [
+    "L' ancien n' est pas une classification juridique qui entraîne une perte automatique de ces droits .",
+    'Ils voulaient plaider pour les personnes âgées .',
+    "Les villes grecques d' Anatolie ont été exclues de l' appartenance à la Confédération Delian .",
+]
+embeddings = model.encode(sentences)
+print(embeddings.shape)
+# [3, 384]
+
+# Get the similarity scores for the embeddings
+similarities = model.similarity(embeddings, embeddings)
+print(similarities.shape)
+# [3, 3]
+```
+
+<!--
+### Direct Usage (Transformers)
+
+<details><summary>Click to see the direct usage in Transformers</summary>
+
+</details>
+-->
+
+<!--
+### Downstream Usage (Sentence Transformers)
+
+You can finetune this model on your own dataset.
+
+<details><summary>Click to expand</summary>
+
+</details>
+-->
+
+<!--
+### Out-of-Scope Use
+
+*List how the model may foreseeably be misused and address what users ought not to do with the model.*
+-->
+
+## Evaluation
+
+### Metrics
+
+#### Semantic Similarity
+* Dataset: `snli-dev`
+* Evaluated with [<code>EmbeddingSimilarityEvaluator</code>](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.EmbeddingSimilarityEvaluator)
+
+| Metric             | Value      |
+|:-------------------|:-----------|
+| pearson_cosine     | 0.3542     |
+| spearman_cosine    | 0.3593     |
+| pearson_manhattan  | 0.3494     |
+| spearman_manhattan | 0.3583     |
+| pearson_euclidean  | 0.3498     |
+| spearman_euclidean | 0.3593     |
+| pearson_dot        | 0.3542     |
+| spearman_dot       | 0.3593     |
+| pearson_max        | 0.3542     |
+| **spearman_max**   | **0.3593** |
+
+<!--
+## Bias, Risks and Limitations
+
+*What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.*
+-->
+
+<!--
+### Recommendations
+
+*What are recommendations with respect to the foreseeable issues? For example, filtering explicit content.*
+-->
+
+## Training Details
+
+### Training Dataset
+
+#### Unnamed Dataset
+
+
+* Size: 100,000 training samples
+* Columns: <code>sentence_0</code>, <code>sentence_1</code>, and <code>label</code>
+* Approximate statistics based on the first 1000 samples:
+  |         | sentence_0                                                                         | sentence_1                                                                        | label                                                         |
+  |:--------|:-----------------------------------------------------------------------------------|:----------------------------------------------------------------------------------|:--------------------------------------------------------------|
+  | type    | string                                                                             | string                                                                            | float                                                         |
+  | details | <ul><li>min: 5 tokens</li><li>mean: 34.31 tokens</li><li>max: 128 tokens</li></ul> | <ul><li>min: 3 tokens</li><li>mean: 18.24 tokens</li><li>max: 51 tokens</li></ul> | <ul><li>min: 0.0</li><li>mean: 0.5</li><li>max: 1.0</li></ul> |
+* Samples:
+  | sentence_0                                                                                                                                                                                                                                                                         | sentence_1                                                                                                  | label            |
+  |:-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:------------------------------------------------------------------------------------------------------------|:-----------------|
+  | <code>We 're off ! "</code>                                                                                                                                                                                                                                                        | <code>We 're not headed off .</code>                                                                        | <code>1.0</code> |
+  | <code>Il y en a eu un ici récemment qui me vient à l' esprit que c' est à propos d' une femme que c' est ridicule je veux dire que c' est presque euh ce serait drôle si ce n' était pas si triste je veux dire cette femme cette femme est sortie et a engagé quelqu' un à</code> | <code>Cette femme a engagé quelqu' un récemment pour le faire et s' est fait prendre immédiatement .</code> | <code>0.5</code> |
+  | <code>Gentilello a précisé qu' il n' avait pas critiqué le processus d' examen par les pairs , mais que les panels qui examinent les interventions en matière d' alcool dans l' eds devraient inclure des représentants de la médecine d' urgence .</code>                         | <code>Gentilello S' est ensuite battu avec un psychiatre sur le parking .</code>                            | <code>0.5</code> |
+* Loss: [<code>CosineSimilarityLoss</code>](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#cosinesimilarityloss) with these parameters:
+  ```json
+  {
+      "loss_fct": "torch.nn.modules.loss.MSELoss"
+  }
+  ```
+
+### Training Hyperparameters
+#### Non-Default Hyperparameters
+
+- `eval_strategy`: steps
+- `per_device_train_batch_size`: 16
+- `per_device_eval_batch_size`: 16
+- `num_train_epochs`: 4
+- `fp16`: True
+- `multi_dataset_batch_sampler`: round_robin
+
+#### All Hyperparameters
+<details><summary>Click to expand</summary>
+
+- `overwrite_output_dir`: False
+- `do_predict`: False
+- `eval_strategy`: steps
+- `prediction_loss_only`: True
+- `per_device_train_batch_size`: 16
+- `per_device_eval_batch_size`: 16
+- `per_gpu_train_batch_size`: None
+- `per_gpu_eval_batch_size`: None
+- `gradient_accumulation_steps`: 1
+- `eval_accumulation_steps`: None
+- `torch_empty_cache_steps`: None
+- `learning_rate`: 5e-05
+- `weight_decay`: 0.0
+- `adam_beta1`: 0.9
+- `adam_beta2`: 0.999
+- `adam_epsilon`: 1e-08
+- `max_grad_norm`: 1
+- `num_train_epochs`: 4
+- `max_steps`: -1
+- `lr_scheduler_type`: linear
+- `lr_scheduler_kwargs`: {}
+- `warmup_ratio`: 0.0
+- `warmup_steps`: 0
+- `log_level`: passive
+- `log_level_replica`: warning
+- `log_on_each_node`: True
+- `logging_nan_inf_filter`: True
+- `save_safetensors`: True
+- `save_on_each_node`: False
+- `save_only_model`: False
+- `restore_callback_states_from_checkpoint`: False
+- `no_cuda`: False
+- `use_cpu`: False
+- `use_mps_device`: False
+- `seed`: 42
+- `data_seed`: None
+- `jit_mode_eval`: False
+- `use_ipex`: False
+- `bf16`: False
+- `fp16`: True
+- `fp16_opt_level`: O1
+- `half_precision_backend`: auto
+- `bf16_full_eval`: False
+- `fp16_full_eval`: False
+- `tf32`: None
+- `local_rank`: 0
+- `ddp_backend`: None
+- `tpu_num_cores`: None
+- `tpu_metrics_debug`: False
+- `debug`: []
+- `dataloader_drop_last`: False
+- `dataloader_num_workers`: 0
+- `dataloader_prefetch_factor`: None
+- `past_index`: -1
+- `disable_tqdm`: False
+- `remove_unused_columns`: True
+- `label_names`: None
+- `load_best_model_at_end`: False
+- `ignore_data_skip`: False
+- `fsdp`: []
+- `fsdp_min_num_params`: 0
+- `fsdp_config`: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
+- `fsdp_transformer_layer_cls_to_wrap`: None
+- `accelerator_config`: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
+- `deepspeed`: None
+- `label_smoothing_factor`: 0.0
+- `optim`: adamw_torch
+- `optim_args`: None
+- `adafactor`: False
+- `group_by_length`: False
+- `length_column_name`: length
+- `ddp_find_unused_parameters`: None
+- `ddp_bucket_cap_mb`: None
+- `ddp_broadcast_buffers`: False
+- `dataloader_pin_memory`: True
+- `dataloader_persistent_workers`: False
+- `skip_memory_metrics`: True
+- `use_legacy_prediction_loop`: False
+- `push_to_hub`: False
+- `resume_from_checkpoint`: None
+- `hub_model_id`: None
+- `hub_strategy`: every_save
+- `hub_private_repo`: False
+- `hub_always_push`: False
+- `gradient_checkpointing`: False
+- `gradient_checkpointing_kwargs`: None
+- `include_inputs_for_metrics`: False
+- `eval_do_concat_batches`: True
+- `fp16_backend`: auto
+- `push_to_hub_model_id`: None
+- `push_to_hub_organization`: None
+- `mp_parameters`: 
+- `auto_find_batch_size`: False
+- `full_determinism`: False
+- `torchdynamo`: None
+- `ray_scope`: last
+- `ddp_timeout`: 1800
+- `torch_compile`: False
+- `torch_compile_backend`: None
+- `torch_compile_mode`: None
+- `dispatch_batches`: None
+- `split_batches`: None
+- `include_tokens_per_second`: False
+- `include_num_input_tokens_seen`: False
+- `neftune_noise_alpha`: None
+- `optim_target_modules`: None
+- `batch_eval_metrics`: False
+- `eval_on_start`: False
+- `eval_use_gather_object`: False
+- `batch_sampler`: batch_sampler
+- `multi_dataset_batch_sampler`: round_robin
+
+</details>
+
+### Training Logs
+| Epoch  | Step  | Training Loss | snli-dev_spearman_max |
+|:------:|:-----:|:-------------:|:---------------------:|
+| 0.08   | 500   | 0.1948        | 0.0484                |
+| 0.16   | 1000  | 0.1752        | 0.1177                |
+| 0.24   | 1500  | 0.1727        | 0.1136                |
+| 0.32   | 2000  | 0.1668        | 0.2050                |
+| 0.4    | 2500  | 0.1673        | 0.2227                |
+| 0.48   | 3000  | 0.1651        | 0.1760                |
+| 0.56   | 3500  | 0.1619        | 0.2195                |
+| 0.64   | 4000  | 0.1625        | 0.2308                |
+| 0.72   | 4500  | 0.1563        | 0.2405                |
+| 0.8    | 5000  | 0.1598        | 0.2773                |
+| 0.88   | 5500  | 0.1589        | 0.2359                |
+| 0.96   | 6000  | 0.1587        | 0.2084                |
+| 1.0    | 6250  | -             | 0.2615                |
+| 1.04   | 6500  | 0.158         | 0.2958                |
+| 1.12   | 7000  | 0.1557        | 0.2887                |
+| 1.2    | 7500  | 0.1544        | 0.2960                |
+| 1.28   | 8000  | 0.1535        | 0.2977                |
+| 1.3600 | 8500  | 0.1559        | 0.2546                |
+| 1.44   | 9000  | 0.1518        | 0.3201                |
+| 1.52   | 9500  | 0.1551        | 0.2894                |
+| 1.6    | 10000 | 0.149         | 0.2981                |
+| 1.6800 | 10500 | 0.152         | 0.3140                |
+| 1.76   | 11000 | 0.1484        | 0.3056                |
+| 1.8400 | 11500 | 0.1497        | 0.3051                |
+| 1.92   | 12000 | 0.1522        | 0.2893                |
+| 2.0    | 12500 | 0.1503        | 0.2944                |
+| 2.08   | 13000 | 0.1496        | 0.3039                |
+| 2.16   | 13500 | 0.1462        | 0.3314                |
+| 2.24   | 14000 | 0.1505        | 0.2470                |
+| 2.32   | 14500 | 0.1457        | 0.3081                |
+| 2.4    | 15000 | 0.1478        | 0.3204                |
+| 2.48   | 15500 | 0.1464        | 0.3248                |
+| 2.56   | 16000 | 0.1442        | 0.3360                |
+| 2.64   | 16500 | 0.1437        | 0.3418                |
+| 2.7200 | 17000 | 0.1416        | 0.3496                |
+| 2.8    | 17500 | 0.1434        | 0.3283                |
+| 2.88   | 18000 | 0.146         | 0.3246                |
+| 2.96   | 18500 | 0.1448        | 0.3352                |
+| 3.0    | 18750 | -             | 0.3248                |
+| 3.04   | 19000 | 0.1445        | 0.3394                |
+| 3.12   | 19500 | 0.1423        | 0.3430                |
+| 3.2    | 20000 | 0.1415        | 0.3410                |
+| 3.2800 | 20500 | 0.1411        | 0.3367                |
+| 3.36   | 21000 | 0.1445        | 0.3497                |
+| 3.44   | 21500 | 0.1383        | 0.3640                |
+| 3.52   | 22000 | 0.1408        | 0.3497                |
+| 3.6    | 22500 | 0.1374        | 0.3452                |
+| 3.68   | 23000 | 0.1401        | 0.3519                |
+| 3.76   | 23500 | 0.137         | 0.3582                |
+| 3.84   | 24000 | 0.1393        | 0.3610                |
+| 3.92   | 24500 | 0.1408        | 0.3575                |
+| 4.0    | 25000 | 0.1388        | 0.3593                |
+
+
+### Framework Versions
+- Python: 3.10.12
+- Sentence Transformers: 3.2.1
+- Transformers: 4.44.2
+- PyTorch: 2.5.0+cu121
+- Accelerate: 0.34.2
+- Datasets: 3.0.2
+- Tokenizers: 0.19.1
+
+## Citation
+
+### BibTeX
+
+#### Sentence Transformers
+```bibtex
+@inproceedings{reimers-2019-sentence-bert,
+    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
+    author = "Reimers, Nils and Gurevych, Iryna",
+    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
+    month = "11",
+    year = "2019",
+    publisher = "Association for Computational Linguistics",
+    url = "https://arxiv.org/abs/1908.10084",
+}
+```
+
+<!--
+## Glossary
+
+*Clearly define terms in order to be accessible across audiences.*
+-->
+
+<!--
+## Model Card Authors
+
+*Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction.*
+-->
+
+<!--
+## Model Card Contact
+
+*Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.*
+-->

config.json

@@ -0,0 +1,26 @@
+{
+  "_name_or_path": "sentence-transformers/all-MiniLM-L12-v2",
+  "architectures": [
+    "BertModel"
+  ],
+  "attention_probs_dropout_prob": 0.1,
+  "classifier_dropout": null,
+  "gradient_checkpointing": false,
+  "hidden_act": "gelu",
+  "hidden_dropout_prob": 0.1,
+  "hidden_size": 384,
+  "initializer_range": 0.02,
+  "intermediate_size": 1536,
+  "layer_norm_eps": 1e-12,
+  "max_position_embeddings": 512,
+  "model_type": "bert",
+  "num_attention_heads": 12,
+  "num_hidden_layers": 12,
+  "pad_token_id": 0,
+  "position_embedding_type": "absolute",
+  "torch_dtype": "float32",
+  "transformers_version": "4.44.2",
+  "type_vocab_size": 2,
+  "use_cache": true,
+  "vocab_size": 30522
+}

config_sentence_transformers.json

@@ -0,0 +1,10 @@
+{
+  "__version__": {
+    "sentence_transformers": "3.2.1",
+    "transformers": "4.44.2",
+    "pytorch": "2.5.0+cu121"
+  },
+  "prompts": {},
+  "default_prompt_name": null,
+  "similarity_fn_name": null
+}

model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4ee41106a5685eceaf6c5c288b847ec2168e743c8554e8521c0094618524e672
+size 133462128

modules.json

@@ -0,0 +1,20 @@
+[
+  {
+    "idx": 0,
+    "name": "0",
+    "path": "",
+    "type": "sentence_transformers.models.Transformer"
+  },
+  {
+    "idx": 1,
+    "name": "1",
+    "path": "1_Pooling",
+    "type": "sentence_transformers.models.Pooling"
+  },
+  {
+    "idx": 2,
+    "name": "2",
+    "path": "2_Normalize",
+    "type": "sentence_transformers.models.Normalize"
+  }
+]

sentence_bert_config.json

@@ -0,0 +1,4 @@
+{
+  "max_seq_length": 128,
+  "do_lower_case": false
+}

special_tokens_map.json

@@ -0,0 +1,37 @@
+{
+  "cls_token": {
+    "content": "[CLS]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "mask_token": {
+    "content": "[MASK]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "[PAD]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "sep_token": {
+    "content": "[SEP]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "unk_token": {
+    "content": "[UNK]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

tokenizer_config.json

@@ -0,0 +1,64 @@
+{
+  "added_tokens_decoder": {
+    "0": {
+      "content": "[PAD]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "100": {
+      "content": "[UNK]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "101": {
+      "content": "[CLS]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "102": {
+      "content": "[SEP]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "103": {
+      "content": "[MASK]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "clean_up_tokenization_spaces": true,
+  "cls_token": "[CLS]",
+  "do_basic_tokenize": true,
+  "do_lower_case": true,
+  "mask_token": "[MASK]",
+  "max_length": 128,
+  "model_max_length": 128,
+  "never_split": null,
+  "pad_to_multiple_of": null,
+  "pad_token": "[PAD]",
+  "pad_token_type_id": 0,
+  "padding_side": "right",
+  "sep_token": "[SEP]",
+  "stride": 0,
+  "strip_accents": null,
+  "tokenize_chinese_chars": true,
+  "tokenizer_class": "BertTokenizer",
+  "truncation_side": "right",
+  "truncation_strategy": "longest_first",
+  "unk_token": "[UNK]"
+}