Add new SentenceTransformer model

.gitattributes

@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+tokenizer.json filter=lfs diff=lfs merge=lfs -text

1_Pooling/config.json

@@ -0,0 +1,10 @@
+{
+  "word_embedding_dimension": 896,
+  "pooling_mode_cls_token": false,
+  "pooling_mode_mean_tokens": true,
+  "pooling_mode_max_tokens": false,
+  "pooling_mode_mean_sqrt_len_tokens": false,
+  "pooling_mode_weightedmean_tokens": false,
+  "pooling_mode_lasttoken": false,
+  "include_prompt": true
+}

README.md

@@ -0,0 +1,666 @@
+---
+tags:
+- sentence-transformers
+- sentence-similarity
+- feature-extraction
+- generated_from_trainer
+- dataset_size:2859594
+- loss:MatryoshkaLoss
+- loss:MultipleNegativesRankingLoss
+base_model: Qwen/Qwen2.5-0.5B-Instruct
+widget:
+- source_sentence: How old is Garry Marshall?
+  sentences:
+  - 'Garry Marshall
+
+    On the morning of July 19, 2016, Marshall died at a hospital in Burbank, California
+    at the age of 81 due to complications of pneumonia after suffering a stroke.[20][21]'
+  - 'Gregg Marshall
+
+    Michael Gregg Marshall (born February 27, 1963) is an American college basketball
+    coach who currently leads the Shockers team at Wichita State University. Marshall
+    has coached his teams to appearances in the NCAA Men''s Division I Basketball
+    Tournament in twelve of his eighteen years as a head coach. He is the most successful
+    head coach in Wichita State University history (261 wins), and is also the most
+    successful head coach in Winthrop University history (194 wins).'
+  - 'Guillotine
+
+    For a period of time after its invention, the guillotine was called a louisette.
+    However, it was later named after Guillotin who had proposed that a less painful
+    method of execution should be found in place of the breaking wheel, though he
+    opposed the death penalty and bemoaned the association of the guillotine with
+    his name.'
+- source_sentence: Are there cherry trees in Cherry Springs State Park?
+  sentences:
+  - 'Cherry Springs State Park
+
+    Awards and press recognition have come to Cherry Springs and its staff. Thom Bemus,
+    who initiated and coordinates the Stars-n-Parks program, was named DCNR''s 2002Volunteer
+    of the Year.[66] In 2007the park''s Dark Sky Programming and staff received the
+    Environmental Education Excellence in Programming award from the Pennsylvania
+    Recreation and Parks Society.[67] Operations manager Chip Harrison and his wife
+    Maxine, who directs the Dark Sky Fund, received a 2008award from the Pennsylvania
+    Outdoor Lighting Council for "steadfast adherence and active promotion of the
+    principles of responsible outdoor lighting at Cherry Springs State Park".[68]
+    The DCNR has named Cherry Springs one of "25 Must-See Pennsylvania State Parks",
+    specifically for having the "darkest night skies on the east coast".[69] Cherry
+    Springs State Park was featured in the national press in 2003when USA Today named
+    it one of "10Great Places to get some stars in your eyes",[70] in 2006when National
+    Geographic Adventure featured it in "Pennsylvania: The Wild, Wild East",[71] and
+    in The New York Times in 2007.[53] All these were before it was named an International
+    Dark Sky Park by the International Dark-Sky Association in 2008.[38]'
+  - 'Cantonese
+
+    Although Cantonese shares a lot of vocabulary with Mandarin, the two varieties
+    are mutually unintelligible because of differences in pronunciation, grammar and
+    lexicon. Sentence structure, in particular the placement of verbs, sometimes differs
+    between the two varieties. A notable difference between Cantonese and Mandarin
+    is how the spoken word is written; both can be recorded verbatim, but very few
+    Cantonese speakers are knowledgeable in the full Cantonese written vocabulary,
+    so a non-verbatim formalized written form is adopted, which is more akin to the
+    Mandarin written form.[4][5] This results in the situation in which a Cantonese
+    and a Mandarin text may look similar but are pronounced differently.'
+  - 'Cherry Springs State Park
+
+    Cherry Springs State Park is an 82-acre (33ha)[a] Pennsylvania state park in Potter
+    County, Pennsylvania, United States. The park was created from land within the
+    Susquehannock State Forest, and is on Pennsylvania Route 44 in West Branch Township.
+    Cherry Springs, named for a large stand of Black Cherry trees in the park, is
+    atop the dissected Allegheny Plateau at an elevation of 2,300 feet (701m). It
+    is popular with astronomers and stargazers for having "some of the darkest night
+    skies on the east coast" of the United States, and was chosen by the Pennsylvania
+    Department of Conservation and Natural Resources (DCNR) and its Bureau of Parks
+    as one of "25 Must-See Pennsylvania State Parks".[4]'
+- source_sentence: How many regions are in Belgium?
+  sentences:
+  - 'Pine City, Minnesota
+
+    Pine City is a city in Pine County, Minnesota, in East Central Minnesota. Pine
+    City is the county seat of, and the largest city in, Pine County.[7] A portion
+    of the city is located on the Mille Lacs Indian Reservation. Founded as a railway
+    town, it quickly became a logging community and the surrounding lakes made it
+    a resort town. Today, it is an arts town and commuter town to jobs in the Minneapolis–Saint
+    Paul metropolitan area.[8] It is also a green city.[9] The population was 3,127
+    at the 2010 census.'
+  - 'Provinces of Belgium
+
+    The country of Belgium is divided into three regions. Two of these regions, the
+    Flemish Region or Flanders, and Walloon Region, or Wallonia, are each subdivided
+    into five provinces. The third region, the Brussels-Capital Region, is not divided
+    into provinces, as it was originally only a small part of a province itself.'
+  - 'United Belgian States
+
+    The United Belgian States was a confederal republic of eight provinces which had
+    their own governments, were sovereign and independent, and were governed directly
+    by the Sovereign Congress (; ), the confederal government. The Sovereign Congress
+    was seated in Brussels and consisted of representatives of each of the eight provinces.
+    The provinces of the republic were divided into 11 smaller separate territories,
+    each with their own regional identities:In 1789, a church-inspired popular revolt
+    broke out in reaction to the emperor''s centralizing and anticlerical policies.
+    Two factions appeared: the "Statists" who opposed the reforms, and the "Vonckists"
+    named for Jan Frans Vonck who initially supported the reforms but then joined
+    the opposition, due to the clumsy way in which the reforms were carried out.'
+- source_sentence: Are there black holes near the galactic nucleus?
+  sentences:
+  - 'Supermassive black hole
+
+    In September 2014, data from different X-ray telescopes has shown that the extremely
+    small, dense, ultracompact dwarf galaxy M60-UCD1 hosts a 20 million solar mass
+    black hole at its center, accounting for more than 10% of the total mass of the
+    galaxy. The discovery is quite surprising, since the black hole is five times
+    more massive than the Milky Way''s black hole despite the galaxy being less than
+    five-thousandths the mass of the Milky Way.'
+  - 'Aquarela do Brasil
+
+    "Aquarela do Brasil" (Portuguese:[akwaˈɾɛlɐ du bɾaˈziw], Watercolor of Brazil),
+    written by Ary Barroso in 1939 and known in the English-speaking world simply
+    as "Brazil", is one of the most famous Brazilian songs.'
+  - 'Supermassive black hole
+
+    The difficulty in forming a supermassive black hole resides in the need for enough
+    matter to be in a small enough volume. This matter needs to have very little angular
+    momentum in order for this to happen. Normally, the process of accretion involves
+    transporting a large initial endowment of angular momentum outwards, and this
+    appears to be the limiting factor in black hole growth. This is a major component
+    of the theory of accretion disks. Gas accretion is the most efficient and also
+    the most conspicuous way in which black holes grow. The majority of the mass growth
+    of supermassive black holes is thought to occur through episodes of rapid gas
+    accretion, which are observable as active galactic nuclei or quasars. Observations
+    reveal that quasars were much more frequent when the Universe was younger, indicating
+    that supermassive black holes formed and grew early. A major constraining factor
+    for theories of supermassive black hole formation is the observation of distant
+    luminous quasars, which indicate that supermassive black holes of billions of
+    solar masses had already formed when the Universe was less than one billion years
+    old. This suggests that supermassive black holes arose very early in the Universe,
+    inside the first massive galaxies.'
+- source_sentence: When did the July Monarchy end?
+  sentences:
+  - 'July Monarchy
+
+    Despite the return of the House of Bourbon to power, France was much changed from
+    the era of the ancien régime. The egalitarianism and liberalism of the revolutionaries
+    remained an important force and the autocracy and hierarchy of the earlier era
+    could not be fully restored. Economic changes, which had been underway long before
+    the revolution, had progressed further during the years of turmoil and were firmly
+    entrenched by 1815. These changes had seen power shift from the noble landowners
+    to the urban merchants. The administrative reforms of Napoleon, such as the Napoleonic
+    Code and efficient bureaucracy, also remained in place. These changes produced
+    a unified central government that was fiscally sound and had much control over
+    all areas of French life, a sharp difference from the complicated mix of feudal
+    and absolutist traditions and institutions of pre-Revolutionary Bourbons.'
+  - 'Wachovia
+
+    Wachovia Corporation began on June 16, 1879 in Winston-Salem, North Carolina as
+    the Wachovia National Bank. The bank was co-founded by James Alexander Gray and
+    William Lemly.[9] In 1911, the bank merged with Wachovia Loan and Trust Company,
+    "the largest trust company between Baltimore and New Orleans",[10] which had been
+    founded on June 15, 1893. Wachovia grew to become one of the largest banks in
+    the Southeast partly on the strength of its accounts from the R.J. Reynolds Tobacco
+    Company, which was also headquartered in Winston-Salem.[11] On December 12, 1986,
+    Wachovia purchased First Atlanta. Founded as Atlanta National Bank on September
+    14, 1865, and later renamed to First National Bank of Atlanta, this institution
+    was the oldest national bank in Atlanta. This purchase made Wachovia one of the
+    few companies with dual headquarters: one in Winston-Salem and one in Atlanta.
+    In 1991, Wachovia entered the South Carolina market by acquiring South Carolina
+    National Corporation,[12] founded as the Bank of Charleston in 1834. In 1998,
+    Wachovia acquired two Virginia-based banks, Jefferson National Bank and Central
+    Fidelity Bank. In 1997, Wachovia acquired both 1st United Bancorp and American
+    Bankshares Inc, giving its first entry into Florida. In 2000, Wachovia made its
+    final purchase, which was Republic Security Bank.'
+  - 'July Monarchy
+
+    The July Monarchy (French: Monarchie de Juillet) was a liberal constitutional
+    monarchy in France under Louis Philippe I, starting with the July Revolution of
+    1830 and ending with the Revolution of 1848. It marks the end of the Bourbon Restoration
+    (1814–1830). It began with the overthrow of the conservative government of Charles
+    X, the last king of the House of Bourbon.'
+pipeline_tag: sentence-similarity
+library_name: sentence-transformers
+metrics:
+- pearson_cosine
+- spearman_cosine
+model-index:
+- name: SentenceTransformer based on Qwen/Qwen2.5-0.5B-Instruct
+  results:
+  - task:
+      type: semantic-similarity
+      name: Semantic Similarity
+    dataset:
+      name: sts dev 896
+      type: sts-dev-896
+    metrics:
+    - type: pearson_cosine
+      value: 0.45729692013517886
+      name: Pearson Cosine
+    - type: spearman_cosine
+      value: 0.49645340246652353
+      name: Spearman Cosine
+  - task:
+      type: semantic-similarity
+      name: Semantic Similarity
+    dataset:
+      name: sts dev 768
+      type: sts-dev-768
+    metrics:
+    - type: pearson_cosine
+      value: 0.4455125981991164
+      name: Pearson Cosine
+    - type: spearman_cosine
+      value: 0.4896539219726307
+      name: Spearman Cosine
+---
+
+# SentenceTransformer based on Qwen/Qwen2.5-0.5B-Instruct
+
+This is a [sentence-transformers](https://www.SBERT.net) model finetuned from [Qwen/Qwen2.5-0.5B-Instruct](https://huggingface.co/Qwen/Qwen2.5-0.5B-Instruct). It maps sentences & paragraphs to a 896-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.
+
+## Model Details
+
+### Model Description
+- **Model Type:** Sentence Transformer
+- **Base model:** [Qwen/Qwen2.5-0.5B-Instruct](https://huggingface.co/Qwen/Qwen2.5-0.5B-Instruct) <!-- at revision 7ae557604adf67be50417f59c2c2f167def9a775 -->
+- **Maximum Sequence Length:** 1024 tokens
+- **Output Dimensionality:** 896 dimensions
+- **Similarity Function:** Cosine Similarity
+<!-- - **Training Dataset:** Unknown -->
+<!-- - **Language:** Unknown -->
+<!-- - **License:** Unknown -->
+
+### Model Sources
+
+- **Documentation:** [Sentence Transformers Documentation](https://sbert.net)
+- **Repository:** [Sentence Transformers on GitHub](https://github.com/UKPLab/sentence-transformers)
+- **Hugging Face:** [Sentence Transformers on Hugging Face](https://huggingface.co/models?library=sentence-transformers)
+
+### Full Model Architecture
+
+```
+SentenceTransformer(
+  (0): Transformer({'max_seq_length': 1024, 'do_lower_case': False}) with Transformer model: Qwen2Model 
+  (1): Pooling({'word_embedding_dimension': 896, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
+)
+```
+
+## Usage
+
+### Direct Usage (Sentence Transformers)
+
+First install the Sentence Transformers library:
+
+```bash
+pip install -U sentence-transformers
+```
+
+Then you can load this model and run inference.
+```python
+from sentence_transformers import SentenceTransformer
+
+# Download from the 🤗 Hub
+model = SentenceTransformer("AlexWortega/qwen1k")
+# Run inference
+sentences = [
+    'When did the July Monarchy end?',
+    'July Monarchy\nThe July Monarchy (French: Monarchie de Juillet) was a liberal constitutional monarchy in France under Louis Philippe I, starting with the July Revolution of 1830 and ending with the Revolution of 1848. It marks the end of the Bourbon Restoration (1814–1830). It began with the overthrow of the conservative government of Charles X, the last king of the House of Bourbon.',
+    'July Monarchy\nDespite the return of the House of Bourbon to power, France was much changed from the era of the ancien régime. The egalitarianism and liberalism of the revolutionaries remained an important force and the autocracy and hierarchy of the earlier era could not be fully restored. Economic changes, which had been underway long before the revolution, had progressed further during the years of turmoil and were firmly entrenched by 1815. These changes had seen power shift from the noble landowners to the urban merchants. The administrative reforms of Napoleon, such as the Napoleonic Code and efficient bureaucracy, also remained in place. These changes produced a unified central government that was fiscally sound and had much control over all areas of French life, a sharp difference from the complicated mix of feudal and absolutist traditions and institutions of pre-Revolutionary Bourbons.',
+]
+embeddings = model.encode(sentences)
+print(embeddings.shape)
+# [3, 896]
+
+# Get the similarity scores for the embeddings
+similarities = model.similarity(embeddings, embeddings)
+print(similarities.shape)
+# [3, 3]
+```
+
+<!--
+### Direct Usage (Transformers)
+
+<details><summary>Click to see the direct usage in Transformers</summary>
+
+</details>
+-->
+
+<!--
+### Downstream Usage (Sentence Transformers)
+
+You can finetune this model on your own dataset.
+
+<details><summary>Click to expand</summary>
+
+</details>
+-->
+
+<!--
+### Out-of-Scope Use
+
+*List how the model may foreseeably be misused and address what users ought not to do with the model.*
+-->
+
+## Evaluation
+
+### Metrics
+
+#### Semantic Similarity
+
+* Datasets: `sts-dev-896` and `sts-dev-768`
+* Evaluated with [<code>EmbeddingSimilarityEvaluator</code>](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.EmbeddingSimilarityEvaluator)
+
+| Metric              | sts-dev-896 | sts-dev-768 |
+|:--------------------|:------------|:------------|
+| pearson_cosine      | 0.4573      | 0.4455      |
+| **spearman_cosine** | **0.4965**  | **0.4897**  |
+
+<!--
+## Bias, Risks and Limitations
+
+*What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.*
+-->
+
+<!--
+### Recommendations
+
+*What are recommendations with respect to the foreseeable issues? For example, filtering explicit content.*
+-->
+
+## Training Details
+
+### Training Dataset
+
+#### Unnamed Dataset
+
+
+* Size: 2,859,594 training samples
+* Columns: <code>query</code>, <code>response</code>, and <code>negative</code>
+* Approximate statistics based on the first 1000 samples:
+  |         | query                                                                            | response                                                                             | negative                                                                            |
+  |:--------|:---------------------------------------------------------------------------------|:-------------------------------------------------------------------------------------|:------------------------------------------------------------------------------------|
+  | type    | string                                                                           | string                                                                               | string                                                                              |
+  | details | <ul><li>min: 4 tokens</li><li>mean: 8.76 tokens</li><li>max: 26 tokens</li></ul> | <ul><li>min: 23 tokens</li><li>mean: 141.88 tokens</li><li>max: 532 tokens</li></ul> | <ul><li>min: 4 tokens</li><li>mean: 134.02 tokens</li><li>max: 472 tokens</li></ul> |
+* Samples:
+  | query                                             | response                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                 | negative                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  |
+  |:--------------------------------------------------|:---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+  | <code>Was there a year 0?</code>                  | <code>Year zero<br>Year zero does not exist in the anno Domini system usually used to number years in the Gregorian calendar and in its predecessor, the Julian calendar. In this system, the year 1 BC is followed by AD 1. However, there is a year zero in astronomical year numbering (where it coincides with the Julian year 1 BC) and in ISO 8601:2004 (where it coincides with the Gregorian year 1 BC) as well as in all Buddhist and Hindu calendars.</code>                                                                                                                                                                                                                                                                                                                                   | <code>504<br>Year 504 (DIV) was a leap year starting on Thursday (link will display the full calendar) of the Julian calendar. At the time, it was known as the Year of the Consulship of Nicomachus without colleague (or, less frequently, year 1257 "Ab urbe condita"). The denomination 504 for this year has been used since the early medieval period, when the Anno Domini calendar era became the prevalent method in Europe for naming years.</code>                                                                                                                                                                                                                                                                                             |
+  | <code>When is the dialectical method used?</code> | <code>Dialectic<br>Dialectic or dialectics (Greek: διαλεκτική, dialektikḗ; related to dialogue), also known as the dialectical method, is at base a discourse between two or more people holding different points of view about a subject but wishing to establish the truth through reasoned arguments. Dialectic resembles debate, but the concept excludes subjective elements such as emotional appeal and the modern pejorative sense of rhetoric.[1][2] Dialectic may be contrasted with the didactic method, wherein one side of the conversation teaches the other. Dialectic is alternatively known as minor logic, as opposed to major logic or critique.</code>                                                                                                                               | <code>Derek Bentley case<br>Another factor in the posthumous defence was that a "confession" recorded by Bentley, which was claimed by the prosecution to be a "verbatim record of dictated monologue", was shown by forensic linguistics methods to have been largely edited by policemen. Linguist Malcolm Coulthard showed that certain patterns, such as the frequency of the word "then" and the grammatical use of "then" after the grammatical subject ("I then" rather than "then I"), were not consistent with Bentley's use of language (his idiolect), as evidenced in court testimony. These patterns fit better the recorded testimony of the policemen involved. This is one of the earliest uses of forensic linguistics on record.</code> |
+  | <code>What do Grasshoppers eat?</code>            | <code>Grasshopper<br>Grasshoppers are plant-eaters, with a few species at times becoming serious pests of cereals, vegetables and pasture, especially when they swarm in their millions as locusts and destroy crops over wide areas. They protect themselves from predators by camouflage; when detected, many species attempt to startle the predator with a brilliantly-coloured wing-flash while jumping and (if adult) launching themselves into the air, usually flying for only a short distance. Other species such as the rainbow grasshopper have warning coloration which deters predators. Grasshoppers are affected by parasites and various diseases, and many predatory creatures feed on both nymphs and adults. The eggs are the subject of attack by parasitoids and predators.</code> | <code>Groundhog<br>Very often the dens of groundhogs provide homes for other animals including skunks, red foxes, and cottontail rabbits. The fox and skunk feed upon field mice, grasshoppers, beetles and other creatures that destroy farm crops. In aiding these animals, the groundhog indirectly helps the farmer. In addition to providing homes for itself and other animals, the groundhog aids in soil improvement by bringing subsoil to the surface. The groundhog is also a valuable game animal and is considered a difficult sport when hunted in a fair manner. In some parts of Appalachia, they are eaten.</code>                                                                                                                       |
+* Loss: [<code>MatryoshkaLoss</code>](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#matryoshkaloss) with these parameters:
+  ```json
+  {
+      "loss": "MultipleNegativesRankingLoss",
+      "matryoshka_dims": [
+          896,
+          768
+      ],
+      "matryoshka_weights": [
+          1,
+          1
+      ],
+      "n_dims_per_step": -1
+  }
+  ```
+
+### Training Hyperparameters
+#### Non-Default Hyperparameters
+
+- `eval_strategy`: steps
+- `per_device_train_batch_size`: 12
+- `per_device_eval_batch_size`: 12
+- `gradient_accumulation_steps`: 4
+- `num_train_epochs`: 1
+- `warmup_ratio`: 0.3
+- `bf16`: True
+- `batch_sampler`: no_duplicates
+
+#### All Hyperparameters
+<details><summary>Click to expand</summary>
+
+- `overwrite_output_dir`: False
+- `do_predict`: False
+- `eval_strategy`: steps
+- `prediction_loss_only`: True
+- `per_device_train_batch_size`: 12
+- `per_device_eval_batch_size`: 12
+- `per_gpu_train_batch_size`: None
+- `per_gpu_eval_batch_size`: None
+- `gradient_accumulation_steps`: 4
+- `eval_accumulation_steps`: None
+- `torch_empty_cache_steps`: None
+- `learning_rate`: 5e-05
+- `weight_decay`: 0.0
+- `adam_beta1`: 0.9
+- `adam_beta2`: 0.999
+- `adam_epsilon`: 1e-08
+- `max_grad_norm`: 1.0
+- `num_train_epochs`: 1
+- `max_steps`: -1
+- `lr_scheduler_type`: linear
+- `lr_scheduler_kwargs`: {}
+- `warmup_ratio`: 0.3
+- `warmup_steps`: 0
+- `log_level`: passive
+- `log_level_replica`: warning
+- `log_on_each_node`: True
+- `logging_nan_inf_filter`: True
+- `save_safetensors`: True
+- `save_on_each_node`: False
+- `save_only_model`: False
+- `restore_callback_states_from_checkpoint`: False
+- `no_cuda`: False
+- `use_cpu`: False
+- `use_mps_device`: False
+- `seed`: 42
+- `data_seed`: None
+- `jit_mode_eval`: False
+- `use_ipex`: False
+- `bf16`: True
+- `fp16`: False
+- `fp16_opt_level`: O1
+- `half_precision_backend`: auto
+- `bf16_full_eval`: False
+- `fp16_full_eval`: False
+- `tf32`: None
+- `local_rank`: 0
+- `ddp_backend`: None
+- `tpu_num_cores`: None
+- `tpu_metrics_debug`: False
+- `debug`: []
+- `dataloader_drop_last`: False
+- `dataloader_num_workers`: 0
+- `dataloader_prefetch_factor`: None
+- `past_index`: -1
+- `disable_tqdm`: False
+- `remove_unused_columns`: True
+- `label_names`: None
+- `load_best_model_at_end`: False
+- `ignore_data_skip`: False
+- `fsdp`: []
+- `fsdp_min_num_params`: 0
+- `fsdp_config`: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
+- `fsdp_transformer_layer_cls_to_wrap`: None
+- `accelerator_config`: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
+- `deepspeed`: None
+- `label_smoothing_factor`: 0.0
+- `optim`: adamw_torch
+- `optim_args`: None
+- `adafactor`: False
+- `group_by_length`: False
+- `length_column_name`: length
+- `ddp_find_unused_parameters`: None
+- `ddp_bucket_cap_mb`: None
+- `ddp_broadcast_buffers`: False
+- `dataloader_pin_memory`: True
+- `dataloader_persistent_workers`: False
+- `skip_memory_metrics`: True
+- `use_legacy_prediction_loop`: False
+- `push_to_hub`: False
+- `resume_from_checkpoint`: None
+- `hub_model_id`: None
+- `hub_strategy`: every_save
+- `hub_private_repo`: False
+- `hub_always_push`: False
+- `gradient_checkpointing`: False
+- `gradient_checkpointing_kwargs`: None
+- `include_inputs_for_metrics`: False
+- `include_for_metrics`: []
+- `eval_do_concat_batches`: True
+- `fp16_backend`: auto
+- `push_to_hub_model_id`: None
+- `push_to_hub_organization`: None
+- `mp_parameters`: 
+- `auto_find_batch_size`: False
+- `full_determinism`: False
+- `torchdynamo`: None
+- `ray_scope`: last
+- `ddp_timeout`: 1800
+- `torch_compile`: False
+- `torch_compile_backend`: None
+- `torch_compile_mode`: None
+- `dispatch_batches`: None
+- `split_batches`: None
+- `include_tokens_per_second`: False
+- `include_num_input_tokens_seen`: False
+- `neftune_noise_alpha`: None
+- `optim_target_modules`: None
+- `batch_eval_metrics`: False
+- `eval_on_start`: False
+- `use_liger_kernel`: False
+- `eval_use_gather_object`: False
+- `average_tokens_across_devices`: False
+- `prompts`: None
+- `batch_sampler`: no_duplicates
+- `multi_dataset_batch_sampler`: proportional
+
+</details>
+
+### Training Logs
+| Epoch  | Step | Training Loss | sts-dev-896_spearman_cosine | sts-dev-768_spearman_cosine |
+|:------:|:----:|:-------------:|:---------------------------:|:---------------------------:|
+| 0.0002 | 10   | 4.4351        | -                           | -                           |
+| 0.0003 | 20   | 4.6508        | -                           | -                           |
+| 0.0005 | 30   | 4.7455        | -                           | -                           |
+| 0.0007 | 40   | 4.5427        | -                           | -                           |
+| 0.0008 | 50   | 4.3982        | -                           | -                           |
+| 0.0010 | 60   | 4.3755        | -                           | -                           |
+| 0.0012 | 70   | 4.4105        | -                           | -                           |
+| 0.0013 | 80   | 5.2227        | -                           | -                           |
+| 0.0015 | 90   | 5.8062        | -                           | -                           |
+| 0.0017 | 100  | 5.7645        | -                           | -                           |
+| 0.0018 | 110  | 5.9261        | -                           | -                           |
+| 0.0020 | 120  | 5.8301        | -                           | -                           |
+| 0.0022 | 130  | 5.7602        | -                           | -                           |
+| 0.0023 | 140  | 5.9392        | -                           | -                           |
+| 0.0025 | 150  | 5.7523        | -                           | -                           |
+| 0.0027 | 160  | 5.8585        | -                           | -                           |
+| 0.0029 | 170  | 5.7916        | -                           | -                           |
+| 0.0030 | 180  | 5.8157        | -                           | -                           |
+| 0.0032 | 190  | 5.7102        | -                           | -                           |
+| 0.0034 | 200  | 5.5844        | -                           | -                           |
+| 0.0035 | 210  | 5.5463        | -                           | -                           |
+| 0.0037 | 220  | 5.5823        | -                           | -                           |
+| 0.0039 | 230  | 5.5514        | -                           | -                           |
+| 0.0040 | 240  | 5.5646        | -                           | -                           |
+| 0.0042 | 250  | 5.5783        | -                           | -                           |
+| 0.0044 | 260  | 5.5344        | -                           | -                           |
+| 0.0045 | 270  | 5.523         | -                           | -                           |
+| 0.0047 | 280  | 5.4969        | -                           | -                           |
+| 0.0049 | 290  | 5.5407        | -                           | -                           |
+| 0.0050 | 300  | 5.6171        | -                           | -                           |
+| 0.0052 | 310  | 5.5581        | -                           | -                           |
+| 0.0054 | 320  | 5.8903        | -                           | -                           |
+| 0.0055 | 330  | 5.8675        | -                           | -                           |
+| 0.0057 | 340  | 5.745         | -                           | -                           |
+| 0.0059 | 350  | 5.6041        | -                           | -                           |
+| 0.0060 | 360  | 5.5476        | -                           | -                           |
+| 0.0062 | 370  | 5.3964        | -                           | -                           |
+| 0.0064 | 380  | 5.3564        | -                           | -                           |
+| 0.0065 | 390  | 5.3054        | -                           | -                           |
+| 0.0067 | 400  | 5.2779        | -                           | -                           |
+| 0.0069 | 410  | 5.206         | -                           | -                           |
+| 0.0070 | 420  | 5.2168        | -                           | -                           |
+| 0.0072 | 430  | 5.1645        | -                           | -                           |
+| 0.0074 | 440  | 5.1797        | -                           | -                           |
+| 0.0076 | 450  | 5.2526        | -                           | -                           |
+| 0.0077 | 460  | 5.1768        | -                           | -                           |
+| 0.0079 | 470  | 5.3519        | -                           | -                           |
+| 0.0081 | 480  | 5.2982        | -                           | -                           |
+| 0.0082 | 490  | 5.3229        | -                           | -                           |
+| 0.0084 | 500  | 5.3758        | -                           | -                           |
+| 0.0086 | 510  | 5.2478        | -                           | -                           |
+| 0.0087 | 520  | 5.1799        | -                           | -                           |
+| 0.0089 | 530  | 5.1088        | -                           | -                           |
+| 0.0091 | 540  | 4.977         | -                           | -                           |
+| 0.0092 | 550  | 4.9108        | -                           | -                           |
+| 0.0094 | 560  | 4.811         | -                           | -                           |
+| 0.0096 | 570  | 4.7203        | -                           | -                           |
+| 0.0097 | 580  | 4.6499        | -                           | -                           |
+| 0.0099 | 590  | 4.4548        | -                           | -                           |
+| 0.0101 | 600  | 4.2891        | -                           | -                           |
+| 0.0102 | 610  | 4.1881        | -                           | -                           |
+| 0.0104 | 620  | 4.6           | -                           | -                           |
+| 0.0106 | 630  | 4.5365        | -                           | -                           |
+| 0.0107 | 640  | 4.3086        | -                           | -                           |
+| 0.0109 | 650  | 4.0452        | -                           | -                           |
+| 0.0111 | 660  | 3.9041        | -                           | -                           |
+| 0.0112 | 670  | 4.3938        | -                           | -                           |
+| 0.0114 | 680  | 4.3198        | -                           | -                           |
+| 0.0116 | 690  | 4.1294        | -                           | -                           |
+| 0.0117 | 700  | 4.077         | -                           | -                           |
+| 0.0119 | 710  | 3.9174        | -                           | -                           |
+| 0.0121 | 720  | 4.1629        | -                           | -                           |
+| 0.0123 | 730  | 3.9611        | -                           | -                           |
+| 0.0124 | 740  | 3.7768        | -                           | -                           |
+| 0.0126 | 750  | 3.5842        | -                           | -                           |
+| 0.0128 | 760  | 3.1196        | -                           | -                           |
+| 0.0129 | 770  | 3.6288        | -                           | -                           |
+| 0.0131 | 780  | 3.273         | -                           | -                           |
+| 0.0133 | 790  | 2.7889        | -                           | -                           |
+| 0.0134 | 800  | 2.5096        | -                           | -                           |
+| 0.0136 | 810  | 1.8878        | -                           | -                           |
+| 0.0138 | 820  | 2.3423        | -                           | -                           |
+| 0.0139 | 830  | 1.7687        | -                           | -                           |
+| 0.0141 | 840  | 2.0781        | -                           | -                           |
+| 0.0143 | 850  | 2.4598        | -                           | -                           |
+| 0.0144 | 860  | 1.7667        | -                           | -                           |
+| 0.0146 | 870  | 2.6247        | -                           | -                           |
+| 0.0148 | 880  | 1.916         | -                           | -                           |
+| 0.0149 | 890  | 2.0817        | -                           | -                           |
+| 0.0151 | 900  | 2.3679        | -                           | -                           |
+| 0.0153 | 910  | 1.418         | -                           | -                           |
+| 0.0154 | 920  | 2.7353        | -                           | -                           |
+| 0.0156 | 930  | 1.992         | -                           | -                           |
+| 0.0158 | 940  | 1.4564        | -                           | -                           |
+| 0.0159 | 950  | 1.4154        | -                           | -                           |
+| 0.0161 | 960  | 0.9499        | -                           | -                           |
+| 0.0163 | 970  | 1.6304        | -                           | -                           |
+| 0.0164 | 980  | 0.9264        | -                           | -                           |
+| 0.0166 | 990  | 1.3278        | -                           | -                           |
+| 0.0168 | 1000 | 1.686         | 0.4965                      | 0.4897                      |
+
+
+### Framework Versions
+- Python: 3.10.12
+- Sentence Transformers: 3.3.0
+- Transformers: 4.46.2
+- PyTorch: 2.1.0+cu118
+- Accelerate: 1.1.1
+- Datasets: 3.1.0
+- Tokenizers: 0.20.3
+
+## Citation
+
+### BibTeX
+
+#### Sentence Transformers
+```bibtex
+@inproceedings{reimers-2019-sentence-bert,
+    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
+    author = "Reimers, Nils and Gurevych, Iryna",
+    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
+    month = "11",
+    year = "2019",
+    publisher = "Association for Computational Linguistics",
+    url = "https://arxiv.org/abs/1908.10084",
+}
+```
+
+#### MatryoshkaLoss
+```bibtex
+@misc{kusupati2024matryoshka,
+    title={Matryoshka Representation Learning},
+    author={Aditya Kusupati and Gantavya Bhatt and Aniket Rege and Matthew Wallingford and Aditya Sinha and Vivek Ramanujan and William Howard-Snyder and Kaifeng Chen and Sham Kakade and Prateek Jain and Ali Farhadi},
+    year={2024},
+    eprint={2205.13147},
+    archivePrefix={arXiv},
+    primaryClass={cs.LG}
+}
+```
+
+#### MultipleNegativesRankingLoss
+```bibtex
+@misc{henderson2017efficient,
+    title={Efficient Natural Language Response Suggestion for Smart Reply},
+    author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
+    year={2017},
+    eprint={1705.00652},
+    archivePrefix={arXiv},
+    primaryClass={cs.CL}
+}
+```
+
+<!--
+## Glossary
+
+*Clearly define terms in order to be accessible across audiences.*
+-->
+
+<!--
+## Model Card Authors
+
+*Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction.*
+-->
+
+<!--
+## Model Card Contact
+
+*Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.*
+-->

added_tokens.json

@@ -0,0 +1,24 @@
+{
+  "</tool_call>": 151658,
+  "<tool_call>": 151657,
+  "<|box_end|>": 151649,
+  "<|box_start|>": 151648,
+  "<|endoftext|>": 151643,
+  "<|file_sep|>": 151664,
+  "<|fim_middle|>": 151660,
+  "<|fim_pad|>": 151662,
+  "<|fim_prefix|>": 151659,
+  "<|fim_suffix|>": 151661,
+  "<|im_end|>": 151645,
+  "<|im_start|>": 151644,
+  "<|image_pad|>": 151655,
+  "<|object_ref_end|>": 151647,
+  "<|object_ref_start|>": 151646,
+  "<|quad_end|>": 151651,
+  "<|quad_start|>": 151650,
+  "<|repo_name|>": 151663,
+  "<|video_pad|>": 151656,
+  "<|vision_end|>": 151653,
+  "<|vision_pad|>": 151654,
+  "<|vision_start|>": 151652
+}

config.json

@@ -0,0 +1,29 @@
+{
+  "_name_or_path": "output/matryoshka_nli_Qwen-Qwen2.5-0.5B-Instruct-2024-11-15_15-50-44/checkpoint-1000",
+  "architectures": [
+    "Qwen2Model"
+  ],
+  "attention_dropout": 0.0,
+  "bos_token_id": 151643,
+  "eos_token_id": 151645,
+  "hidden_act": "silu",
+  "hidden_size": 896,
+  "initializer_range": 0.02,
+  "intermediate_size": 4864,
+  "max_position_embeddings": 32768,
+  "max_window_layers": 21,
+  "model_type": "qwen2",
+  "num_attention_heads": 14,
+  "num_hidden_layers": 24,
+  "num_key_value_heads": 2,
+  "rms_norm_eps": 1e-06,
+  "rope_scaling": null,
+  "rope_theta": 1000000.0,
+  "sliding_window": null,
+  "tie_word_embeddings": true,
+  "torch_dtype": "float32",
+  "transformers_version": "4.46.2",
+  "use_cache": true,
+  "use_sliding_window": false,
+  "vocab_size": 151936
+}

config_sentence_transformers.json

@@ -0,0 +1,10 @@
+{
+  "__version__": {
+    "sentence_transformers": "3.3.0",
+    "transformers": "4.46.2",
+    "pytorch": "2.1.0+cu118"
+  },
+  "prompts": {},
+  "default_prompt_name": null,
+  "similarity_fn_name": "cosine"
+}

model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:178eab352f9baa7f648f04c24ecbf7e5fbd5cb3384f4b309278e0e90d45070e4
+size 1976161736

modules.json

@@ -0,0 +1,14 @@
+[
+  {
+    "idx": 0,
+    "name": "0",
+    "path": "",
+    "type": "sentence_transformers.models.Transformer"
+  },
+  {
+    "idx": 1,
+    "name": "1",
+    "path": "1_Pooling",
+    "type": "sentence_transformers.models.Pooling"
+  }
+]

sentence_bert_config.json

@@ -0,0 +1,4 @@
+{
+  "max_seq_length": 1024,
+  "do_lower_case": false
+}

special_tokens_map.json

@@ -0,0 +1,31 @@
+{
+  "additional_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>",
+    "<|object_ref_start|>",
+    "<|object_ref_end|>",
+    "<|box_start|>",
+    "<|box_end|>",
+    "<|quad_start|>",
+    "<|quad_end|>",
+    "<|vision_start|>",
+    "<|vision_end|>",
+    "<|vision_pad|>",
+    "<|image_pad|>",
+    "<|video_pad|>"
+  ],
+  "eos_token": {
+    "content": "<|im_end|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "<|endoftext|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

tokenizer.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:08e9a56ecf7013f6cbd7c201447cfb8e0f78e53b28ba234afa66a8904c817250
+size 11422163

tokenizer_config.json

@@ -0,0 +1,214 @@
+{
+  "add_bos_token": false,
+  "add_prefix_space": false,
+  "added_tokens_decoder": {
+    "151643": {
+      "content": "<|endoftext|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151644": {
+      "content": "<|im_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151645": {
+      "content": "<|im_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151646": {
+      "content": "<|object_ref_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151647": {
+      "content": "<|object_ref_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151648": {
+      "content": "<|box_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151649": {
+      "content": "<|box_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151650": {
+      "content": "<|quad_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151651": {
+      "content": "<|quad_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151652": {
+      "content": "<|vision_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151653": {
+      "content": "<|vision_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151654": {
+      "content": "<|vision_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151655": {
+      "content": "<|image_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151656": {
+      "content": "<|video_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151657": {
+      "content": "<tool_call>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151658": {
+      "content": "</tool_call>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151659": {
+      "content": "<|fim_prefix|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151660": {
+      "content": "<|fim_middle|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151661": {
+      "content": "<|fim_suffix|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151662": {
+      "content": "<|fim_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151663": {
+      "content": "<|repo_name|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151664": {
+      "content": "<|file_sep|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    }
+  },
+  "additional_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>",
+    "<|object_ref_start|>",
+    "<|object_ref_end|>",
+    "<|box_start|>",
+    "<|box_end|>",
+    "<|quad_start|>",
+    "<|quad_end|>",
+    "<|vision_start|>",
+    "<|vision_end|>",
+    "<|vision_pad|>",
+    "<|image_pad|>",
+    "<|video_pad|>"
+  ],
+  "bos_token": null,
+  "chat_template": "{%- if tools %}\n    {{- '<|im_start|>system\\n' }}\n    {%- if messages[0]['role'] == 'system' %}\n        {{- messages[0]['content'] }}\n    {%- else %}\n        {{- 'You are Qwen, created by Alibaba Cloud. You are a helpful assistant.' }}\n    {%- endif %}\n    {{- \"\\n\\n# Tools\\n\\nYou may call one or more functions to assist with the user query.\\n\\nYou are provided with function signatures within <tools></tools> XML tags:\\n<tools>\" }}\n    {%- for tool in tools %}\n        {{- \"\\n\" }}\n        {{- tool | tojson }}\n    {%- endfor %}\n    {{- \"\\n</tools>\\n\\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\\n<tool_call>\\n{\\\"name\\\": <function-name>, \\\"arguments\\\": <args-json-object>}\\n</tool_call><|im_end|>\\n\" }}\n{%- else %}\n    {%- if messages[0]['role'] == 'system' %}\n        {{- '<|im_start|>system\\n' + messages[0]['content'] + '<|im_end|>\\n' }}\n    {%- else %}\n        {{- '<|im_start|>system\\nYou are Qwen, created by Alibaba Cloud. You are a helpful assistant.<|im_end|>\\n' }}\n    {%- endif %}\n{%- endif %}\n{%- for message in messages %}\n    {%- if (message.role == \"user\") or (message.role == \"system\" and not loop.first) or (message.role == \"assistant\" and not message.tool_calls) %}\n        {{- '<|im_start|>' + message.role + '\\n' + message.content + '<|im_end|>' + '\\n' }}\n    {%- elif message.role == \"assistant\" %}\n        {{- '<|im_start|>' + message.role }}\n        {%- if message.content %}\n            {{- '\\n' + message.content }}\n        {%- endif %}\n        {%- for tool_call in message.tool_calls %}\n            {%- if tool_call.function is defined %}\n                {%- set tool_call = tool_call.function %}\n            {%- endif %}\n            {{- '\\n<tool_call>\\n{\"name\": \"' }}\n            {{- tool_call.name }}\n            {{- '\", \"arguments\": ' }}\n            {{- tool_call.arguments | tojson }}\n            {{- '}\\n</tool_call>' }}\n        {%- endfor %}\n        {{- '<|im_end|>\\n' }}\n    {%- elif message.role == \"tool\" %}\n        {%- if (loop.index0 == 0) or (messages[loop.index0 - 1].role != \"tool\") %}\n            {{- '<|im_start|>user' }}\n        {%- endif %}\n        {{- '\\n<tool_response>\\n' }}\n        {{- message.content }}\n        {{- '\\n</tool_response>' }}\n        {%- if loop.last or (messages[loop.index0 + 1].role != \"tool\") %}\n            {{- '<|im_end|>\\n' }}\n        {%- endif %}\n    {%- endif %}\n{%- endfor %}\n{%- if add_generation_prompt %}\n    {{- '<|im_start|>assistant\\n' }}\n{%- endif %}\n",
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|im_end|>",
+  "errors": "replace",
+  "max_length": 1024,
+  "model_max_length": 1024,
+  "pad_to_multiple_of": null,
+  "pad_token": "<|endoftext|>",
+  "pad_token_type_id": 0,
+  "padding_side": "right",
+  "split_special_tokens": false,
+  "stride": 0,
+  "tokenizer_class": "Qwen2Tokenizer",
+  "truncation_side": "right",
+  "truncation_strategy": "longest_first",
+  "unk_token": null
+}