---
base_model: sentence-transformers/stsb-distilbert-base
library_name: sentence-transformers
metrics:
- cosine_accuracy@1
- cosine_accuracy@3
- cosine_accuracy@5
- cosine_accuracy@10
- cosine_precision@1
- cosine_precision@3
- cosine_precision@5
- cosine_precision@10
- cosine_recall@1
- cosine_recall@3
- cosine_recall@5
- cosine_recall@10
- cosine_ndcg@10
- cosine_mrr@10
- cosine_map@100
- dot_accuracy@1
- dot_accuracy@3
- dot_accuracy@5
- dot_accuracy@10
- dot_precision@1
- dot_precision@3
- dot_precision@5
- dot_precision@10
- dot_recall@1
- dot_recall@3
- dot_recall@5
- dot_recall@10
- dot_ndcg@10
- dot_mrr@10
- dot_map@100
pipeline_tag: sentence-similarity
tags:
- sentence-transformers
- sentence-similarity
- feature-extraction
- generated_from_trainer
- dataset_size:622302
- loss:MultipleNegativesRankingLoss
widget:
- source_sentence: Does fTO Genotype interact with Improvement in Aerobic Fitness
on Body Weight Loss During Lifestyle Intervention?
sentences:
- The study population count 46 550 male workers, 1670 (3.6%) of whom incurred at
least one work-related injury requiring admission to hospital within a period
of 5 years following hearing tests conducted between 1987 and 2005. The noise
exposure and hearing loss-related data were gathered during occupational noise-induced
hearing loss (NIHL) screening. The hospital data were used to identify all members
of the study population who were admitted, and the reason for admission. Finally,
access to the death-related data made it possible to identify participants who
died during the course of the study. Cox proportional hazards model taking into
account hearing status, noise levels, age and cumulative duration of noise exposure
at the time of the hearing test established the risk of work-related injuries
leading to admission to hospital.
- Carriers of a hereditary mutation in BRCA are at high risk for breast and ovarian
cancer. The first person from a family known to carry the mutation, the index
person, has to share genetic information with relatives. This study is aimed at
determining the number of relatives tested for a BRCA mutation, and the exploration
of facilitating and debilitating factors in the transmission of genetic information
from index patient to relatives.
- Not every participant responds with a comparable body weight loss to lifestyle
intervention, despite the same compliance. Genetic factors may explain parts of
this difference. Variation in fat mass and obesity-associated gene (FTO) is the
strongest common genetic determinant of body weight. The aim of the present study
was to evaluate the impact of FTO genotype differences in the link between improvement
of fitness and reduction of body weight during a lifestyle intervention.
- source_sentence: Is family history of exceptional longevity associated with lower
serum uric acid levels in Ashkenazi Jews?
sentences:
- To evaluate the effect of fasting on gastric emptying in mice.
- To test whether lower serum uric acid (UA) levels are associated with longevity
independent of renal function.
- Inducible NOS mRNA expression was significantly lower in CF patients with and
without bacterial infection than in healthy children (0.22 and 0.23 v 0.76; p=0.002
and p=0.01, respectively). Low levels of iNOS gene expression were accompanied
by low levels of iNOS protein expression as detected by Western blot analysis.
- source_sentence: Do hepatocellular carcinomas compromise quantitative tests of liver
function?
sentences:
- MEPE had no effect on glomerular filtration rate or single-nephron filtration
rate, but it increased phosphate excretion significantly. In animals infused with
vehicle alone (time controls), no significant change was seen in either the proximal
tubular fluid:plasma phosphate concentration ratio (TF/P(Pi)) or the fraction
of filtered phosphate reaching the late proximal convoluted tubule (FD(Pi)); whereas
in rats infused with MEPE, TF/P(Pi) increased from 0.49 ± 0.07 to 0.68 ± 0.04
(n = 22; P = 0.01) and FD(Pi) increased from 0.20 ± 0.03 to 0.33 ± 0.03 (n = 22;
P < 0.01).
- Hepatocellular carcinoma, which usually develops in cirrhotic livers, is one of
the most frequent cancers worldwide. If and how far hepatoma growth influences
liver function is unclear. Therefore, we compared a broad panel of quantitative
tests of liver function in cirrhotic patients with and without hepatocellular
carcinoma.
- A study was undertaken to measure cough frequency in children with stable asthma
using a validated monitoring device, and to assess the correlation between cough
frequency and the degree and type of airway inflammation.
- source_sentence: Does hand-assisted laparoscopic digestive surgery provide safety
and tactile sensation for malignancy or obesity?
sentences:
- In human aortic endothelial cells (HAECs) exposed to high glucose and aortas of
diabetic mice, activation of p66(Shc) by protein kinase C βII (PKCβII) persisted
after returning to normoglycemia. Persistent p66(Shc) upregulation and mitochondrial
translocation were associated with continued reactive oxygen species (ROS) production,
reduced nitric oxide bioavailability, and apoptosis. We show that p66(Shc) gene
overexpression was epigenetically regulated by promoter CpG hypomethylation and
general control nonderepressible 5-induced histone 3 acetylation. Furthermore,
p66(Shc)-derived ROS production maintained PKCβII upregulation and PKCβII-dependent
inhibitory phosphorylation of endothelial nitric oxide synthase at Thr-495, leading
to a detrimental vicious cycle despite restoration of normoglycemia. Moreover,
p66(Shc) activation accounted for the persistent elevation of the advanced glycated
end product precursor methylglyoxal. In vitro and in vivo gene silencing of p66(Shc),
performed at the time of glucose normalization, blunted ROS production, restored
endothelium-dependent vasorelaxation, and attenuated apoptosis by limiting cytochrome
c release, caspase 3 activity, and cleavage of poly (ADP-ribose) polymerase.
- Recently, 13 of our patients underwent hand-assisted advanced laparoscopic surgery
using this device. In this series, we had two cases of gastrectomy, two cases
of gastric bypass for morbid obesity, two Whipple cases for periampullary tumor,
and seven cases of bowel resection. On the basis of this series, we were able
to assess the utility of this device.
- 'Healthy men and women (n = 13; age: 48 +/- 17 y) were studied on 2 occasions:
after > or = 48 h with no exercise and 17 h after a 60-min bout of endurance exercise.
During each trial, brachial artery flow mediated dilation (FMD) was used to assess
endothelial function before and after the ingestion of a candy bar and soft drink.
Glucose, insulin, and thiobarbituric acid-reactive substances (TBARS), a marker
of oxidative stress, were measured in blood obtained during each FMD measurement.
The insulin sensitivity index was calculated from the glucose and insulin data.'
- source_sentence: Do correlations between plasma-neuropeptides and temperament dimensions
differ between suicidal patients and healthy controls?
sentences:
- Decreased plasma levels of plasma-neuropeptide Y (NPY) and plasma-corticotropin
releasing hormone (CRH), and increased levels of plasma delta-sleep inducing peptide
(DSIP) in suicide attempters with mood disorders have previously been observed.
This study was performed in order to further understand the clinical relevance
of these findings.
- Brain death was induced in Wistar rats by intracranial balloon inflation. Pulmonary
capillary leak was estimated using radioiodinated albumin. Development of pulmonary
edema was assessed by measurement of wet and dry lung weights. Cell surface expression
of CD11b/CD18 by neutrophils was determined using flow cytometry. Enzyme-linked
immunosorbent assays were used to measure the levels of TNFalpha, IL-1beta, CINC-1,
and CINC-3 in serum and bronchoalveolar lavage. Quantitative reverse-transcription
polymerase chain reaction was used to determine the expression of cytokine mRNA
(IL-1beta, CINC-1 and CINC-3) in lung tissue.
- 'Seven hundred fifty patients entered the study. One hundred sixty-eight patients
(22.4%) presented with a total of 193 extracutaneous manifestations, as follows:
articular (47.2%), neurologic (17.1%), vascular (9.3%), ocular (8.3%), gastrointestinal
(6.2%), respiratory (2.6%), cardiac (1%), and renal (1%). Other autoimmune conditions
were present in 7.3% of patients. Neurologic involvement consisted of epilepsy,
central nervous system vasculitis, peripheral neuropathy, vascular malformations,
headache, and neuroimaging abnormalities. Ocular manifestations were episcleritis,
uveitis, xerophthalmia, glaucoma, and papilledema. In more than one-fourth of
these children, articular, neurologic, and ocular involvements were unrelated
to the site of skin lesions. Raynaud''s phenomenon was reported in 16 patients.
Respiratory involvement consisted essentially of restrictive lung disease. Gastrointestinal
involvement was reported in 12 patients and consisted exclusively of gastroesophageal
reflux. Thirty patients (4%) had multiple extracutaneous features, but systemic
sclerosis (SSc) developed in only 1 patient. In patients with extracutaneous involvement,
the prevalence of antinuclear antibodies and rheumatoid factor was significantly
higher than that among patients with only skin involvement. However, Scl-70 and
anticentromere, markers of SSc, were not significantly increased.'
model-index:
- name: SentenceTransformer based on sentence-transformers/stsb-distilbert-base
results:
- task:
type: information-retrieval
name: Information Retrieval
dataset:
name: med eval dev
type: med-eval-dev
metrics:
- type: cosine_accuracy@1
value: 0.9825
name: Cosine Accuracy@1
- type: cosine_accuracy@3
value: 0.998
name: Cosine Accuracy@3
- type: cosine_accuracy@5
value: 0.9985
name: Cosine Accuracy@5
- type: cosine_accuracy@10
value: 0.9985
name: Cosine Accuracy@10
- type: cosine_precision@1
value: 0.9825
name: Cosine Precision@1
- type: cosine_precision@3
value: 0.8438333333333332
name: Cosine Precision@3
- type: cosine_precision@5
value: 0.5588
name: Cosine Precision@5
- type: cosine_precision@10
value: 0.29309999999999997
name: Cosine Precision@10
- type: cosine_recall@1
value: 0.3413382936507936
name: Cosine Recall@1
- type: cosine_recall@3
value: 0.8453946428571428
name: Cosine Recall@3
- type: cosine_recall@5
value: 0.9191847222222223
name: Cosine Recall@5
- type: cosine_recall@10
value: 0.9578416666666667
name: Cosine Recall@10
- type: cosine_ndcg@10
value: 0.9461928701093355
name: Cosine Ndcg@10
- type: cosine_mrr@10
value: 0.9899583333333333
name: Cosine Mrr@10
- type: cosine_map@100
value: 0.9168772609607218
name: Cosine Map@100
- type: dot_accuracy@1
value: 0.9705
name: Dot Accuracy@1
- type: dot_accuracy@3
value: 0.9955
name: Dot Accuracy@3
- type: dot_accuracy@5
value: 0.9985
name: Dot Accuracy@5
- type: dot_accuracy@10
value: 0.999
name: Dot Accuracy@10
- type: dot_precision@1
value: 0.9705
name: Dot Precision@1
- type: dot_precision@3
value: 0.8141666666666666
name: Dot Precision@3
- type: dot_precision@5
value: 0.546
name: Dot Precision@5
- type: dot_precision@10
value: 0.28995
name: Dot Precision@10
- type: dot_recall@1
value: 0.3365662698412698
name: Dot Recall@1
- type: dot_recall@3
value: 0.8156482142857142
name: Dot Recall@3
- type: dot_recall@5
value: 0.8994174603174604
name: Dot Recall@5
- type: dot_recall@10
value: 0.9480904761904763
name: Dot Recall@10
- type: dot_ndcg@10
value: 0.9297315742366127
name: Dot Ndcg@10
- type: dot_mrr@10
value: 0.9828083333333333
name: Dot Mrr@10
- type: dot_map@100
value: 0.8926507948277561
name: Dot Map@100
---
# SentenceTransformer based on sentence-transformers/stsb-distilbert-base
This is a [sentence-transformers](https://www.SBERT.net) model finetuned from [sentence-transformers/stsb-distilbert-base](https://huggingface.co/sentence-transformers/stsb-distilbert-base). It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.
## Model Details
### Model Description
- **Model Type:** Sentence Transformer
- **Base model:** [sentence-transformers/stsb-distilbert-base](https://huggingface.co/sentence-transformers/stsb-distilbert-base)
- **Maximum Sequence Length:** 128 tokens
- **Output Dimensionality:** 768 tokens
- **Similarity Function:** Cosine Similarity
### Model Sources
- **Documentation:** [Sentence Transformers Documentation](https://sbert.net)
- **Repository:** [Sentence Transformers on GitHub](https://github.com/UKPLab/sentence-transformers)
- **Hugging Face:** [Sentence Transformers on Hugging Face](https://huggingface.co/models?library=sentence-transformers)
### Full Model Architecture
```
SentenceTransformer(
(0): Transformer({'max_seq_length': 128, 'do_lower_case': False}) with Transformer model: DistilBertModel
(1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
)
```
## Usage
### Direct Usage (Sentence Transformers)
First install the Sentence Transformers library:
```bash
pip install -U sentence-transformers
```
Then you can load this model and run inference.
```python
from sentence_transformers import SentenceTransformer
# Download from the 🤗 Hub
model = SentenceTransformer("alpha-brain/stsb-distilbert-base-mnrl")
# Run inference
sentences = [
'Do correlations between plasma-neuropeptides and temperament dimensions differ between suicidal patients and healthy controls?',
'Decreased plasma levels of plasma-neuropeptide Y (NPY) and plasma-corticotropin releasing hormone (CRH), and increased levels of plasma delta-sleep inducing peptide (DSIP) in suicide attempters with mood disorders have previously been observed. This study was performed in order to further understand the clinical relevance of these findings.',
"Seven hundred fifty patients entered the study. One hundred sixty-eight patients (22.4%) presented with a total of 193 extracutaneous manifestations, as follows: articular (47.2%), neurologic (17.1%), vascular (9.3%), ocular (8.3%), gastrointestinal (6.2%), respiratory (2.6%), cardiac (1%), and renal (1%). Other autoimmune conditions were present in 7.3% of patients. Neurologic involvement consisted of epilepsy, central nervous system vasculitis, peripheral neuropathy, vascular malformations, headache, and neuroimaging abnormalities. Ocular manifestations were episcleritis, uveitis, xerophthalmia, glaucoma, and papilledema. In more than one-fourth of these children, articular, neurologic, and ocular involvements were unrelated to the site of skin lesions. Raynaud's phenomenon was reported in 16 patients. Respiratory involvement consisted essentially of restrictive lung disease. Gastrointestinal involvement was reported in 12 patients and consisted exclusively of gastroesophageal reflux. Thirty patients (4%) had multiple extracutaneous features, but systemic sclerosis (SSc) developed in only 1 patient. In patients with extracutaneous involvement, the prevalence of antinuclear antibodies and rheumatoid factor was significantly higher than that among patients with only skin involvement. However, Scl-70 and anticentromere, markers of SSc, were not significantly increased.",
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 768]
# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]
```
## Evaluation
### Metrics
#### Information Retrieval
* Dataset: `med-eval-dev`
* Evaluated with [InformationRetrievalEvaluator](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.InformationRetrievalEvaluator)
| Metric | Value |
|:--------------------|:-----------|
| cosine_accuracy@1 | 0.9825 |
| cosine_accuracy@3 | 0.998 |
| cosine_accuracy@5 | 0.9985 |
| cosine_accuracy@10 | 0.9985 |
| cosine_precision@1 | 0.9825 |
| cosine_precision@3 | 0.8438 |
| cosine_precision@5 | 0.5588 |
| cosine_precision@10 | 0.2931 |
| cosine_recall@1 | 0.3413 |
| cosine_recall@3 | 0.8454 |
| cosine_recall@5 | 0.9192 |
| cosine_recall@10 | 0.9578 |
| cosine_ndcg@10 | 0.9462 |
| cosine_mrr@10 | 0.99 |
| **cosine_map@100** | **0.9169** |
| dot_accuracy@1 | 0.9705 |
| dot_accuracy@3 | 0.9955 |
| dot_accuracy@5 | 0.9985 |
| dot_accuracy@10 | 0.999 |
| dot_precision@1 | 0.9705 |
| dot_precision@3 | 0.8142 |
| dot_precision@5 | 0.546 |
| dot_precision@10 | 0.2899 |
| dot_recall@1 | 0.3366 |
| dot_recall@3 | 0.8156 |
| dot_recall@5 | 0.8994 |
| dot_recall@10 | 0.9481 |
| dot_ndcg@10 | 0.9297 |
| dot_mrr@10 | 0.9828 |
| dot_map@100 | 0.8927 |
## Training Details
### Training Dataset
#### Unnamed Dataset
* Size: 622,302 training samples
* Columns: question and contexts
* Approximate statistics based on the first 1000 samples:
| | question | contexts |
|:--------|:----------------------------------------------------------------------------------|:-----------------------------------------------------------------------------------|
| type | string | string |
| details | - min: 9 tokens
- mean: 27.35 tokens
- max: 60 tokens
| - min: 5 tokens
- mean: 88.52 tokens
- max: 128 tokens
|
* Samples:
| question | contexts |
|:--------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| Does low-level human equivalent gestational lead exposure produce sex-specific motor and coordination abnormalities and late-onset obesity in year-old mice? | Low-level developmental lead exposure is linked to cognitive and neurological disorders in children. However, the long-term effects of gestational lead exposure (GLE) have received little attention. |
| Does insulin in combination with selenium inhibit HG/Pal-induced cardiomyocyte apoptosis by Cbl-b regulating p38MAPK/CBP/Ku70 pathway? | In this study, we investigated whether insulin and selenium in combination (In/Se) suppresses cardiomyocyte apoptosis and whether this protection is mediated by Cbl-b regulating p38MAPK/CBP/Ku70 pathway. |
| Does arthroscopic subacromial decompression result in normal shoulder function after two years in less than 50 % of patients? | The aim of this study was to evaluate the outcome two years after arthroscopic subacromial decompression using the Western Ontario Rotator-Cuff (WORC) index and a diagram-based questionnaire to self-assess active shoulder range of motion (ROM). |
* Loss: [MultipleNegativesRankingLoss](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#multiplenegativesrankingloss) with these parameters:
```json
{
"scale": 20.0,
"similarity_fct": "cos_sim"
}
```
### Evaluation Dataset
#### Unnamed Dataset
* Size: 32,753 evaluation samples
* Columns: question and contexts
* Approximate statistics based on the first 1000 samples:
| | question | contexts |
|:--------|:-----------------------------------------------------------------------------------|:-----------------------------------------------------------------------------------|
| type | string | string |
| details | - min: 11 tokens
- mean: 27.52 tokens
- max: 56 tokens
| - min: 3 tokens
- mean: 88.59 tokens
- max: 128 tokens
|
* Samples:
| question | contexts |
|:---------------------------------------------------------------------------------------------------------------------------------------------------------------|:--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| Does [ Chemical components from essential oil of Pandanus amaryllifolius leave ]? | The essential oil of Pandanus amaryllifolius leaves was analyzed by gas chromatography-mass spectrum, and the relative content of each component was determined by area normalization method. |
| Is elevated C-reactive protein associated with the tumor depth of invasion but not with disease recurrence in stage II and III colorectal cancer? | We previously demonstrated that elevated serum C-reactive protein (CRP) level is associated with depth of tumor invasion in operable colorectal cancer. There is also increasing evidence to show that raised CRP concentration is associated with poor survival in patients with colorectal cancer. The purpose of this study was to investigate the correlation between preoperative CRP concentrations and short-term disease recurrence in cases with stage II and III colorectal cancer. |
| Do neuropeptide Y and peptide YY protect from weight loss caused by Bacille Calmette-Guérin in mice? | Deletion of PYY and NPY aggravated the BCG-induced loss of body weight, which was most pronounced in NPY-/-;PYY-/- mice (maximum loss: 15%). The weight loss in NPY-/-;PYY-/- mice did not normalize during the 2 week observation period. BCG suppressed the circadian pattern of locomotion, exploration and food intake. However, these changes took a different time course than the prolonged weight loss caused by BCG in NPY-/-;PYY-/- mice. The effect of BCG to increase circulating IL-6 (measured 16 days post-treatment) remained unaltered by knockout of PYY, NPY or NPY plus PYY. |
* Loss: [MultipleNegativesRankingLoss](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#multiplenegativesrankingloss) with these parameters:
```json
{
"scale": 20.0,
"similarity_fct": "cos_sim"
}
```
### Training Hyperparameters
#### Non-Default Hyperparameters
- `eval_strategy`: steps
- `per_device_train_batch_size`: 64
- `num_train_epochs`: 1
#### All Hyperparameters
Click to expand
- `overwrite_output_dir`: False
- `do_predict`: False
- `eval_strategy`: steps
- `prediction_loss_only`: True
- `per_device_train_batch_size`: 64
- `per_device_eval_batch_size`: 8
- `per_gpu_train_batch_size`: None
- `per_gpu_eval_batch_size`: None
- `gradient_accumulation_steps`: 1
- `eval_accumulation_steps`: None
- `torch_empty_cache_steps`: None
- `learning_rate`: 5e-05
- `weight_decay`: 0.0
- `adam_beta1`: 0.9
- `adam_beta2`: 0.999
- `adam_epsilon`: 1e-08
- `max_grad_norm`: 1.0
- `num_train_epochs`: 1
- `max_steps`: -1
- `lr_scheduler_type`: linear
- `lr_scheduler_kwargs`: {}
- `warmup_ratio`: 0.0
- `warmup_steps`: 0
- `log_level`: passive
- `log_level_replica`: warning
- `log_on_each_node`: True
- `logging_nan_inf_filter`: True
- `save_safetensors`: True
- `save_on_each_node`: False
- `save_only_model`: False
- `restore_callback_states_from_checkpoint`: False
- `no_cuda`: False
- `use_cpu`: False
- `use_mps_device`: False
- `seed`: 42
- `data_seed`: None
- `jit_mode_eval`: False
- `use_ipex`: False
- `bf16`: False
- `fp16`: False
- `fp16_opt_level`: O1
- `half_precision_backend`: auto
- `bf16_full_eval`: False
- `fp16_full_eval`: False
- `tf32`: None
- `local_rank`: 0
- `ddp_backend`: None
- `tpu_num_cores`: None
- `tpu_metrics_debug`: False
- `debug`: []
- `dataloader_drop_last`: False
- `dataloader_num_workers`: 0
- `dataloader_prefetch_factor`: None
- `past_index`: -1
- `disable_tqdm`: False
- `remove_unused_columns`: True
- `label_names`: None
- `load_best_model_at_end`: False
- `ignore_data_skip`: False
- `fsdp`: []
- `fsdp_min_num_params`: 0
- `fsdp_config`: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
- `fsdp_transformer_layer_cls_to_wrap`: None
- `accelerator_config`: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
- `deepspeed`: None
- `label_smoothing_factor`: 0.0
- `optim`: adamw_torch
- `optim_args`: None
- `adafactor`: False
- `group_by_length`: False
- `length_column_name`: length
- `ddp_find_unused_parameters`: None
- `ddp_bucket_cap_mb`: None
- `ddp_broadcast_buffers`: False
- `dataloader_pin_memory`: True
- `dataloader_persistent_workers`: False
- `skip_memory_metrics`: True
- `use_legacy_prediction_loop`: False
- `push_to_hub`: False
- `resume_from_checkpoint`: None
- `hub_model_id`: None
- `hub_strategy`: every_save
- `hub_private_repo`: False
- `hub_always_push`: False
- `gradient_checkpointing`: False
- `gradient_checkpointing_kwargs`: None
- `include_inputs_for_metrics`: False
- `eval_do_concat_batches`: True
- `fp16_backend`: auto
- `push_to_hub_model_id`: None
- `push_to_hub_organization`: None
- `mp_parameters`:
- `auto_find_batch_size`: False
- `full_determinism`: False
- `torchdynamo`: None
- `ray_scope`: last
- `ddp_timeout`: 1800
- `torch_compile`: False
- `torch_compile_backend`: None
- `torch_compile_mode`: None
- `dispatch_batches`: None
- `split_batches`: None
- `include_tokens_per_second`: False
- `include_num_input_tokens_seen`: False
- `neftune_noise_alpha`: None
- `optim_target_modules`: None
- `batch_eval_metrics`: False
- `eval_on_start`: False
- `eval_use_gather_object`: False
- `batch_sampler`: batch_sampler
- `multi_dataset_batch_sampler`: proportional
Click to expand
| Epoch | Step | Training Loss | loss | med-eval-dev_cosine_map@100 |
|:------:|:----:|:-------------:|:------:|:---------------------------:|
| 0 | 0 | - | - | 0.3328 |
| 0.0103 | 100 | 0.7953 | - | - |
| 0.0206 | 200 | 0.5536 | - | - |
| 0.0257 | 250 | - | 0.1041 | 0.7474 |
| 0.0309 | 300 | 0.4755 | - | - |
| 0.0411 | 400 | 0.4464 | - | - |
| 0.0514 | 500 | 0.3986 | 0.0761 | 0.7786 |
| 0.0617 | 600 | 0.357 | - | - |
| 0.0720 | 700 | 0.3519 | - | - |
| 0.0771 | 750 | - | 0.0685 | 0.8029 |
| 0.0823 | 800 | 0.3197 | - | - |
| 0.0926 | 900 | 0.3247 | - | - |
| 0.1028 | 1000 | 0.3048 | 0.0549 | 0.8108 |
| 0.1131 | 1100 | 0.2904 | - | - |
| 0.1234 | 1200 | 0.281 | - | - |
| 0.1285 | 1250 | - | 0.0503 | 0.8181 |
| 0.1337 | 1300 | 0.2673 | - | - |
| 0.1440 | 1400 | 0.2645 | - | - |
| 0.1543 | 1500 | 0.2511 | 0.0457 | 0.8332 |
| 0.1645 | 1600 | 0.2541 | - | - |
| 0.1748 | 1700 | 0.2614 | - | - |
| 0.1800 | 1750 | - | 0.0401 | 0.8380 |
| 0.1851 | 1800 | 0.2263 | - | - |
| 0.1954 | 1900 | 0.2466 | - | - |
| 0.2057 | 2000 | 0.2297 | 0.0365 | 0.8421 |
| 0.2160 | 2100 | 0.2225 | - | - |
| 0.2262 | 2200 | 0.212 | - | - |
| 0.2314 | 2250 | - | 0.0344 | 0.8563 |
| 0.2365 | 2300 | 0.2257 | - | - |
| 0.2468 | 2400 | 0.1953 | - | - |
| 0.2571 | 2500 | 0.1961 | 0.0348 | 0.8578 |
| 0.2674 | 2600 | 0.1888 | - | - |
| 0.2777 | 2700 | 0.2039 | - | - |
| 0.2828 | 2750 | - | 0.0319 | 0.8610 |
| 0.2879 | 2800 | 0.1939 | - | - |
| 0.2982 | 2900 | 0.202 | - | - |
| 0.3085 | 3000 | 0.1915 | 0.0292 | 0.8678 |
| 0.3188 | 3100 | 0.1987 | - | - |
| 0.3291 | 3200 | 0.1877 | - | - |
| 0.3342 | 3250 | - | 0.0275 | 0.8701 |
| 0.3394 | 3300 | 0.1874 | - | - |
| 0.3497 | 3400 | 0.1689 | - | - |
| 0.3599 | 3500 | 0.169 | 0.0281 | 0.8789 |
| 0.3702 | 3600 | 0.1631 | - | - |
| 0.3805 | 3700 | 0.1611 | - | - |
| 0.3856 | 3750 | - | 0.0263 | 0.8814 |
| 0.3908 | 3800 | 0.1764 | - | - |
| 0.4011 | 3900 | 0.1796 | - | - |
| 0.4114 | 4000 | 0.1729 | 0.0249 | 0.8805 |
| 0.4216 | 4100 | 0.1551 | - | - |
| 0.4319 | 4200 | 0.1543 | - | - |
| 0.4371 | 4250 | - | 0.0241 | 0.8867 |
| 0.4422 | 4300 | 0.1549 | - | - |
| 0.4525 | 4400 | 0.1432 | - | - |
| 0.4628 | 4500 | 0.1592 | 0.0219 | 0.8835 |
| 0.4731 | 4600 | 0.1517 | - | - |
| 0.4833 | 4700 | 0.1463 | - | - |
| 0.4885 | 4750 | - | 0.0228 | 0.8928 |
| 0.4936 | 4800 | 0.1525 | - | - |
| 0.5039 | 4900 | 0.1426 | - | - |
| 0.5142 | 5000 | 0.1524 | 0.0209 | 0.8903 |
| 0.5245 | 5100 | 0.1443 | - | - |
| 0.5348 | 5200 | 0.1468 | - | - |
| 0.5399 | 5250 | - | 0.0212 | 0.8948 |
| 0.5450 | 5300 | 0.151 | - | - |
| 0.5553 | 5400 | 0.1443 | - | - |
| 0.5656 | 5500 | 0.1438 | 0.0212 | 0.8982 |
| 0.5759 | 5600 | 0.1409 | - | - |
| 0.5862 | 5700 | 0.1346 | - | - |
| 0.5913 | 5750 | - | 0.0207 | 0.8983 |
| 0.5965 | 5800 | 0.1315 | - | - |
| 0.6067 | 5900 | 0.1425 | - | - |
| 0.6170 | 6000 | 0.136 | 0.0188 | 0.8970 |
| 0.6273 | 6100 | 0.1426 | - | - |
| 0.6376 | 6200 | 0.1353 | - | - |
| 0.6427 | 6250 | - | 0.0185 | 0.8969 |
| 0.6479 | 6300 | 0.1269 | - | - |
| 0.6582 | 6400 | 0.1159 | - | - |
| 0.6684 | 6500 | 0.1311 | 0.0184 | 0.9028 |
| 0.6787 | 6600 | 0.1179 | - | - |
| 0.6890 | 6700 | 0.115 | - | - |
| 0.6942 | 6750 | - | 0.0184 | 0.9046 |
| 0.6993 | 6800 | 0.1254 | - | - |
| 0.7096 | 6900 | 0.1233 | - | - |
| 0.7199 | 7000 | 0.122 | 0.0174 | 0.9042 |
| 0.7302 | 7100 | 0.1238 | - | - |
| 0.7404 | 7200 | 0.1257 | - | - |
| 0.7456 | 7250 | - | 0.0175 | 0.9074 |
| 0.7507 | 7300 | 0.1222 | - | - |
| 0.7610 | 7400 | 0.1194 | - | - |
| 0.7713 | 7500 | 0.1284 | 0.0166 | 0.9080 |
| 0.7816 | 7600 | 0.1147 | - | - |
| 0.7919 | 7700 | 0.1182 | - | - |
| 0.7970 | 7750 | - | 0.0170 | 0.9116 |
| 0.8021 | 7800 | 0.1157 | - | - |
| 0.8124 | 7900 | 0.1299 | - | - |
| 0.8227 | 8000 | 0.114 | 0.0163 | 0.9105 |
| 0.8330 | 8100 | 0.1141 | - | - |
| 0.8433 | 8200 | 0.1195 | - | - |
| 0.8484 | 8250 | - | 0.0160 | 0.9112 |
| 0.8536 | 8300 | 0.1073 | - | - |
| 0.8638 | 8400 | 0.1044 | - | - |
| 0.8741 | 8500 | 0.1083 | 0.0160 | 0.9153 |
| 0.8844 | 8600 | 0.1103 | - | - |
| 0.8947 | 8700 | 0.1145 | - | - |
| 0.8998 | 8750 | - | 0.0154 | 0.9133 |
| 0.9050 | 8800 | 0.1083 | - | - |
| 0.9153 | 8900 | 0.1205 | - | - |
| 0.9255 | 9000 | 0.1124 | 0.0153 | 0.9162 |
| 0.9358 | 9100 | 0.1067 | - | - |
| 0.9461 | 9200 | 0.116 | - | - |
| 0.9513 | 9250 | - | 0.0152 | 0.9171 |
| 0.9564 | 9300 | 0.1126 | - | - |
| 0.9667 | 9400 | 0.1075 | - | - |
| 0.9770 | 9500 | 0.1128 | 0.0149 | 0.9169 |
| 0.9872 | 9600 | 0.1143 | - | - |
| 0.9975 | 9700 | 0.1175 | - | - |