TomDubois12/fine-tuned-model

SentenceTransformer based on sentence-transformers/all-distilroberta-v1

This is a sentence-transformers model finetuned from sentence-transformers/all-distilroberta-v1. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

• Model Type: Sentence Transformer
• Base model: sentence-transformers/all-distilroberta-v1
• Maximum Sequence Length: 512 tokens
• Output Dimensionality: 768 tokens
• Similarity Function: Cosine Similarity

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 512, 'do_lower_case': False}) with Transformer model: RobertaModel 
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Normalize()
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("TomDubois12/fine-tuned-model")
# Run inference
sentences = [
    'Panorama de l’existant sur les capteurs et analyseurs en ligne pour la mesure des parametres physico-chimiques dans l’eau',
    "Le travail de compilation des différents capteurs et analyseurs a été réalisé à partir de différentes sources d'information comme l'annuaire du Guide de l'eau, les sites web des sociétés et les salons professionnels. 71 fabricants ont ainsi été recensés. Un classement a été effectué en considérant: les sondes in situ et les capteurs (1 à 3 paramètres et 4 paramètres et plus), les analyseurs en ligne (avec et sans réactifs, in situ) et les appareils portables. Des retours d'expériences sur le fonctionnement des stations de mesure en continu ont été réalisés pour quatre types d'eau (les cours d'eau, les eaux souterraines, les eaux de rejets et les eaux marines) à travers des entretiens téléphoniques avec les gestionnaires des stations de mesure en France et via la littérature pour les stations situées en Europe. Il en ressort que la configuration de la grande majorité des stations est basée sur un pompage de l'eau dans un local technique par rapport aux stations autonomes in situ. Les paramètres qui sont le plus souvent mesurés sont le pH, la conductivité, l'oxygène dissous, la température, la turbidité, les nutriments (ammonium, nitrates, phosphates) et la matière organique (carbone organique, absorbance spécifique à 254 nm). En fonction des besoins, les micropolluants (notamment métaux, hydrocarbures et HAP), la chlorophylle et les cyanobactéries ainsi que la toxicité sont occasionnellement mesurés. D'une manière générale, les capteurs et analyseurs sont jugés robustes et fiables. Certaines difficultés ont pu être mises en évidence, par exemple les dérives pour les capteurs mesurant l'ammonium. La maintenance associée aux stations de mesure peut être très importante en termes de temps passé et de cout des réactifs. Des études en amont ont souvent été engagées pour vérifier la fiabilité des résultats obtenus, notamment à travers la comparaison avec des mesures de contrôle et des prélèvements suivis d'analyses en laboratoire. Enfin, certains gestionnaires ont mis en place des contrôles qualité rigoureux et fréquents, ceci afin de s'assurer du bon fonctionnement et de la stabilité des capteurs dans le temps.",
    'Bilayer graphene is an intriguing material in that its electronic structure can be altered by changing the stacking order or the relative twist angle, yielding a new class of low-dimensional carbon system. Twisted bilayer graphene can be obtained by (i) thermal decomposition of SiC; (ii) chemical vapor deposition (CVD) on metal catalysts; (iii) folding graphene; or (iv) stacking graphene layers one atop the other, the latter of which suffers from interlayer contamination. Existing synthesis protocols, however, usually result in graphene with polycrystalline structures. The present study investigates bilayer graphene grown by ambient pressure CVD on polycrystalline Cu. Controlling the nucleation in early stage growth allows the constituent layers to form single hexagonal crystals. New Raman active modes are shown to result from the twist, with the angle determined by transmission electron microscopy. The successful growth of single-crystal bilayer graphene provides an attractive jumping-off point for systematic studies of interlayer coupling in misoriented few-layer graphene systems with well-defined geometry.',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 768]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]

Training Details

Training Dataset

Unnamed Dataset

• Size: 4,224 training samples
• Columns: sentence_0, sentence_1, and label
• Approximate statistics based on the first 1000 samples:

  	sentence_0	sentence_1	label
  type	string	string	int
  details	min: 6 tokens mean: 21.55 tokens max: 86 tokens	min: 2 tokens mean: 177.38 tokens max: 512 tokens	0: ~67.00% 1: ~33.00%

• Samples:

  sentence_0	sentence_1	label
  High-Pressure Elastic Properties of Solid Argon to 70 GPa	The acoustic velocities, adiabatic elastic constants, bulk modulus, elastic anisotropy, Cauchy violation, and density in an ideal solid argon (Ar) have been determined at high pressures up to 70 GPa in a diamond anvil cell by making new approaches of Brillouin spectroscopy. These results place the first complete study for elastic properties of dense Ar and provide an improved basis for making the theoretical calculations of rare-gas solids over a wide range of compression.	1
  Direct Voltammetric Detection of DNA and pH Sensing on Epitaxial Graphene: An Insight into the Role of Oxygenated Defects	In this paper, we carried out detailed electrochemical studies of epitaxial graphene (EG) using inner-sphere and outer-sphere redox mediators. The EG sample was anodized systematically to investigate the effect of edge plane defects on the heterogeneous charge transfer kinetics and capacitive noise. We found that anodized EG, consisting of oxygen-related defects, is a superior biosensing platform for the detection of nucleic acids, uric acids (UA), dopamine (DA), and ascorbic acids (AA). Mixtures of nucleic acids (A, T, C, G) or biomolecules (AA, UA, DA) can be resolved as individual peaks using differential pulse voltammetry. In fact, an anodized EG voltammetric sensor can realize the simultaneous detection of all four DNA bases in double stranded DNA (dsDNA) without a prehydrolysis step, and it can also differentiate single stranded DNA from dsDNA. Our results show that graphene with high edge plane defects, as opposed to pristine graphene, is the choice platform in high resolution electrochemical sensing.	1
  Scanning Electrochemical Microscopy of Carbon Nanomaterials and Graphite	We present a comprehensive study of the chiral-index assignment of carbon nanotubes in aqueous suspensions by resonant Raman scattering of the radial breathing mode. We determine the energies of the first optical transition in metallic tubes and of the second optical transition in semiconducting tubes for more than 50 chiral indices. The assignment is unique and does not depend on empirical parameters. The systematics of the so-called branches in the Kataura plot are discussed; many properties of the tubes are similar for members of the same branch. We show how the radial breathing modes observed in a single Raman spectrum can be easily assigned based on these systematics. In addition, empirical fits provide the energies and radial breathing modes for all metallic and semiconducting nanotubes with diameters between 0.6 and 1.5 nm. We discuss the relation between the frequency of the radial breathing mode and tube diameter. Finally, from the Raman intensities we obtain information on the electron-phonon coupling.	0

• Loss: CosineSimilarityLoss with these parameters:

  {
      "loss_fct": "torch.nn.modules.loss.MSELoss"
  }
  

Training Hyperparameters

Non-Default Hyperparameters

• per_device_train_batch_size: 16
• per_device_eval_batch_size: 16
• multi_dataset_batch_sampler: round_robin

All Hyperparameters

Click to expand

• overwrite_output_dir: False
• do_predict: False
• eval_strategy: no
• prediction_loss_only: True
• per_device_train_batch_size: 16
• per_device_eval_batch_size: 16
• per_gpu_train_batch_size: None
• per_gpu_eval_batch_size: None
• gradient_accumulation_steps: 1
• eval_accumulation_steps: None
• torch_empty_cache_steps: None
• learning_rate: 5e-05
• weight_decay: 0.0
• adam_beta1: 0.9
• adam_beta2: 0.999
• adam_epsilon: 1e-08
• max_grad_norm: 1
• num_train_epochs: 3
• max_steps: -1
• lr_scheduler_type: linear
• lr_scheduler_kwargs: {}
• warmup_ratio: 0.0
• warmup_steps: 0
• log_level: passive
• log_level_replica: warning
• log_on_each_node: True
• logging_nan_inf_filter: True
• save_safetensors: True
• save_on_each_node: False
• save_only_model: False
• restore_callback_states_from_checkpoint: False
• no_cuda: False
• use_cpu: False
• use_mps_device: False
• seed: 42
• data_seed: None
• jit_mode_eval: False
• use_ipex: False
• bf16: False
• fp16: False
• fp16_opt_level: O1
• half_precision_backend: auto
• bf16_full_eval: False
• fp16_full_eval: False
• tf32: None
• local_rank: 0
• ddp_backend: None
• tpu_num_cores: None
• tpu_metrics_debug: False
• debug: []
• dataloader_drop_last: False
• dataloader_num_workers: 0
• dataloader_prefetch_factor: None
• past_index: -1
• disable_tqdm: False
• remove_unused_columns: True
• label_names: None
• load_best_model_at_end: False
• ignore_data_skip: False
• fsdp: []
• fsdp_min_num_params: 0
• fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
• fsdp_transformer_layer_cls_to_wrap: None
• accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
• deepspeed: None
• label_smoothing_factor: 0.0
• optim: adamw_torch
• optim_args: None
• adafactor: False
• group_by_length: False
• length_column_name: length
• ddp_find_unused_parameters: None
• ddp_bucket_cap_mb: None
• ddp_broadcast_buffers: False
• dataloader_pin_memory: True
• dataloader_persistent_workers: False
• skip_memory_metrics: True
• use_legacy_prediction_loop: False
• push_to_hub: False
• resume_from_checkpoint: None
• hub_model_id: None
• hub_strategy: every_save
• hub_private_repo: False
• hub_always_push: False
• gradient_checkpointing: False
• gradient_checkpointing_kwargs: None
• include_inputs_for_metrics: False
• eval_do_concat_batches: True
• fp16_backend: auto
• push_to_hub_model_id: None
• push_to_hub_organization: None
• mp_parameters:
• auto_find_batch_size: False
• full_determinism: False
• torchdynamo: None
• ray_scope: last
• ddp_timeout: 1800
• torch_compile: False
• torch_compile_backend: None
• torch_compile_mode: None
• dispatch_batches: None
• split_batches: None
• include_tokens_per_second: False
• include_num_input_tokens_seen: False
• neftune_noise_alpha: None
• optim_target_modules: None
• batch_eval_metrics: False
• eval_on_start: False
• use_liger_kernel: False
• eval_use_gather_object: False
• batch_sampler: batch_sampler
• multi_dataset_batch_sampler: round_robin

Training Logs

Epoch	Step	Training Loss
1.8939	500	0.0778

Framework Versions

• Python: 3.12.7
• Sentence Transformers: 3.1.1
• Transformers: 4.45.2
• PyTorch: 2.5.1+cpu
• Accelerate: 1.1.1
• Datasets: 3.1.0
• Tokenizers: 0.20.3

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}