SGPT-125M-weightedmean-msmarco-specb-bitfit
Usage
For usage instructions, refer to our codebase: https://github.com/Muennighoff/sgpt
Evaluation Results
For eval results, refer to the eval folder or our paper: https://arxiv.org/abs/2202.08904
Training
The model was trained with the parameters:
DataLoader:
torch.utils.data.dataloader.DataLoader
of length 15600 with parameters:
{'batch_size': 32, 'sampler': 'torch.utils.data.sampler.RandomSampler', 'batch_sampler': 'torch.utils.data.sampler.BatchSampler'}
Loss:
sentence_transformers.losses.MultipleNegativesRankingLoss.MultipleNegativesRankingLoss
with parameters:
{'scale': 20.0, 'similarity_fct': 'cos_sim'}
Parameters of the fit()-Method:
{
"epochs": 10,
"evaluation_steps": 0,
"evaluator": "NoneType",
"max_grad_norm": 1,
"optimizer_class": "<class 'transformers.optimization.AdamW'>",
"optimizer_params": {
"lr": 0.0002
},
"scheduler": "WarmupLinear",
"steps_per_epoch": null,
"warmup_steps": 1000,
"weight_decay": 0.01
}
Full Model Architecture
SentenceTransformer(
(0): Transformer({'max_seq_length': 300, 'do_lower_case': False}) with Transformer model: GPTNeoModel
(1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': True, 'pooling_mode_lasttoken': False})
)
Citing & Authors
@article{muennighoff2022sgpt,
title={SGPT: GPT Sentence Embeddings for Semantic Search},
author={Muennighoff, Niklas},
journal={arXiv preprint arXiv:2202.08904},
year={2022}
}
- Downloads last month
- 619
This model does not have enough activity to be deployed to Inference API (serverless) yet. Increase its social
visibility and check back later, or deploy to Inference Endpoints (dedicated)
instead.
Spaces using Muennighoff/SGPT-125M-weightedmean-msmarco-specb-bitfit 7
Evaluation results
- accuracy on MTEB AmazonCounterfactualClassification (en)test set self-reported61.239
- ap on MTEB AmazonCounterfactualClassification (en)test set self-reported25.854
- f1 on MTEB AmazonCounterfactualClassification (en)test set self-reported55.752
- accuracy on MTEB AmazonCounterfactualClassification (de)test set self-reported56.884
- ap on MTEB AmazonCounterfactualClassification (de)test set self-reported72.673
- f1 on MTEB AmazonCounterfactualClassification (de)test set self-reported54.450
- accuracy on MTEB AmazonCounterfactualClassification (en-ext)test set self-reported58.276
- ap on MTEB AmazonCounterfactualClassification (en-ext)test set self-reported14.067
- f1 on MTEB AmazonCounterfactualClassification (en-ext)test set self-reported48.172
- accuracy on MTEB AmazonCounterfactualClassification (ja)test set self-reported54.647