SentenceTransformer
This is a sentence-transformers model trained on the triplets dataset. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.
Model Details
Model Description
- Model Type: Sentence Transformer
- Maximum Sequence Length: 8192 tokens
- Output Dimensionality: 768 tokens
- Similarity Function: Cosine Similarity
- Training Dataset:
Model Sources
Full Model Architecture
SentenceTransformer(
(0): Transformer({'max_seq_length': 8192, 'do_lower_case': False}) with Transformer model: NomicBertModel
(1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
)
Usage
Direct Usage (Sentence Transformers)
First install the Sentence Transformers library:
pip install -U sentence-transformers
Then you can load this model and run inference.
from sentence_transformers import SentenceTransformer
model = SentenceTransformer("lv12/esci-nomic-embed-text-v1_5_4")
sentences = [
'search_query: karoke set 2 microphone for adults',
'search_document: Starion KS829-B Bluetooth Karaoke Machine l Pedestal Design w/Light Show l Two Karaoke Microphones, Starion, Black',
'search_document: EARISE T26 Portable Karaoke Machine Bluetooth Speaker with Wireless Microphone, Rechargeable PA System with FM Radio, Audio Recording, Remote Control, Supports TF Card/USB, Perfect for Party, EARISE, ',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
Evaluation
Metrics
Triplet
Metric |
Value |
cosine_accuracy |
0.7298 |
dot_accuracy |
0.2832 |
manhattan_accuracy |
0.7282 |
euclidean_accuracy |
0.7299 |
max_accuracy |
0.7299 |
Semantic Similarity
Metric |
Value |
pearson_cosine |
0.4148 |
spearman_cosine |
0.3997 |
pearson_manhattan |
0.3771 |
spearman_manhattan |
0.3699 |
pearson_euclidean |
0.3778 |
spearman_euclidean |
0.3708 |
pearson_dot |
0.3814 |
spearman_dot |
0.3817 |
pearson_max |
0.4148 |
spearman_max |
0.3997 |
Information Retrieval
Metric |
Value |
cosine_accuracy@10 |
0.967 |
cosine_precision@10 |
0.6951 |
cosine_recall@10 |
0.6217 |
cosine_ndcg@10 |
0.83 |
cosine_mrr@10 |
0.9111 |
cosine_map@10 |
0.7758 |
dot_accuracy@10 |
0.946 |
dot_precision@10 |
0.6369 |
dot_recall@10 |
0.5693 |
dot_ndcg@10 |
0.7669 |
dot_mrr@10 |
0.8754 |
dot_map@10 |
0.6962 |
Training Details
Training Dataset
triplets
Evaluation Dataset
triplets
Training Hyperparameters
Non-Default Hyperparameters
per_device_train_batch_size
: 64
per_device_eval_batch_size
: 16
gradient_accumulation_steps
: 2
learning_rate
: 1e-07
num_train_epochs
: 5
lr_scheduler_type
: polynomial
lr_scheduler_kwargs
: {'lr_end': 1e-08, 'power': 2.0}
warmup_ratio
: 0.05
dataloader_drop_last
: True
dataloader_num_workers
: 4
dataloader_prefetch_factor
: 4
load_best_model_at_end
: True
gradient_checkpointing
: True
auto_find_batch_size
: True
batch_sampler
: no_duplicates
All Hyperparameters
Click to expand
overwrite_output_dir
: False
do_predict
: False
prediction_loss_only
: True
per_device_train_batch_size
: 64
per_device_eval_batch_size
: 16
per_gpu_train_batch_size
: None
per_gpu_eval_batch_size
: None
gradient_accumulation_steps
: 2
eval_accumulation_steps
: None
learning_rate
: 1e-07
weight_decay
: 0.0
adam_beta1
: 0.9
adam_beta2
: 0.999
adam_epsilon
: 1e-08
max_grad_norm
: 1.0
num_train_epochs
: 5
max_steps
: -1
lr_scheduler_type
: polynomial
lr_scheduler_kwargs
: {'lr_end': 1e-08, 'power': 2.0}
warmup_ratio
: 0.05
warmup_steps
: 0
log_level
: passive
log_level_replica
: warning
log_on_each_node
: True
logging_nan_inf_filter
: True
save_safetensors
: True
save_on_each_node
: False
save_only_model
: False
no_cuda
: False
use_cpu
: False
use_mps_device
: False
seed
: 42
data_seed
: None
jit_mode_eval
: False
use_ipex
: False
bf16
: False
fp16
: False
fp16_opt_level
: O1
half_precision_backend
: auto
bf16_full_eval
: False
fp16_full_eval
: False
tf32
: None
local_rank
: 0
ddp_backend
: None
tpu_num_cores
: None
tpu_metrics_debug
: False
debug
: []
dataloader_drop_last
: True
dataloader_num_workers
: 4
dataloader_prefetch_factor
: 4
past_index
: -1
disable_tqdm
: False
remove_unused_columns
: True
label_names
: None
load_best_model_at_end
: True
ignore_data_skip
: False
fsdp
: []
fsdp_min_num_params
: 0
fsdp_config
: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
fsdp_transformer_layer_cls_to_wrap
: None
accelerator_config
: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True}
deepspeed
: None
label_smoothing_factor
: 0.0
optim
: adamw_torch
optim_args
: None
adafactor
: False
group_by_length
: False
length_column_name
: length
ddp_find_unused_parameters
: None
ddp_bucket_cap_mb
: None
ddp_broadcast_buffers
: False
dataloader_pin_memory
: True
dataloader_persistent_workers
: False
skip_memory_metrics
: True
use_legacy_prediction_loop
: False
push_to_hub
: False
resume_from_checkpoint
: None
hub_model_id
: None
hub_strategy
: every_save
hub_private_repo
: False
hub_always_push
: False
gradient_checkpointing
: True
gradient_checkpointing_kwargs
: None
include_inputs_for_metrics
: False
fp16_backend
: auto
push_to_hub_model_id
: None
push_to_hub_organization
: None
mp_parameters
:
auto_find_batch_size
: True
full_determinism
: False
torchdynamo
: None
ray_scope
: last
ddp_timeout
: 1800
torch_compile
: False
torch_compile_backend
: None
torch_compile_mode
: None
dispatch_batches
: None
split_batches
: None
include_tokens_per_second
: False
include_num_input_tokens_seen
: False
neftune_noise_alpha
: None
batch_sampler
: no_duplicates
multi_dataset_batch_sampler
: proportional
Training Logs
Click to expand
Epoch |
Step |
Training Loss |
triplets loss |
cosine_accuracy |
cosine_map@10 |
spearman_cosine |
0.0008 |
10 |
0.7505 |
- |
- |
- |
- |
0.0016 |
20 |
0.7499 |
- |
- |
- |
- |
0.0024 |
30 |
0.7524 |
- |
- |
- |
- |
0.0032 |
40 |
0.7486 |
- |
- |
- |
- |
0.004 |
50 |
0.7493 |
- |
- |
- |
- |
0.0048 |
60 |
0.7476 |
- |
- |
- |
- |
0.0056 |
70 |
0.7483 |
- |
- |
- |
- |
0.0064 |
80 |
0.7487 |
- |
- |
- |
- |
0.0072 |
90 |
0.7496 |
- |
- |
- |
- |
0.008 |
100 |
0.7515 |
0.7559 |
0.7263 |
0.7684 |
0.3941 |
0.0088 |
110 |
0.7523 |
- |
- |
- |
- |
0.0096 |
120 |
0.7517 |
- |
- |
- |
- |
0.0104 |
130 |
0.7534 |
- |
- |
- |
- |
0.0112 |
140 |
0.746 |
- |
- |
- |
- |
0.012 |
150 |
0.7528 |
- |
- |
- |
- |
0.0128 |
160 |
0.7511 |
- |
- |
- |
- |
0.0136 |
170 |
0.7491 |
- |
- |
- |
- |
0.0144 |
180 |
0.752 |
- |
- |
- |
- |
0.0152 |
190 |
0.7512 |
- |
- |
- |
- |
0.016 |
200 |
0.7513 |
0.7557 |
0.7259 |
0.7688 |
0.3942 |
0.0168 |
210 |
0.7505 |
- |
- |
- |
- |
0.0176 |
220 |
0.7481 |
- |
- |
- |
- |
0.0184 |
230 |
0.7516 |
- |
- |
- |
- |
0.0192 |
240 |
0.7504 |
- |
- |
- |
- |
0.02 |
250 |
0.7498 |
- |
- |
- |
- |
0.0208 |
260 |
0.7506 |
- |
- |
- |
- |
0.0216 |
270 |
0.7486 |
- |
- |
- |
- |
0.0224 |
280 |
0.7471 |
- |
- |
- |
- |
0.0232 |
290 |
0.7511 |
- |
- |
- |
- |
0.024 |
300 |
0.7506 |
0.7553 |
0.7258 |
0.7692 |
0.3943 |
0.0248 |
310 |
0.7485 |
- |
- |
- |
- |
0.0256 |
320 |
0.7504 |
- |
- |
- |
- |
0.0264 |
330 |
0.7456 |
- |
- |
- |
- |
0.0272 |
340 |
0.7461 |
- |
- |
- |
- |
0.028 |
350 |
0.7496 |
- |
- |
- |
- |
0.0288 |
360 |
0.7518 |
- |
- |
- |
- |
0.0296 |
370 |
0.7514 |
- |
- |
- |
- |
0.0304 |
380 |
0.7479 |
- |
- |
- |
- |
0.0312 |
390 |
0.7507 |
- |
- |
- |
- |
0.032 |
400 |
0.7511 |
0.7547 |
0.7258 |
0.7695 |
0.3945 |
0.0328 |
410 |
0.7491 |
- |
- |
- |
- |
0.0336 |
420 |
0.7487 |
- |
- |
- |
- |
0.0344 |
430 |
0.7496 |
- |
- |
- |
- |
0.0352 |
440 |
0.7464 |
- |
- |
- |
- |
0.036 |
450 |
0.7518 |
- |
- |
- |
- |
0.0368 |
460 |
0.7481 |
- |
- |
- |
- |
0.0376 |
470 |
0.7493 |
- |
- |
- |
- |
0.0384 |
480 |
0.753 |
- |
- |
- |
- |
0.0392 |
490 |
0.7475 |
- |
- |
- |
- |
0.04 |
500 |
0.7498 |
0.7540 |
0.7262 |
0.7700 |
0.3948 |
0.0408 |
510 |
0.7464 |
- |
- |
- |
- |
0.0416 |
520 |
0.7506 |
- |
- |
- |
- |
0.0424 |
530 |
0.747 |
- |
- |
- |
- |
0.0432 |
540 |
0.7462 |
- |
- |
- |
- |
0.044 |
550 |
0.75 |
- |
- |
- |
- |
0.0448 |
560 |
0.7522 |
- |
- |
- |
- |
0.0456 |
570 |
0.7452 |
- |
- |
- |
- |
0.0464 |
580 |
0.7475 |
- |
- |
- |
- |
0.0472 |
590 |
0.7507 |
- |
- |
- |
- |
0.048 |
600 |
0.7494 |
0.7531 |
0.7269 |
0.7707 |
0.3951 |
0.0488 |
610 |
0.7525 |
- |
- |
- |
- |
0.0496 |
620 |
0.7446 |
- |
- |
- |
- |
0.0504 |
630 |
0.7457 |
- |
- |
- |
- |
0.0512 |
640 |
0.7462 |
- |
- |
- |
- |
0.052 |
650 |
0.7478 |
- |
- |
- |
- |
0.0528 |
660 |
0.7459 |
- |
- |
- |
- |
0.0536 |
670 |
0.7465 |
- |
- |
- |
- |
0.0544 |
680 |
0.7495 |
- |
- |
- |
- |
0.0552 |
690 |
0.7513 |
- |
- |
- |
- |
0.056 |
700 |
0.7445 |
0.7520 |
0.7274 |
0.7705 |
0.3954 |
0.0568 |
710 |
0.7446 |
- |
- |
- |
- |
0.0576 |
720 |
0.746 |
- |
- |
- |
- |
0.0584 |
730 |
0.7452 |
- |
- |
- |
- |
0.0592 |
740 |
0.7459 |
- |
- |
- |
- |
0.06 |
750 |
0.7419 |
- |
- |
- |
- |
0.0608 |
760 |
0.7462 |
- |
- |
- |
- |
0.0616 |
770 |
0.7414 |
- |
- |
- |
- |
0.0624 |
780 |
0.7444 |
- |
- |
- |
- |
0.0632 |
790 |
0.7419 |
- |
- |
- |
- |
0.064 |
800 |
0.7438 |
0.7508 |
0.7273 |
0.7712 |
0.3957 |
0.0648 |
810 |
0.7503 |
- |
- |
- |
- |
0.0656 |
820 |
0.7402 |
- |
- |
- |
- |
0.0664 |
830 |
0.7435 |
- |
- |
- |
- |
0.0672 |
840 |
0.741 |
- |
- |
- |
- |
0.068 |
850 |
0.7386 |
- |
- |
- |
- |
0.0688 |
860 |
0.7416 |
- |
- |
- |
- |
0.0696 |
870 |
0.7473 |
- |
- |
- |
- |
0.0704 |
880 |
0.7438 |
- |
- |
- |
- |
0.0712 |
890 |
0.7458 |
- |
- |
- |
- |
0.072 |
900 |
0.7446 |
0.7494 |
0.7279 |
0.7718 |
0.3961 |
0.0728 |
910 |
0.7483 |
- |
- |
- |
- |
0.0736 |
920 |
0.7458 |
- |
- |
- |
- |
0.0744 |
930 |
0.7473 |
- |
- |
- |
- |
0.0752 |
940 |
0.7431 |
- |
- |
- |
- |
0.076 |
950 |
0.7428 |
- |
- |
- |
- |
0.0768 |
960 |
0.7385 |
- |
- |
- |
- |
0.0776 |
970 |
0.7438 |
- |
- |
- |
- |
0.0784 |
980 |
0.7406 |
- |
- |
- |
- |
0.0792 |
990 |
0.7426 |
- |
- |
- |
- |
0.08 |
1000 |
0.7372 |
0.7478 |
0.7282 |
0.7725 |
0.3965 |
0.0808 |
1010 |
0.7396 |
- |
- |
- |
- |
0.0816 |
1020 |
0.7398 |
- |
- |
- |
- |
0.0824 |
1030 |
0.7376 |
- |
- |
- |
- |
0.0832 |
1040 |
0.7417 |
- |
- |
- |
- |
0.084 |
1050 |
0.7408 |
- |
- |
- |
- |
0.0848 |
1060 |
0.7415 |
- |
- |
- |
- |
0.0856 |
1070 |
0.7468 |
- |
- |
- |
- |
0.0864 |
1080 |
0.7427 |
- |
- |
- |
- |
0.0872 |
1090 |
0.7371 |
- |
- |
- |
- |
0.088 |
1100 |
0.7375 |
0.7460 |
0.7279 |
0.7742 |
0.3970 |
0.0888 |
1110 |
0.7434 |
- |
- |
- |
- |
0.0896 |
1120 |
0.7441 |
- |
- |
- |
- |
0.0904 |
1130 |
0.7378 |
- |
- |
- |
- |
0.0912 |
1140 |
0.735 |
- |
- |
- |
- |
0.092 |
1150 |
0.739 |
- |
- |
- |
- |
0.0928 |
1160 |
0.7408 |
- |
- |
- |
- |
0.0936 |
1170 |
0.7346 |
- |
- |
- |
- |
0.0944 |
1180 |
0.7389 |
- |
- |
- |
- |
0.0952 |
1190 |
0.7367 |
- |
- |
- |
- |
0.096 |
1200 |
0.7358 |
0.7440 |
0.729 |
0.7747 |
0.3975 |
0.0968 |
1210 |
0.7381 |
- |
- |
- |
- |
0.0976 |
1220 |
0.7405 |
- |
- |
- |
- |
0.0984 |
1230 |
0.7348 |
- |
- |
- |
- |
0.0992 |
1240 |
0.737 |
- |
- |
- |
- |
0.1 |
1250 |
0.7393 |
- |
- |
- |
- |
0.1008 |
1260 |
0.7411 |
- |
- |
- |
- |
0.1016 |
1270 |
0.7359 |
- |
- |
- |
- |
0.1024 |
1280 |
0.7276 |
- |
- |
- |
- |
0.1032 |
1290 |
0.7364 |
- |
- |
- |
- |
0.104 |
1300 |
0.7333 |
0.7418 |
0.7293 |
0.7747 |
0.3979 |
0.1048 |
1310 |
0.7367 |
- |
- |
- |
- |
0.1056 |
1320 |
0.7352 |
- |
- |
- |
- |
0.1064 |
1330 |
0.7333 |
- |
- |
- |
- |
0.1072 |
1340 |
0.737 |
- |
- |
- |
- |
0.108 |
1350 |
0.7361 |
- |
- |
- |
- |
0.1088 |
1360 |
0.7299 |
- |
- |
- |
- |
0.1096 |
1370 |
0.7339 |
- |
- |
- |
- |
0.1104 |
1380 |
0.7349 |
- |
- |
- |
- |
0.1112 |
1390 |
0.7318 |
- |
- |
- |
- |
0.112 |
1400 |
0.7336 |
0.7394 |
0.7292 |
0.7749 |
0.3983 |
0.1128 |
1410 |
0.7326 |
- |
- |
- |
- |
0.1136 |
1420 |
0.7317 |
- |
- |
- |
- |
0.1144 |
1430 |
0.7315 |
- |
- |
- |
- |
0.1152 |
1440 |
0.7321 |
- |
- |
- |
- |
0.116 |
1450 |
0.7284 |
- |
- |
- |
- |
0.1168 |
1460 |
0.7308 |
- |
- |
- |
- |
0.1176 |
1470 |
0.7287 |
- |
- |
- |
- |
0.1184 |
1480 |
0.727 |
- |
- |
- |
- |
0.1192 |
1490 |
0.7298 |
- |
- |
- |
- |
0.12 |
1500 |
0.7306 |
0.7368 |
0.7301 |
0.7755 |
0.3988 |
0.1208 |
1510 |
0.7269 |
- |
- |
- |
- |
0.1216 |
1520 |
0.7299 |
- |
- |
- |
- |
0.1224 |
1530 |
0.7256 |
- |
- |
- |
- |
0.1232 |
1540 |
0.721 |
- |
- |
- |
- |
0.124 |
1550 |
0.7274 |
- |
- |
- |
- |
0.1248 |
1560 |
0.7251 |
- |
- |
- |
- |
0.1256 |
1570 |
0.7248 |
- |
- |
- |
- |
0.1264 |
1580 |
0.7244 |
- |
- |
- |
- |
0.1272 |
1590 |
0.7275 |
- |
- |
- |
- |
0.128 |
1600 |
0.7264 |
0.7339 |
0.7298 |
0.7756 |
0.3991 |
0.1288 |
1610 |
0.7252 |
- |
- |
- |
- |
0.1296 |
1620 |
0.7287 |
- |
- |
- |
- |
0.1304 |
1630 |
0.7263 |
- |
- |
- |
- |
0.1312 |
1640 |
0.7216 |
- |
- |
- |
- |
0.132 |
1650 |
0.7231 |
- |
- |
- |
- |
0.1328 |
1660 |
0.728 |
- |
- |
- |
- |
0.1336 |
1670 |
0.7309 |
- |
- |
- |
- |
0.1344 |
1680 |
0.7243 |
- |
- |
- |
- |
0.1352 |
1690 |
0.7239 |
- |
- |
- |
- |
0.136 |
1700 |
0.7219 |
0.7309 |
0.7302 |
0.7768 |
0.3994 |
0.1368 |
1710 |
0.7212 |
- |
- |
- |
- |
0.1376 |
1720 |
0.7217 |
- |
- |
- |
- |
0.1384 |
1730 |
0.7118 |
- |
- |
- |
- |
0.1392 |
1740 |
0.7226 |
- |
- |
- |
- |
0.14 |
1750 |
0.7185 |
- |
- |
- |
- |
0.1408 |
1760 |
0.7228 |
- |
- |
- |
- |
0.1416 |
1770 |
0.7257 |
- |
- |
- |
- |
0.1424 |
1780 |
0.7177 |
- |
- |
- |
- |
0.1432 |
1790 |
0.722 |
- |
- |
- |
- |
0.144 |
1800 |
0.712 |
0.7276 |
0.7307 |
0.7763 |
0.3997 |
0.1448 |
1810 |
0.7193 |
- |
- |
- |
- |
0.1456 |
1820 |
0.7138 |
- |
- |
- |
- |
0.1464 |
1830 |
0.7171 |
- |
- |
- |
- |
0.1472 |
1840 |
0.7191 |
- |
- |
- |
- |
0.148 |
1850 |
0.7172 |
- |
- |
- |
- |
0.1488 |
1860 |
0.7168 |
- |
- |
- |
- |
0.1496 |
1870 |
0.7111 |
- |
- |
- |
- |
0.1504 |
1880 |
0.7203 |
- |
- |
- |
- |
0.1512 |
1890 |
0.7095 |
- |
- |
- |
- |
0.152 |
1900 |
0.7064 |
0.7240 |
0.7301 |
0.7762 |
0.3998 |
0.1528 |
1910 |
0.7147 |
- |
- |
- |
- |
0.1536 |
1920 |
0.7098 |
- |
- |
- |
- |
0.1544 |
1930 |
0.7193 |
- |
- |
- |
- |
0.1552 |
1940 |
0.7096 |
- |
- |
- |
- |
0.156 |
1950 |
0.7107 |
- |
- |
- |
- |
0.1568 |
1960 |
0.7146 |
- |
- |
- |
- |
0.1576 |
1970 |
0.7106 |
- |
- |
- |
- |
0.1584 |
1980 |
0.7079 |
- |
- |
- |
- |
0.1592 |
1990 |
0.7097 |
- |
- |
- |
- |
0.16 |
2000 |
0.71 |
0.7202 |
0.7298 |
0.7758 |
0.3997 |
Framework Versions
- Python: 3.10.12
- Sentence Transformers: 3.0.1
- Transformers: 4.38.2
- PyTorch: 2.1.2+cu121
- Accelerate: 0.27.2
- Datasets: 2.19.1
- Tokenizers: 0.15.2
Citation
BibTeX
Sentence Transformers
@inproceedings{reimers-2019-sentence-bert,
title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
author = "Reimers, Nils and Gurevych, Iryna",
booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
month = "11",
year = "2019",
publisher = "Association for Computational Linguistics",
url = "https://arxiv.org/abs/1908.10084",
}
TripletLoss
@misc{hermans2017defense,
title={In Defense of the Triplet Loss for Person Re-Identification},
author={Alexander Hermans and Lucas Beyer and Bastian Leibe},
year={2017},
eprint={1703.07737},
archivePrefix={arXiv},
primaryClass={cs.CV}
}