--- language: - en license: apache-2.0 tags: - sentence-transformers - sentence-similarity - feature-extraction - generated_from_trainer - dataset_size:50000 - loss:CachedGISTEmbedLoss base_model: microsoft/mpnet-base widget: - source_sentence: what does the accounts receivable turnover measure? sentences: - The accounts receivable turnover ratio is an accounting measure used to quantify a company's effectiveness in collecting its receivables or money owed by clients. The ratio shows how well a company uses and manages the credit it extends to customers and how quickly that short-term debt is collected or is paid. - Capital budgeting, and investment appraisal, is the planning process used to determine whether an organization's long term investments such as new machinery, replacement of machinery, new plants, new products, and research development projects are worth the funding of cash through the firm's capitalization structure ( ... - The accounts receivable turnover ratio is an accounting measure used to quantify a company's effectiveness in collecting its receivables or money owed by clients. The ratio shows how well a company uses and manages the credit it extends to customers and how quickly that short-term debt is collected or is paid. - source_sentence: does gabapentin cause liver problems? sentences: - Gabapentin has no appreciable liver metabolism, yet, suspected cases of gabapentin-induced hepatotoxicity have been reported. Per literature review, two cases of possible gabapentin-induced liver injury have been reported. - Strongholds are a type of story mission which only unlocks after enough progression through the game. There are three Stronghold's during the first section of progression through The Division 2. You'll need to complete the first two and have reached level 30 before being able to unlock the final Stronghold. - The most-common side effects attributed to Gabapentin include mild sedation, ataxia, and occasional diarrhea. Sedation can be minimized by tapering from a smaller starting dose to the desired dose. When treating seizures, it is ideal to wean off the drug to reduce the risk of withdrawal seizures. - source_sentence: how long should you wait to give blood after eating? sentences: - Until the bleeding has stopped it is natural to taste blood or to see traces of blood in your saliva. You may stop using gauze after the flow stops – usually around 8 hours after surgery. - Before donation The first and most important rule—never donate blood on an empty stomach. “Eat a wholesome meal about 2-3 hours before donating to keep your blood sugar stable," says Dr Chaturvedi. The timing of the meal is important too. You need to allow the food to be digested properly before the blood is drawn. - While grid computing involves virtualizing computing resources to store massive amounts of data, whereas cloud computing is where an application doesn't access resources directly, rather it accesses them through a service over the internet. ... - source_sentence: what is the difference between chicken francese and chicken marsala? sentences: - Chicken is the species name, equivalent to our “human.” Rooster is an adult male, equivalent to “man.” Hen is an adult female, equivalent to “woman.” Cockerel is a juvenile male, equivalent to “boy/young man.” - What is 99 kg in pounds? - 99 kg is equal to 218.26 pounds. - The difference between the two is for Francese, the chicken breast is first dipped in flour, then into a beaten egg mixture, before being cooked. For piccata, the chicken is first dipped in egg and then in flour. Both are then simmered in a lemony butter sauce, but the piccata sauce includes capers.” - source_sentence: what energy is released when coal is burned? sentences: - When coal is burned, it reacts with the oxygen in the air. This chemical reaction converts the stored solar energy into thermal energy, which is released as heat. But it also produces carbon dioxide and methane. - When coal is burned it releases a number of airborne toxins and pollutants. They include mercury, lead, sulfur dioxide, nitrogen oxides, particulates, and various other heavy metals. - Squad Building Challenges allow you to exchange sets of players for coins, packs, and special items in FUT 20. Each of these challenges come with specific requirements, such as including players from certain teams. ... Live SBCs are time-limited challenges which often give out unique, high-rated versions of players. datasets: - tomaarsen/gooaq-hard-negatives pipeline_tag: sentence-similarity library_name: sentence-transformers metrics: - cosine_accuracy@1 - cosine_accuracy@3 - cosine_accuracy@5 - cosine_accuracy@10 - cosine_precision@1 - cosine_precision@3 - cosine_precision@5 - cosine_precision@10 - cosine_recall@1 - cosine_recall@3 - cosine_recall@5 - cosine_recall@10 - cosine_ndcg@10 - cosine_mrr@10 - cosine_map@100 co2_eq_emissions: emissions: 40.54325678627484 energy_consumed: 0.10430421450436282 source: codecarbon training_type: fine-tuning on_cloud: false cpu_model: 13th Gen Intel(R) Core(TM) i7-13700K ram_total_size: 31.777088165283203 hours_used: 0.301 hardware_used: 1 x NVIDIA GeForce RTX 3090 model-index: - name: MPNet base trained on Natural Questions pairs results: - task: type: information-retrieval name: Information Retrieval dataset: name: NanoClimateFEVER type: NanoClimateFEVER metrics: - type: cosine_accuracy@1 value: 0.22 name: Cosine Accuracy@1 - type: cosine_accuracy@3 value: 0.44 name: Cosine Accuracy@3 - type: cosine_accuracy@5 value: 0.52 name: Cosine Accuracy@5 - type: cosine_accuracy@10 value: 0.72 name: Cosine Accuracy@10 - type: cosine_precision@1 value: 0.22 name: Cosine Precision@1 - type: cosine_precision@3 value: 0.16666666666666663 name: Cosine Precision@3 - type: cosine_precision@5 value: 0.12 name: Cosine Precision@5 - type: cosine_precision@10 value: 0.09399999999999999 name: Cosine Precision@10 - type: cosine_recall@1 value: 0.09333333333333332 name: Cosine Recall@1 - type: cosine_recall@3 value: 0.195 name: Cosine Recall@3 - type: cosine_recall@5 value: 0.2333333333333333 name: Cosine Recall@5 - type: cosine_recall@10 value: 0.37233333333333335 name: Cosine Recall@10 - type: cosine_ndcg@10 value: 0.2744024872493329 name: Cosine Ndcg@10 - type: cosine_mrr@10 value: 0.3594365079365079 name: Cosine Mrr@10 - type: cosine_map@100 value: 0.20181676147957636 name: Cosine Map@100 - task: type: information-retrieval name: Information Retrieval dataset: name: NanoDBPedia type: NanoDBPedia metrics: - type: cosine_accuracy@1 value: 0.46 name: Cosine Accuracy@1 - type: cosine_accuracy@3 value: 0.62 name: Cosine Accuracy@3 - type: cosine_accuracy@5 value: 0.76 name: Cosine Accuracy@5 - type: cosine_accuracy@10 value: 0.82 name: Cosine Accuracy@10 - type: cosine_precision@1 value: 0.46 name: Cosine Precision@1 - type: cosine_precision@3 value: 0.38666666666666666 name: Cosine Precision@3 - type: cosine_precision@5 value: 0.38799999999999996 name: Cosine Precision@5 - type: cosine_precision@10 value: 0.344 name: Cosine Precision@10 - type: cosine_recall@1 value: 0.03065300183409328 name: Cosine Recall@1 - type: cosine_recall@3 value: 0.07730098142643593 name: Cosine Recall@3 - type: cosine_recall@5 value: 0.14588470319900892 name: Cosine Recall@5 - type: cosine_recall@10 value: 0.22159653924772912 name: Cosine Recall@10 - type: cosine_ndcg@10 value: 0.3920743245484332 name: Cosine Ndcg@10 - type: cosine_mrr@10 value: 0.567 name: Cosine Mrr@10 - type: cosine_map@100 value: 0.28153419189397744 name: Cosine Map@100 - task: type: information-retrieval name: Information Retrieval dataset: name: NanoFEVER type: NanoFEVER metrics: - type: cosine_accuracy@1 value: 0.38 name: Cosine Accuracy@1 - type: cosine_accuracy@3 value: 0.54 name: Cosine Accuracy@3 - type: cosine_accuracy@5 value: 0.58 name: Cosine Accuracy@5 - type: cosine_accuracy@10 value: 0.68 name: Cosine Accuracy@10 - type: cosine_precision@1 value: 0.38 name: Cosine Precision@1 - type: cosine_precision@3 value: 0.18 name: Cosine Precision@3 - type: cosine_precision@5 value: 0.12 name: Cosine Precision@5 - type: cosine_precision@10 value: 0.07 name: Cosine Precision@10 - type: cosine_recall@1 value: 0.37 name: Cosine Recall@1 - type: cosine_recall@3 value: 0.52 name: Cosine Recall@3 - type: cosine_recall@5 value: 0.57 name: Cosine Recall@5 - type: cosine_recall@10 value: 0.66 name: Cosine Recall@10 - type: cosine_ndcg@10 value: 0.5156585003907987 name: Cosine Ndcg@10 - type: cosine_mrr@10 value: 0.4756666666666666 name: Cosine Mrr@10 - type: cosine_map@100 value: 0.47620972127897226 name: Cosine Map@100 - task: type: information-retrieval name: Information Retrieval dataset: name: NanoFiQA2018 type: NanoFiQA2018 metrics: - type: cosine_accuracy@1 value: 0.28 name: Cosine Accuracy@1 - type: cosine_accuracy@3 value: 0.5 name: Cosine Accuracy@3 - type: cosine_accuracy@5 value: 0.52 name: Cosine Accuracy@5 - type: cosine_accuracy@10 value: 0.58 name: Cosine Accuracy@10 - type: cosine_precision@1 value: 0.28 name: Cosine Precision@1 - type: cosine_precision@3 value: 0.22 name: Cosine Precision@3 - type: cosine_precision@5 value: 0.16399999999999998 name: Cosine Precision@5 - type: cosine_precision@10 value: 0.09799999999999999 name: Cosine Precision@10 - type: cosine_recall@1 value: 0.1371904761904762 name: Cosine Recall@1 - type: cosine_recall@3 value: 0.3226904761904762 name: Cosine Recall@3 - type: cosine_recall@5 value: 0.3682142857142857 name: Cosine Recall@5 - type: cosine_recall@10 value: 0.43073809523809525 name: Cosine Recall@10 - type: cosine_ndcg@10 value: 0.3420135901424927 name: Cosine Ndcg@10 - type: cosine_mrr@10 value: 0.38405555555555554 name: Cosine Mrr@10 - type: cosine_map@100 value: 0.2826394452885763 name: Cosine Map@100 - task: type: information-retrieval name: Information Retrieval dataset: name: NanoHotpotQA type: NanoHotpotQA metrics: - type: cosine_accuracy@1 value: 0.34 name: Cosine Accuracy@1 - type: cosine_accuracy@3 value: 0.52 name: Cosine Accuracy@3 - type: cosine_accuracy@5 value: 0.62 name: Cosine Accuracy@5 - type: cosine_accuracy@10 value: 0.72 name: Cosine Accuracy@10 - type: cosine_precision@1 value: 0.34 name: Cosine Precision@1 - type: cosine_precision@3 value: 0.19333333333333333 name: Cosine Precision@3 - type: cosine_precision@5 value: 0.14400000000000002 name: Cosine Precision@5 - type: cosine_precision@10 value: 0.09200000000000001 name: Cosine Precision@10 - type: cosine_recall@1 value: 0.17 name: Cosine Recall@1 - type: cosine_recall@3 value: 0.29 name: Cosine Recall@3 - type: cosine_recall@5 value: 0.36 name: Cosine Recall@5 - type: cosine_recall@10 value: 0.46 name: Cosine Recall@10 - type: cosine_ndcg@10 value: 0.3723049657456267 name: Cosine Ndcg@10 - type: cosine_mrr@10 value: 0.4570793650793651 name: Cosine Mrr@10 - type: cosine_map@100 value: 0.2995175868330484 name: Cosine Map@100 - task: type: information-retrieval name: Information Retrieval dataset: name: NanoMSMARCO type: NanoMSMARCO metrics: - type: cosine_accuracy@1 value: 0.1 name: Cosine Accuracy@1 - type: cosine_accuracy@3 value: 0.28 name: Cosine Accuracy@3 - type: cosine_accuracy@5 value: 0.52 name: Cosine Accuracy@5 - type: cosine_accuracy@10 value: 0.68 name: Cosine Accuracy@10 - type: cosine_precision@1 value: 0.1 name: Cosine Precision@1 - type: cosine_precision@3 value: 0.09333333333333332 name: Cosine Precision@3 - type: cosine_precision@5 value: 0.10400000000000001 name: Cosine Precision@5 - type: cosine_precision@10 value: 0.068 name: Cosine Precision@10 - type: cosine_recall@1 value: 0.1 name: Cosine Recall@1 - type: cosine_recall@3 value: 0.28 name: Cosine Recall@3 - type: cosine_recall@5 value: 0.52 name: Cosine Recall@5 - type: cosine_recall@10 value: 0.68 name: Cosine Recall@10 - type: cosine_ndcg@10 value: 0.36083481845261806 name: Cosine Ndcg@10 - type: cosine_mrr@10 value: 0.26157142857142857 name: Cosine Mrr@10 - type: cosine_map@100 value: 0.27215692684924997 name: Cosine Map@100 - task: type: information-retrieval name: Information Retrieval dataset: name: NanoNFCorpus type: NanoNFCorpus metrics: - type: cosine_accuracy@1 value: 0.26 name: Cosine Accuracy@1 - type: cosine_accuracy@3 value: 0.38 name: Cosine Accuracy@3 - type: cosine_accuracy@5 value: 0.44 name: Cosine Accuracy@5 - type: cosine_accuracy@10 value: 0.5 name: Cosine Accuracy@10 - type: cosine_precision@1 value: 0.26 name: Cosine Precision@1 - type: cosine_precision@3 value: 0.21333333333333332 name: Cosine Precision@3 - type: cosine_precision@5 value: 0.19599999999999998 name: Cosine Precision@5 - type: cosine_precision@10 value: 0.13799999999999998 name: Cosine Precision@10 - type: cosine_recall@1 value: 0.01122167476431692 name: Cosine Recall@1 - type: cosine_recall@3 value: 0.02047531859468654 name: Cosine Recall@3 - type: cosine_recall@5 value: 0.03079316493603994 name: Cosine Recall@5 - type: cosine_recall@10 value: 0.0422192068561938 name: Cosine Recall@10 - type: cosine_ndcg@10 value: 0.1654539374427929 name: Cosine Ndcg@10 - type: cosine_mrr@10 value: 0.3367460317460317 name: Cosine Mrr@10 - type: cosine_map@100 value: 0.04901233559063261 name: Cosine Map@100 - task: type: information-retrieval name: Information Retrieval dataset: name: NanoNQ type: NanoNQ metrics: - type: cosine_accuracy@1 value: 0.14 name: Cosine Accuracy@1 - type: cosine_accuracy@3 value: 0.36 name: Cosine Accuracy@3 - type: cosine_accuracy@5 value: 0.44 name: Cosine Accuracy@5 - type: cosine_accuracy@10 value: 0.58 name: Cosine Accuracy@10 - type: cosine_precision@1 value: 0.14 name: Cosine Precision@1 - type: cosine_precision@3 value: 0.11999999999999998 name: Cosine Precision@3 - type: cosine_precision@5 value: 0.08800000000000002 name: Cosine Precision@5 - type: cosine_precision@10 value: 0.06000000000000001 name: Cosine Precision@10 - type: cosine_recall@1 value: 0.13 name: Cosine Recall@1 - type: cosine_recall@3 value: 0.34 name: Cosine Recall@3 - type: cosine_recall@5 value: 0.41 name: Cosine Recall@5 - type: cosine_recall@10 value: 0.55 name: Cosine Recall@10 - type: cosine_ndcg@10 value: 0.33223439819785083 name: Cosine Ndcg@10 - type: cosine_mrr@10 value: 0.2734365079365079 name: Cosine Mrr@10 - type: cosine_map@100 value: 0.2764557370904448 name: Cosine Map@100 - task: type: information-retrieval name: Information Retrieval dataset: name: NanoQuoraRetrieval type: NanoQuoraRetrieval metrics: - type: cosine_accuracy@1 value: 0.82 name: Cosine Accuracy@1 - type: cosine_accuracy@3 value: 0.9 name: Cosine Accuracy@3 - type: cosine_accuracy@5 value: 0.92 name: Cosine Accuracy@5 - type: cosine_accuracy@10 value: 0.96 name: Cosine Accuracy@10 - type: cosine_precision@1 value: 0.82 name: Cosine Precision@1 - type: cosine_precision@3 value: 0.3666666666666666 name: Cosine Precision@3 - type: cosine_precision@5 value: 0.244 name: Cosine Precision@5 - type: cosine_precision@10 value: 0.13399999999999998 name: Cosine Precision@10 - type: cosine_recall@1 value: 0.7206666666666667 name: Cosine Recall@1 - type: cosine_recall@3 value: 0.8553333333333333 name: Cosine Recall@3 - type: cosine_recall@5 value: 0.8993333333333333 name: Cosine Recall@5 - type: cosine_recall@10 value: 0.9566666666666666 name: Cosine Recall@10 - type: cosine_ndcg@10 value: 0.8807317086981499 name: Cosine Ndcg@10 - type: cosine_mrr@10 value: 0.8616666666666666 name: Cosine Mrr@10 - type: cosine_map@100 value: 0.8525831566094724 name: Cosine Map@100 - task: type: information-retrieval name: Information Retrieval dataset: name: NanoSCIDOCS type: NanoSCIDOCS metrics: - type: cosine_accuracy@1 value: 0.34 name: Cosine Accuracy@1 - type: cosine_accuracy@3 value: 0.48 name: Cosine Accuracy@3 - type: cosine_accuracy@5 value: 0.54 name: Cosine Accuracy@5 - type: cosine_accuracy@10 value: 0.66 name: Cosine Accuracy@10 - type: cosine_precision@1 value: 0.34 name: Cosine Precision@1 - type: cosine_precision@3 value: 0.24666666666666667 name: Cosine Precision@3 - type: cosine_precision@5 value: 0.212 name: Cosine Precision@5 - type: cosine_precision@10 value: 0.14800000000000002 name: Cosine Precision@10 - type: cosine_recall@1 value: 0.07066666666666668 name: Cosine Recall@1 - type: cosine_recall@3 value: 0.15366666666666667 name: Cosine Recall@3 - type: cosine_recall@5 value: 0.21866666666666668 name: Cosine Recall@5 - type: cosine_recall@10 value: 0.30466666666666664 name: Cosine Recall@10 - type: cosine_ndcg@10 value: 0.28968259227673265 name: Cosine Ndcg@10 - type: cosine_mrr@10 value: 0.4286349206349206 name: Cosine Mrr@10 - type: cosine_map@100 value: 0.22985309744949503 name: Cosine Map@100 - task: type: information-retrieval name: Information Retrieval dataset: name: NanoArguAna type: NanoArguAna metrics: - type: cosine_accuracy@1 value: 0.18 name: Cosine Accuracy@1 - type: cosine_accuracy@3 value: 0.56 name: Cosine Accuracy@3 - type: cosine_accuracy@5 value: 0.62 name: Cosine Accuracy@5 - type: cosine_accuracy@10 value: 0.84 name: Cosine Accuracy@10 - type: cosine_precision@1 value: 0.18 name: Cosine Precision@1 - type: cosine_precision@3 value: 0.18666666666666668 name: Cosine Precision@3 - type: cosine_precision@5 value: 0.124 name: Cosine Precision@5 - type: cosine_precision@10 value: 0.08399999999999999 name: Cosine Precision@10 - type: cosine_recall@1 value: 0.18 name: Cosine Recall@1 - type: cosine_recall@3 value: 0.56 name: Cosine Recall@3 - type: cosine_recall@5 value: 0.62 name: Cosine Recall@5 - type: cosine_recall@10 value: 0.84 name: Cosine Recall@10 - type: cosine_ndcg@10 value: 0.49726259302609505 name: Cosine Ndcg@10 - type: cosine_mrr@10 value: 0.389079365079365 name: Cosine Mrr@10 - type: cosine_map@100 value: 0.3967117258845785 name: Cosine Map@100 - task: type: information-retrieval name: Information Retrieval dataset: name: NanoSciFact type: NanoSciFact metrics: - type: cosine_accuracy@1 value: 0.38 name: Cosine Accuracy@1 - type: cosine_accuracy@3 value: 0.46 name: Cosine Accuracy@3 - type: cosine_accuracy@5 value: 0.48 name: Cosine Accuracy@5 - type: cosine_accuracy@10 value: 0.62 name: Cosine Accuracy@10 - type: cosine_precision@1 value: 0.38 name: Cosine Precision@1 - type: cosine_precision@3 value: 0.16666666666666663 name: Cosine Precision@3 - type: cosine_precision@5 value: 0.10400000000000001 name: Cosine Precision@5 - type: cosine_precision@10 value: 0.068 name: Cosine Precision@10 - type: cosine_recall@1 value: 0.345 name: Cosine Recall@1 - type: cosine_recall@3 value: 0.44 name: Cosine Recall@3 - type: cosine_recall@5 value: 0.46 name: Cosine Recall@5 - type: cosine_recall@10 value: 0.605 name: Cosine Recall@10 - type: cosine_ndcg@10 value: 0.47012843706683605 name: Cosine Ndcg@10 - type: cosine_mrr@10 value: 0.4409285714285714 name: Cosine Mrr@10 - type: cosine_map@100 value: 0.43840522432574647 name: Cosine Map@100 - task: type: information-retrieval name: Information Retrieval dataset: name: NanoTouche2020 type: NanoTouche2020 metrics: - type: cosine_accuracy@1 value: 0.5306122448979592 name: Cosine Accuracy@1 - type: cosine_accuracy@3 value: 0.7551020408163265 name: Cosine Accuracy@3 - type: cosine_accuracy@5 value: 0.8571428571428571 name: Cosine Accuracy@5 - type: cosine_accuracy@10 value: 0.9387755102040817 name: Cosine Accuracy@10 - type: cosine_precision@1 value: 0.5306122448979592 name: Cosine Precision@1 - type: cosine_precision@3 value: 0.45578231292517 name: Cosine Precision@3 - type: cosine_precision@5 value: 0.4040816326530612 name: Cosine Precision@5 - type: cosine_precision@10 value: 0.336734693877551 name: Cosine Precision@10 - type: cosine_recall@1 value: 0.03881638827876476 name: Cosine Recall@1 - type: cosine_recall@3 value: 0.10008002766114979 name: Cosine Recall@3 - type: cosine_recall@5 value: 0.13975964122053652 name: Cosine Recall@5 - type: cosine_recall@10 value: 0.22966349775526734 name: Cosine Recall@10 - type: cosine_ndcg@10 value: 0.39339080810676896 name: Cosine Ndcg@10 - type: cosine_mrr@10 value: 0.6553206997084549 name: Cosine Mrr@10 - type: cosine_map@100 value: 0.31344772891929434 name: Cosine Map@100 - task: type: nano-beir name: Nano BEIR dataset: name: NanoBEIR mean type: NanoBEIR_mean metrics: - type: cosine_accuracy@1 value: 0.3408163265306122 name: Cosine Accuracy@1 - type: cosine_accuracy@3 value: 0.5227001569858712 name: Cosine Accuracy@3 - type: cosine_accuracy@5 value: 0.6013186813186814 name: Cosine Accuracy@5 - type: cosine_accuracy@10 value: 0.7152904238618524 name: Cosine Accuracy@10 - type: cosine_precision@1 value: 0.3408163265306122 name: Cosine Precision@1 - type: cosine_precision@3 value: 0.23044479330193612 name: Cosine Precision@3 - type: cosine_precision@5 value: 0.1855447409733124 name: Cosine Precision@5 - type: cosine_precision@10 value: 0.13344113029827318 name: Cosine Precision@10 - type: cosine_recall@1 value: 0.18442678521033212 name: Cosine Recall@1 - type: cosine_recall@3 value: 0.31958052337482684 name: Cosine Recall@3 - type: cosine_recall@5 value: 0.3827680868002465 name: Cosine Recall@5 - type: cosine_recall@10 value: 0.4886833850587655 name: Cosine Recall@10 - type: cosine_ndcg@10 value: 0.4066287047188099 name: Cosine Ndcg@10 - type: cosine_mrr@10 value: 0.4531247913084647 name: Cosine Mrr@10 - type: cosine_map@100 value: 0.33618027996100497 name: Cosine Map@100 --- # MPNet base trained on Natural Questions pairs This is a [sentence-transformers](https://www.SBERT.net) model finetuned from [microsoft/mpnet-base](https://huggingface.co/microsoft/mpnet-base) on the [gooaq-hard-negatives](https://huggingface.co/datasets/tomaarsen/gooaq-hard-negatives) dataset. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more. ## Model Details ### Model Description - **Model Type:** Sentence Transformer - **Base model:** [microsoft/mpnet-base](https://huggingface.co/microsoft/mpnet-base) - **Maximum Sequence Length:** 512 tokens - **Output Dimensionality:** 768 dimensions - **Similarity Function:** Cosine Similarity - **Training Dataset:** - [gooaq-hard-negatives](https://huggingface.co/datasets/tomaarsen/gooaq-hard-negatives) - **Language:** en - **License:** apache-2.0 ### Model Sources - **Documentation:** [Sentence Transformers Documentation](https://sbert.net) - **Repository:** [Sentence Transformers on GitHub](https://github.com/UKPLab/sentence-transformers) - **Hugging Face:** [Sentence Transformers on Hugging Face](https://huggingface.co/models?library=sentence-transformers) ### Full Model Architecture ``` SentenceTransformer( (0): Transformer({'max_seq_length': 512, 'do_lower_case': False}) with Transformer model: MPNetModel (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True}) ) ``` ## Usage ### Direct Usage (Sentence Transformers) First install the Sentence Transformers library: ```bash pip install -U sentence-transformers ``` Then you can load this model and run inference. ```python from sentence_transformers import SentenceTransformer # Download from the 🤗 Hub model = SentenceTransformer("tomaarsen/mpnet-base-nq-cgist-triplet-neg-gte") # Run inference sentences = [ 'what energy is released when coal is burned?', 'When coal is burned, it reacts with the oxygen in the air. This chemical reaction converts the stored solar energy into thermal energy, which is released as heat. But it also produces carbon dioxide and methane.', 'When coal is burned it releases a number of airborne toxins and pollutants. They include mercury, lead, sulfur dioxide, nitrogen oxides, particulates, and various other heavy metals.', ] embeddings = model.encode(sentences) print(embeddings.shape) # [3, 768] # Get the similarity scores for the embeddings similarities = model.similarity(embeddings, embeddings) print(similarities.shape) # [3, 3] ``` ## Evaluation ### Metrics #### Information Retrieval * Datasets: `NanoClimateFEVER`, `NanoDBPedia`, `NanoFEVER`, `NanoFiQA2018`, `NanoHotpotQA`, `NanoMSMARCO`, `NanoNFCorpus`, `NanoNQ`, `NanoQuoraRetrieval`, `NanoSCIDOCS`, `NanoArguAna`, `NanoSciFact` and `NanoTouche2020` * Evaluated with [InformationRetrievalEvaluator](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.InformationRetrievalEvaluator) | Metric | NanoClimateFEVER | NanoDBPedia | NanoFEVER | NanoFiQA2018 | NanoHotpotQA | NanoMSMARCO | NanoNFCorpus | NanoNQ | NanoQuoraRetrieval | NanoSCIDOCS | NanoArguAna | NanoSciFact | NanoTouche2020 | |:--------------------|:-----------------|:------------|:-----------|:-------------|:-------------|:------------|:-------------|:-----------|:-------------------|:------------|:------------|:------------|:---------------| | cosine_accuracy@1 | 0.22 | 0.46 | 0.38 | 0.28 | 0.34 | 0.1 | 0.26 | 0.14 | 0.82 | 0.34 | 0.18 | 0.38 | 0.5306 | | cosine_accuracy@3 | 0.44 | 0.62 | 0.54 | 0.5 | 0.52 | 0.28 | 0.38 | 0.36 | 0.9 | 0.48 | 0.56 | 0.46 | 0.7551 | | cosine_accuracy@5 | 0.52 | 0.76 | 0.58 | 0.52 | 0.62 | 0.52 | 0.44 | 0.44 | 0.92 | 0.54 | 0.62 | 0.48 | 0.8571 | | cosine_accuracy@10 | 0.72 | 0.82 | 0.68 | 0.58 | 0.72 | 0.68 | 0.5 | 0.58 | 0.96 | 0.66 | 0.84 | 0.62 | 0.9388 | | cosine_precision@1 | 0.22 | 0.46 | 0.38 | 0.28 | 0.34 | 0.1 | 0.26 | 0.14 | 0.82 | 0.34 | 0.18 | 0.38 | 0.5306 | | cosine_precision@3 | 0.1667 | 0.3867 | 0.18 | 0.22 | 0.1933 | 0.0933 | 0.2133 | 0.12 | 0.3667 | 0.2467 | 0.1867 | 0.1667 | 0.4558 | | cosine_precision@5 | 0.12 | 0.388 | 0.12 | 0.164 | 0.144 | 0.104 | 0.196 | 0.088 | 0.244 | 0.212 | 0.124 | 0.104 | 0.4041 | | cosine_precision@10 | 0.094 | 0.344 | 0.07 | 0.098 | 0.092 | 0.068 | 0.138 | 0.06 | 0.134 | 0.148 | 0.084 | 0.068 | 0.3367 | | cosine_recall@1 | 0.0933 | 0.0307 | 0.37 | 0.1372 | 0.17 | 0.1 | 0.0112 | 0.13 | 0.7207 | 0.0707 | 0.18 | 0.345 | 0.0388 | | cosine_recall@3 | 0.195 | 0.0773 | 0.52 | 0.3227 | 0.29 | 0.28 | 0.0205 | 0.34 | 0.8553 | 0.1537 | 0.56 | 0.44 | 0.1001 | | cosine_recall@5 | 0.2333 | 0.1459 | 0.57 | 0.3682 | 0.36 | 0.52 | 0.0308 | 0.41 | 0.8993 | 0.2187 | 0.62 | 0.46 | 0.1398 | | cosine_recall@10 | 0.3723 | 0.2216 | 0.66 | 0.4307 | 0.46 | 0.68 | 0.0422 | 0.55 | 0.9567 | 0.3047 | 0.84 | 0.605 | 0.2297 | | **cosine_ndcg@10** | **0.2744** | **0.3921** | **0.5157** | **0.342** | **0.3723** | **0.3608** | **0.1655** | **0.3322** | **0.8807** | **0.2897** | **0.4973** | **0.4701** | **0.3934** | | cosine_mrr@10 | 0.3594 | 0.567 | 0.4757 | 0.3841 | 0.4571 | 0.2616 | 0.3367 | 0.2734 | 0.8617 | 0.4286 | 0.3891 | 0.4409 | 0.6553 | | cosine_map@100 | 0.2018 | 0.2815 | 0.4762 | 0.2826 | 0.2995 | 0.2722 | 0.049 | 0.2765 | 0.8526 | 0.2299 | 0.3967 | 0.4384 | 0.3134 | #### Nano BEIR * Dataset: `NanoBEIR_mean` * Evaluated with [NanoBEIREvaluator](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.NanoBEIREvaluator) | Metric | Value | |:--------------------|:-----------| | cosine_accuracy@1 | 0.3408 | | cosine_accuracy@3 | 0.5227 | | cosine_accuracy@5 | 0.6013 | | cosine_accuracy@10 | 0.7153 | | cosine_precision@1 | 0.3408 | | cosine_precision@3 | 0.2304 | | cosine_precision@5 | 0.1855 | | cosine_precision@10 | 0.1334 | | cosine_recall@1 | 0.1844 | | cosine_recall@3 | 0.3196 | | cosine_recall@5 | 0.3828 | | cosine_recall@10 | 0.4887 | | **cosine_ndcg@10** | **0.4066** | | cosine_mrr@10 | 0.4531 | | cosine_map@100 | 0.3362 | ## Training Details ### Training Dataset #### gooaq-hard-negatives * Dataset: [gooaq-hard-negatives](https://huggingface.co/datasets/tomaarsen/gooaq-hard-negatives) at [87594a1](https://huggingface.co/datasets/tomaarsen/gooaq-hard-negatives/tree/87594a1e6c58e88b5843afa9da3a97ffd75d01c2) * Size: 50,000 training samples * Columns: question, answer, and negative * Approximate statistics based on the first 1000 samples: | | question | answer | negative | |:--------|:----------------------------------------------------------------------------------|:------------------------------------------------------------------------------------|:------------------------------------------------------------------------------------| | type | string | string | string | | details | | | | * Samples: | question | answer | negative | |:--------------------------------------------------------------------------------------|:--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | what is the difference between calories from fat and total fat? | Fat has more than twice as many calories per gram as carbohydrates and proteins. A gram of fat has about 9 calories, while a gram of carbohydrate or protein has about 4 calories. In other words, you could eat twice as much carbohydrates or proteins as fat for the same amount of calories. | Fat has more than twice as many calories per gram as carbohydrates and proteins. A gram of fat has about 9 calories, while a gram of carbohydrate or protein has about 4 calories. In other words, you could eat twice as much carbohydrates or proteins as fat for the same amount of calories. | | what is the difference between return transcript and account transcript? | A tax return transcript usually meets the needs of lending institutions offering mortgages and student loans. ... Tax Account Transcript - shows basic data such as return type, marital status, adjusted gross income, taxable income and all payment types. It also shows changes made after you filed your original return. | Trial balance is not a financial statement whereas a balance sheet is a financial statement. Trial balance is solely used for internal purposes whereas a balance sheet is used for purposes other than internal i.e. external. In a trial balance, each and every account is divided into debit (dr.) and credit (cr.) | | how long does my dog need to fast before sedation? | Now, guidelines are aimed towards 6-8 hours before surgery. This pre-op fasting time is much more beneficial for your pets because you have enough food in there to neutralize the stomach acid, preventing it from coming up the esophagus that causes regurgitation under anesthetic. | Try not to let your pooch rapidly wolf down his/her food! Do not let the dog play or exercise (e.g. go for a walk) for at least two hours after having a meal. Ensure continuous fresh water is available to avoid your pet gulping down a large amount after eating. | * Loss: [CachedGISTEmbedLoss](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#cachedgistembedloss) with these parameters: ```json {'guide': SentenceTransformer( (0): Transformer({'max_seq_length': 256, 'do_lower_case': False}) with Transformer model: BertModel (1): Pooling({'word_embedding_dimension': 384, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True}) (2): Normalize() ), 'temperature': 0.01} ``` ### Evaluation Dataset #### gooaq-hard-negatives * Dataset: [gooaq-hard-negatives](https://huggingface.co/datasets/tomaarsen/gooaq-hard-negatives) at [87594a1](https://huggingface.co/datasets/tomaarsen/gooaq-hard-negatives/tree/87594a1e6c58e88b5843afa9da3a97ffd75d01c2) * Size: 10,048,700 evaluation samples * Columns: question, answer, and negative * Approximate statistics based on the first 1000 samples: | | question | answer | negative | |:--------|:----------------------------------------------------------------------------------|:------------------------------------------------------------------------------------|:------------------------------------------------------------------------------------| | type | string | string | string | | details | | | | * Samples: | question | answer | negative | |:--------------------------------------------------------------------|:----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | how is height width and length written? | The Graphics' industry standard is width by height (width x height). Meaning that when you write your measurements, you write them from your point of view, beginning with the width. | The Graphics' industry standard is width by height (width x height). Meaning that when you write your measurements, you write them from your point of view, beginning with the width. That's important. | | what is the difference between pork shoulder and loin? | All the recipes I've found for pulled pork recommends a shoulder/butt. Shoulders take longer to cook than a loin, because they're tougher. Loins are lean, while shoulders have marbled fat inside. | They are extracted from the loin, which runs from the hip to the shoulder, and it has a small strip of meat called the tenderloin. Unlike other pork, this pork chop is cut from four major sections, which are the shoulder, also known as the blade chops, ribs chops, loin chops, and the last, which is the sirloin chops. | | is the yin yang symbol religious? | The ubiquitous yin-yang symbol holds its roots in Taoism/Daoism, a Chinese religion and philosophy. The yin, the dark swirl, is associated with shadows, femininity, and the trough of a wave; the yang, the light swirl, represents brightness, passion and growth. | Yin energy is in the calm colors around you, in the soft music, in the soothing sound of a water fountain, or the relaxing images of water. Yang (active energy) is the feng shui energy expressed in strong, vibrant sounds and colors, bright lights, upward moving energy, tall plants, etc. | * Loss: [CachedGISTEmbedLoss](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#cachedgistembedloss) with these parameters: ```json {'guide': SentenceTransformer( (0): Transformer({'max_seq_length': 256, 'do_lower_case': False}) with Transformer model: BertModel (1): Pooling({'word_embedding_dimension': 384, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True}) (2): Normalize() ), 'temperature': 0.01} ``` ### Training Hyperparameters #### Non-Default Hyperparameters - `eval_strategy`: steps - `per_device_train_batch_size`: 2048 - `per_device_eval_batch_size`: 2048 - `learning_rate`: 2e-05 - `num_train_epochs`: 1 - `warmup_ratio`: 0.1 - `seed`: 12 - `bf16`: True #### All Hyperparameters
Click to expand - `overwrite_output_dir`: False - `do_predict`: False - `eval_strategy`: steps - `prediction_loss_only`: True - `per_device_train_batch_size`: 2048 - `per_device_eval_batch_size`: 2048 - `per_gpu_train_batch_size`: None - `per_gpu_eval_batch_size`: None - `gradient_accumulation_steps`: 1 - `eval_accumulation_steps`: None - `torch_empty_cache_steps`: None - `learning_rate`: 2e-05 - `weight_decay`: 0.0 - `adam_beta1`: 0.9 - `adam_beta2`: 0.999 - `adam_epsilon`: 1e-08 - `max_grad_norm`: 1.0 - `num_train_epochs`: 1 - `max_steps`: -1 - `lr_scheduler_type`: linear - `lr_scheduler_kwargs`: {} - `warmup_ratio`: 0.1 - `warmup_steps`: 0 - `log_level`: passive - `log_level_replica`: warning - `log_on_each_node`: True - `logging_nan_inf_filter`: True - `save_safetensors`: True - `save_on_each_node`: False - `save_only_model`: False - `restore_callback_states_from_checkpoint`: False - `no_cuda`: False - `use_cpu`: False - `use_mps_device`: False - `seed`: 12 - `data_seed`: None - `jit_mode_eval`: False - `use_ipex`: False - `bf16`: True - `fp16`: False - `fp16_opt_level`: O1 - `half_precision_backend`: auto - `bf16_full_eval`: False - `fp16_full_eval`: False - `tf32`: None - `local_rank`: 0 - `ddp_backend`: None - `tpu_num_cores`: None - `tpu_metrics_debug`: False - `debug`: [] - `dataloader_drop_last`: False - `dataloader_num_workers`: 0 - `dataloader_prefetch_factor`: None - `past_index`: -1 - `disable_tqdm`: False - `remove_unused_columns`: True - `label_names`: None - `load_best_model_at_end`: False - `ignore_data_skip`: False - `fsdp`: [] - `fsdp_min_num_params`: 0 - `fsdp_config`: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False} - `fsdp_transformer_layer_cls_to_wrap`: None - `accelerator_config`: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None} - `deepspeed`: None - `label_smoothing_factor`: 0.0 - `optim`: adamw_torch - `optim_args`: None - `adafactor`: False - `group_by_length`: False - `length_column_name`: length - `ddp_find_unused_parameters`: None - `ddp_bucket_cap_mb`: None - `ddp_broadcast_buffers`: False - `dataloader_pin_memory`: True - `dataloader_persistent_workers`: False - `skip_memory_metrics`: True - `use_legacy_prediction_loop`: False - `push_to_hub`: False - `resume_from_checkpoint`: None - `hub_model_id`: None - `hub_strategy`: every_save - `hub_private_repo`: False - `hub_always_push`: False - `gradient_checkpointing`: False - `gradient_checkpointing_kwargs`: None - `include_inputs_for_metrics`: False - `include_for_metrics`: [] - `eval_do_concat_batches`: True - `fp16_backend`: auto - `push_to_hub_model_id`: None - `push_to_hub_organization`: None - `mp_parameters`: - `auto_find_batch_size`: False - `full_determinism`: False - `torchdynamo`: None - `ray_scope`: last - `ddp_timeout`: 1800 - `torch_compile`: False - `torch_compile_backend`: None - `torch_compile_mode`: None - `dispatch_batches`: None - `split_batches`: None - `include_tokens_per_second`: False - `include_num_input_tokens_seen`: False - `neftune_noise_alpha`: None - `optim_target_modules`: None - `batch_eval_metrics`: False - `eval_on_start`: False - `use_liger_kernel`: False - `eval_use_gather_object`: False - `average_tokens_across_devices`: False - `prompts`: None - `batch_sampler`: batch_sampler - `multi_dataset_batch_sampler`: proportional
### Training Logs | Epoch | Step | Training Loss | Validation Loss | NanoClimateFEVER_cosine_ndcg@10 | NanoDBPedia_cosine_ndcg@10 | NanoFEVER_cosine_ndcg@10 | NanoFiQA2018_cosine_ndcg@10 | NanoHotpotQA_cosine_ndcg@10 | NanoMSMARCO_cosine_ndcg@10 | NanoNFCorpus_cosine_ndcg@10 | NanoNQ_cosine_ndcg@10 | NanoQuoraRetrieval_cosine_ndcg@10 | NanoSCIDOCS_cosine_ndcg@10 | NanoArguAna_cosine_ndcg@10 | NanoSciFact_cosine_ndcg@10 | NanoTouche2020_cosine_ndcg@10 | NanoBEIR_mean_cosine_ndcg@10 | |:-----:|:----:|:-------------:|:---------------:|:-------------------------------:|:--------------------------:|:------------------------:|:---------------------------:|:---------------------------:|:--------------------------:|:---------------------------:|:---------------------:|:---------------------------------:|:--------------------------:|:--------------------------:|:--------------------------:|:-----------------------------:|:----------------------------:| | 0.04 | 1 | 11.5141 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.2 | 5 | 9.4407 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.4 | 10 | 5.6005 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.6 | 15 | 3.7323 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.8 | 20 | 2.7976 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 1.0 | 25 | 2.1899 | 1.3429 | 0.2744 | 0.3921 | 0.5157 | 0.3420 | 0.3723 | 0.3608 | 0.1655 | 0.3322 | 0.8807 | 0.2897 | 0.4973 | 0.4701 | 0.3934 | 0.4066 | ### Environmental Impact Carbon emissions were measured using [CodeCarbon](https://github.com/mlco2/codecarbon). - **Energy Consumed**: 0.104 kWh - **Carbon Emitted**: 0.041 kg of CO2 - **Hours Used**: 0.301 hours ### Training Hardware - **On Cloud**: No - **GPU Model**: 1 x NVIDIA GeForce RTX 3090 - **CPU Model**: 13th Gen Intel(R) Core(TM) i7-13700K - **RAM Size**: 31.78 GB ### Framework Versions - Python: 3.11.6 - Sentence Transformers: 3.4.0.dev0 - Transformers: 4.46.2 - PyTorch: 2.5.0+cu121 - Accelerate: 0.35.0.dev0 - Datasets: 2.20.0 - Tokenizers: 0.20.3 ## Citation ### BibTeX #### Sentence Transformers ```bibtex @inproceedings{reimers-2019-sentence-bert, title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks", author = "Reimers, Nils and Gurevych, Iryna", booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing", month = "11", year = "2019", publisher = "Association for Computational Linguistics", url = "https://arxiv.org/abs/1908.10084", } ```