metadata

language:
  - en
license: apache-2.0
tags:
  - sentence-transformers
  - sentence-similarity
  - feature-extraction
  - generated_from_trainer
  - dataset_size:50000
  - loss:CachedGISTEmbedLoss
base_model: microsoft/mpnet-base
widget:
  - source_sentence: what does the accounts receivable turnover measure?
    sentences:
      - >-
        The accounts receivable turnover ratio is an accounting measure used to
        quantify a company's effectiveness in collecting its receivables or
        money owed by clients. The ratio shows how well a company uses and
        manages the credit it extends to customers and how quickly that
        short-term debt is collected or is paid.
      - >-
        Capital budgeting, and investment appraisal, is the planning process
        used to determine whether an organization's long term investments such
        as new machinery, replacement of machinery, new plants, new products,
        and research development projects are worth the funding of cash through
        the firm's capitalization structure ( ...
      - >-
        The accounts receivable turnover ratio is an accounting measure used to
        quantify a company's effectiveness in collecting its receivables or
        money owed by clients. The ratio shows how well a company uses and
        manages the credit it extends to customers and how quickly that
        short-term debt is collected or is paid.
  - source_sentence: does gabapentin cause liver problems?
    sentences:
      - >-
        Gabapentin has no appreciable liver metabolism, yet, suspected cases of
        gabapentin-induced hepatotoxicity have been reported. Per literature
        review, two cases of possible gabapentin-induced liver injury have been
        reported.
      - >-
        Strongholds are a type of story mission which only unlocks after enough
        progression through the game. There are three Stronghold's during the
        first section of progression through The Division 2. You'll need to
        complete the first two and have reached level 30 before being able to
        unlock the final Stronghold.
      - >-
        The most-common side effects attributed to Gabapentin include mild
        sedation, ataxia, and occasional diarrhea. Sedation can be minimized by
        tapering from a smaller starting dose to the desired dose. When treating
        seizures, it is ideal to wean off the drug to reduce the risk of
        withdrawal seizures.
  - source_sentence: how long should you wait to give blood after eating?
    sentences:
      - >-
        Until the bleeding has stopped it is natural to taste blood or to see
        traces of blood in your saliva. You may stop using gauze after the flow
        stops – usually around 8 hours after surgery.
      - >-
        Before donation The first and most important rule—never donate blood on
        an empty stomach. “Eat a wholesome meal about 2-3 hours before donating
        to keep your blood sugar stable," says Dr Chaturvedi. The timing of the
        meal is important too. You need to allow the food to be digested
        properly before the blood is drawn.
      - >-
        While grid computing involves virtualizing computing resources to store
        massive amounts of data, whereas cloud computing is where an application
        doesn't access resources directly, rather it accesses them through a
        service over the internet. ...
  - source_sentence: what is the difference between chicken francese and chicken marsala?
    sentences:
      - >-
        Chicken is the species name, equivalent to our “human.” Rooster is an
        adult male, equivalent to “man.” Hen is an adult female, equivalent to
        “woman.” Cockerel is a juvenile male, equivalent to “boy/young man.”
      - What is 99 kg in pounds? - 99 kg is equal to 218.26 pounds.
      - >-
        The difference between the two is for Francese, the chicken breast is
        first dipped in flour, then into a beaten egg mixture, before being
        cooked. For piccata, the chicken is first dipped in egg and then in
        flour. Both are then simmered in a lemony butter sauce, but the piccata
        sauce includes capers.”
  - source_sentence: what energy is released when coal is burned?
    sentences:
      - >-
        When coal is burned, it reacts with the oxygen in the air. This chemical
        reaction converts the stored solar energy into thermal energy, which is
        released as heat. But it also produces carbon dioxide and methane.
      - >-
        When coal is burned it releases a number of airborne toxins and
        pollutants. They include mercury, lead, sulfur dioxide, nitrogen oxides,
        particulates, and various other heavy metals.
      - >-
        Squad Building Challenges allow you to exchange sets of players for
        coins, packs, and special items in FUT 20. Each of these challenges come
        with specific requirements, such as including players from certain
        teams. ... Live SBCs are time-limited challenges which often give out
        unique, high-rated versions of players.
datasets:
  - tomaarsen/gooaq-hard-negatives
pipeline_tag: sentence-similarity
library_name: sentence-transformers
metrics:
  - cosine_accuracy@1
  - cosine_accuracy@3
  - cosine_accuracy@5
  - cosine_accuracy@10
  - cosine_precision@1
  - cosine_precision@3
  - cosine_precision@5
  - cosine_precision@10
  - cosine_recall@1
  - cosine_recall@3
  - cosine_recall@5
  - cosine_recall@10
  - cosine_ndcg@10
  - cosine_mrr@10
  - cosine_map@100
co2_eq_emissions:
  emissions: 40.414352043491995
  energy_consumed: 0.10397258579449552
  source: codecarbon
  training_type: fine-tuning
  on_cloud: false
  cpu_model: 13th Gen Intel(R) Core(TM) i7-13700K
  ram_total_size: 31.777088165283203
  hours_used: 0.297
  hardware_used: 1 x NVIDIA GeForce RTX 3090
model-index:
  - name: MPNet base trained on Natural Questions pairs
    results:
      - task:
          type: information-retrieval
          name: Information Retrieval
        dataset:
          name: NanoClimateFEVER
          type: NanoClimateFEVER
        metrics:
          - type: cosine_accuracy@1
            value: 0.22
            name: Cosine Accuracy@1
          - type: cosine_accuracy@3
            value: 0.44
            name: Cosine Accuracy@3
          - type: cosine_accuracy@5
            value: 0.5
            name: Cosine Accuracy@5
          - type: cosine_accuracy@10
            value: 0.72
            name: Cosine Accuracy@10
          - type: cosine_precision@1
            value: 0.22
            name: Cosine Precision@1
          - type: cosine_precision@3
            value: 0.16666666666666663
            name: Cosine Precision@3
          - type: cosine_precision@5
            value: 0.11599999999999999
            name: Cosine Precision@5
          - type: cosine_precision@10
            value: 0.092
            name: Cosine Precision@10
          - type: cosine_recall@1
            value: 0.09333333333333332
            name: Cosine Recall@1
          - type: cosine_recall@3
            value: 0.195
            name: Cosine Recall@3
          - type: cosine_recall@5
            value: 0.22666666666666666
            name: Cosine Recall@5
          - type: cosine_recall@10
            value: 0.36733333333333335
            name: Cosine Recall@10
          - type: cosine_ndcg@10
            value: 0.27399441682859144
            name: Cosine Ndcg@10
          - type: cosine_mrr@10
            value: 0.36331746031746026
            name: Cosine Mrr@10
          - type: cosine_map@100
            value: 0.20383864617106084
            name: Cosine Map@100
      - task:
          type: information-retrieval
          name: Information Retrieval
        dataset:
          name: NanoDBPedia
          type: NanoDBPedia
        metrics:
          - type: cosine_accuracy@1
            value: 0.46
            name: Cosine Accuracy@1
          - type: cosine_accuracy@3
            value: 0.64
            name: Cosine Accuracy@3
          - type: cosine_accuracy@5
            value: 0.76
            name: Cosine Accuracy@5
          - type: cosine_accuracy@10
            value: 0.84
            name: Cosine Accuracy@10
          - type: cosine_precision@1
            value: 0.46
            name: Cosine Precision@1
          - type: cosine_precision@3
            value: 0.4
            name: Cosine Precision@3
          - type: cosine_precision@5
            value: 0.3760000000000001
            name: Cosine Precision@5
          - type: cosine_precision@10
            value: 0.34
            name: Cosine Precision@10
          - type: cosine_recall@1
            value: 0.03065300183409328
            name: Cosine Recall@1
          - type: cosine_recall@3
            value: 0.07824513873584021
            name: Cosine Recall@3
          - type: cosine_recall@5
            value: 0.12190077086725051
            name: Cosine Recall@5
          - type: cosine_recall@10
            value: 0.21649668807903738
            name: Cosine Recall@10
          - type: cosine_ndcg@10
            value: 0.38922276974007985
            name: Cosine Ndcg@10
          - type: cosine_mrr@10
            value: 0.5768571428571428
            name: Cosine Mrr@10
          - type: cosine_map@100
            value: 0.27830317958127815
            name: Cosine Map@100
      - task:
          type: information-retrieval
          name: Information Retrieval
        dataset:
          name: NanoFEVER
          type: NanoFEVER
        metrics:
          - type: cosine_accuracy@1
            value: 0.38
            name: Cosine Accuracy@1
          - type: cosine_accuracy@3
            value: 0.54
            name: Cosine Accuracy@3
          - type: cosine_accuracy@5
            value: 0.58
            name: Cosine Accuracy@5
          - type: cosine_accuracy@10
            value: 0.7
            name: Cosine Accuracy@10
          - type: cosine_precision@1
            value: 0.38
            name: Cosine Precision@1
          - type: cosine_precision@3
            value: 0.18
            name: Cosine Precision@3
          - type: cosine_precision@5
            value: 0.12
            name: Cosine Precision@5
          - type: cosine_precision@10
            value: 0.07200000000000001
            name: Cosine Precision@10
          - type: cosine_recall@1
            value: 0.37
            name: Cosine Recall@1
          - type: cosine_recall@3
            value: 0.52
            name: Cosine Recall@3
          - type: cosine_recall@5
            value: 0.57
            name: Cosine Recall@5
          - type: cosine_recall@10
            value: 0.68
            name: Cosine Recall@10
          - type: cosine_ndcg@10
            value: 0.5226736410648857
            name: Cosine Ndcg@10
          - type: cosine_mrr@10
            value: 0.4790238095238095
            name: Cosine Mrr@10
          - type: cosine_map@100
            value: 0.4780826341570998
            name: Cosine Map@100
      - task:
          type: information-retrieval
          name: Information Retrieval
        dataset:
          name: NanoFiQA2018
          type: NanoFiQA2018
        metrics:
          - type: cosine_accuracy@1
            value: 0.28
            name: Cosine Accuracy@1
          - type: cosine_accuracy@3
            value: 0.5
            name: Cosine Accuracy@3
          - type: cosine_accuracy@5
            value: 0.52
            name: Cosine Accuracy@5
          - type: cosine_accuracy@10
            value: 0.58
            name: Cosine Accuracy@10
          - type: cosine_precision@1
            value: 0.28
            name: Cosine Precision@1
          - type: cosine_precision@3
            value: 0.22
            name: Cosine Precision@3
          - type: cosine_precision@5
            value: 0.16
            name: Cosine Precision@5
          - type: cosine_precision@10
            value: 0.09599999999999997
            name: Cosine Precision@10
          - type: cosine_recall@1
            value: 0.1371904761904762
            name: Cosine Recall@1
          - type: cosine_recall@3
            value: 0.32535714285714284
            name: Cosine Recall@3
          - type: cosine_recall@5
            value: 0.3653571428571428
            name: Cosine Recall@5
          - type: cosine_recall@10
            value: 0.42940476190476184
            name: Cosine Recall@10
          - type: cosine_ndcg@10
            value: 0.34407947120145826
            name: Cosine Ndcg@10
          - type: cosine_mrr@10
            value: 0.3881666666666667
            name: Cosine Mrr@10
          - type: cosine_map@100
            value: 0.2878001762783797
            name: Cosine Map@100
      - task:
          type: information-retrieval
          name: Information Retrieval
        dataset:
          name: NanoHotpotQA
          type: NanoHotpotQA
        metrics:
          - type: cosine_accuracy@1
            value: 0.34
            name: Cosine Accuracy@1
          - type: cosine_accuracy@3
            value: 0.58
            name: Cosine Accuracy@3
          - type: cosine_accuracy@5
            value: 0.64
            name: Cosine Accuracy@5
          - type: cosine_accuracy@10
            value: 0.74
            name: Cosine Accuracy@10
          - type: cosine_precision@1
            value: 0.34
            name: Cosine Precision@1
          - type: cosine_precision@3
            value: 0.21333333333333332
            name: Cosine Precision@3
          - type: cosine_precision@5
            value: 0.14800000000000002
            name: Cosine Precision@5
          - type: cosine_precision@10
            value: 0.094
            name: Cosine Precision@10
          - type: cosine_recall@1
            value: 0.17
            name: Cosine Recall@1
          - type: cosine_recall@3
            value: 0.32
            name: Cosine Recall@3
          - type: cosine_recall@5
            value: 0.37
            name: Cosine Recall@5
          - type: cosine_recall@10
            value: 0.47
            name: Cosine Recall@10
          - type: cosine_ndcg@10
            value: 0.3814617400581295
            name: Cosine Ndcg@10
          - type: cosine_mrr@10
            value: 0.4695793650793651
            name: Cosine Mrr@10
          - type: cosine_map@100
            value: 0.30635937490171045
            name: Cosine Map@100
      - task:
          type: information-retrieval
          name: Information Retrieval
        dataset:
          name: NanoMSMARCO
          type: NanoMSMARCO
        metrics:
          - type: cosine_accuracy@1
            value: 0.12
            name: Cosine Accuracy@1
          - type: cosine_accuracy@3
            value: 0.3
            name: Cosine Accuracy@3
          - type: cosine_accuracy@5
            value: 0.56
            name: Cosine Accuracy@5
          - type: cosine_accuracy@10
            value: 0.66
            name: Cosine Accuracy@10
          - type: cosine_precision@1
            value: 0.12
            name: Cosine Precision@1
          - type: cosine_precision@3
            value: 0.1
            name: Cosine Precision@3
          - type: cosine_precision@5
            value: 0.11200000000000002
            name: Cosine Precision@5
          - type: cosine_precision@10
            value: 0.066
            name: Cosine Precision@10
          - type: cosine_recall@1
            value: 0.12
            name: Cosine Recall@1
          - type: cosine_recall@3
            value: 0.3
            name: Cosine Recall@3
          - type: cosine_recall@5
            value: 0.56
            name: Cosine Recall@5
          - type: cosine_recall@10
            value: 0.66
            name: Cosine Recall@10
          - type: cosine_ndcg@10
            value: 0.3660111210554949
            name: Cosine Ndcg@10
          - type: cosine_mrr@10
            value: 0.27407936507936503
            name: Cosine Mrr@10
          - type: cosine_map@100
            value: 0.2870484235732714
            name: Cosine Map@100
      - task:
          type: information-retrieval
          name: Information Retrieval
        dataset:
          name: NanoNFCorpus
          type: NanoNFCorpus
        metrics:
          - type: cosine_accuracy@1
            value: 0.32
            name: Cosine Accuracy@1
          - type: cosine_accuracy@3
            value: 0.4
            name: Cosine Accuracy@3
          - type: cosine_accuracy@5
            value: 0.44
            name: Cosine Accuracy@5
          - type: cosine_accuracy@10
            value: 0.5
            name: Cosine Accuracy@10
          - type: cosine_precision@1
            value: 0.32
            name: Cosine Precision@1
          - type: cosine_precision@3
            value: 0.21333333333333332
            name: Cosine Precision@3
          - type: cosine_precision@5
            value: 0.196
            name: Cosine Precision@5
          - type: cosine_precision@10
            value: 0.144
            name: Cosine Precision@10
          - type: cosine_recall@1
            value: 0.012173283062756207
            name: Cosine Recall@1
          - type: cosine_recall@3
            value: 0.02038195250132044
            name: Cosine Recall@3
          - type: cosine_recall@5
            value: 0.028711609969173105
            name: Cosine Recall@5
          - type: cosine_recall@10
            value: 0.04001132454412933
            name: Cosine Recall@10
          - type: cosine_ndcg@10
            value: 0.17348600988460589
            name: Cosine Ndcg@10
          - type: cosine_mrr@10
            value: 0.37288095238095237
            name: Cosine Mrr@10
          - type: cosine_map@100
            value: 0.04905096401443591
            name: Cosine Map@100
      - task:
          type: information-retrieval
          name: Information Retrieval
        dataset:
          name: NanoNQ
          type: NanoNQ
        metrics:
          - type: cosine_accuracy@1
            value: 0.16
            name: Cosine Accuracy@1
          - type: cosine_accuracy@3
            value: 0.38
            name: Cosine Accuracy@3
          - type: cosine_accuracy@5
            value: 0.46
            name: Cosine Accuracy@5
          - type: cosine_accuracy@10
            value: 0.58
            name: Cosine Accuracy@10
          - type: cosine_precision@1
            value: 0.16
            name: Cosine Precision@1
          - type: cosine_precision@3
            value: 0.12666666666666665
            name: Cosine Precision@3
          - type: cosine_precision@5
            value: 0.09200000000000001
            name: Cosine Precision@5
          - type: cosine_precision@10
            value: 0.06000000000000001
            name: Cosine Precision@10
          - type: cosine_recall@1
            value: 0.15
            name: Cosine Recall@1
          - type: cosine_recall@3
            value: 0.36
            name: Cosine Recall@3
          - type: cosine_recall@5
            value: 0.43
            name: Cosine Recall@5
          - type: cosine_recall@10
            value: 0.54
            name: Cosine Recall@10
          - type: cosine_ndcg@10
            value: 0.34266003995975836
            name: Cosine Ndcg@10
          - type: cosine_mrr@10
            value: 0.2898015873015873
            name: Cosine Mrr@10
          - type: cosine_map@100
            value: 0.29298840458552056
            name: Cosine Map@100
      - task:
          type: information-retrieval
          name: Information Retrieval
        dataset:
          name: NanoQuoraRetrieval
          type: NanoQuoraRetrieval
        metrics:
          - type: cosine_accuracy@1
            value: 0.8
            name: Cosine Accuracy@1
          - type: cosine_accuracy@3
            value: 0.9
            name: Cosine Accuracy@3
          - type: cosine_accuracy@5
            value: 0.92
            name: Cosine Accuracy@5
          - type: cosine_accuracy@10
            value: 0.96
            name: Cosine Accuracy@10
          - type: cosine_precision@1
            value: 0.8
            name: Cosine Precision@1
          - type: cosine_precision@3
            value: 0.3666666666666666
            name: Cosine Precision@3
          - type: cosine_precision@5
            value: 0.244
            name: Cosine Precision@5
          - type: cosine_precision@10
            value: 0.13399999999999998
            name: Cosine Precision@10
          - type: cosine_recall@1
            value: 0.7006666666666667
            name: Cosine Recall@1
          - type: cosine_recall@3
            value: 0.8553333333333333
            name: Cosine Recall@3
          - type: cosine_recall@5
            value: 0.8993333333333333
            name: Cosine Recall@5
          - type: cosine_recall@10
            value: 0.9566666666666666
            name: Cosine Recall@10
          - type: cosine_ndcg@10
            value: 0.8760206475896655
            name: Cosine Ndcg@10
          - type: cosine_mrr@10
            value: 0.8545238095238095
            name: Cosine Mrr@10
          - type: cosine_map@100
            value: 0.8464432234432234
            name: Cosine Map@100
      - task:
          type: information-retrieval
          name: Information Retrieval
        dataset:
          name: NanoSCIDOCS
          type: NanoSCIDOCS
        metrics:
          - type: cosine_accuracy@1
            value: 0.36
            name: Cosine Accuracy@1
          - type: cosine_accuracy@3
            value: 0.48
            name: Cosine Accuracy@3
          - type: cosine_accuracy@5
            value: 0.54
            name: Cosine Accuracy@5
          - type: cosine_accuracy@10
            value: 0.68
            name: Cosine Accuracy@10
          - type: cosine_precision@1
            value: 0.36
            name: Cosine Precision@1
          - type: cosine_precision@3
            value: 0.25999999999999995
            name: Cosine Precision@3
          - type: cosine_precision@5
            value: 0.21600000000000003
            name: Cosine Precision@5
          - type: cosine_precision@10
            value: 0.15200000000000002
            name: Cosine Precision@10
          - type: cosine_recall@1
            value: 0.07566666666666669
            name: Cosine Recall@1
          - type: cosine_recall@3
            value: 0.16166666666666665
            name: Cosine Recall@3
          - type: cosine_recall@5
            value: 0.22266666666666668
            name: Cosine Recall@5
          - type: cosine_recall@10
            value: 0.31466666666666665
            name: Cosine Recall@10
          - type: cosine_ndcg@10
            value: 0.2989790025477086
            name: Cosine Ndcg@10
          - type: cosine_mrr@10
            value: 0.44196825396825395
            name: Cosine Mrr@10
          - type: cosine_map@100
            value: 0.23644335943955802
            name: Cosine Map@100
      - task:
          type: information-retrieval
          name: Information Retrieval
        dataset:
          name: NanoArguAna
          type: NanoArguAna
        metrics:
          - type: cosine_accuracy@1
            value: 0.18
            name: Cosine Accuracy@1
          - type: cosine_accuracy@3
            value: 0.56
            name: Cosine Accuracy@3
          - type: cosine_accuracy@5
            value: 0.64
            name: Cosine Accuracy@5
          - type: cosine_accuracy@10
            value: 0.84
            name: Cosine Accuracy@10
          - type: cosine_precision@1
            value: 0.18
            name: Cosine Precision@1
          - type: cosine_precision@3
            value: 0.18666666666666668
            name: Cosine Precision@3
          - type: cosine_precision@5
            value: 0.128
            name: Cosine Precision@5
          - type: cosine_precision@10
            value: 0.08399999999999999
            name: Cosine Precision@10
          - type: cosine_recall@1
            value: 0.18
            name: Cosine Recall@1
          - type: cosine_recall@3
            value: 0.56
            name: Cosine Recall@3
          - type: cosine_recall@5
            value: 0.64
            name: Cosine Recall@5
          - type: cosine_recall@10
            value: 0.84
            name: Cosine Recall@10
          - type: cosine_ndcg@10
            value: 0.5000949127836057
            name: Cosine Ndcg@10
          - type: cosine_mrr@10
            value: 0.392515873015873
            name: Cosine Mrr@10
          - type: cosine_map@100
            value: 0.4000896669795642
            name: Cosine Map@100
      - task:
          type: information-retrieval
          name: Information Retrieval
        dataset:
          name: NanoSciFact
          type: NanoSciFact
        metrics:
          - type: cosine_accuracy@1
            value: 0.36
            name: Cosine Accuracy@1
          - type: cosine_accuracy@3
            value: 0.46
            name: Cosine Accuracy@3
          - type: cosine_accuracy@5
            value: 0.48
            name: Cosine Accuracy@5
          - type: cosine_accuracy@10
            value: 0.62
            name: Cosine Accuracy@10
          - type: cosine_precision@1
            value: 0.36
            name: Cosine Precision@1
          - type: cosine_precision@3
            value: 0.16666666666666663
            name: Cosine Precision@3
          - type: cosine_precision@5
            value: 0.10400000000000001
            name: Cosine Precision@5
          - type: cosine_precision@10
            value: 0.068
            name: Cosine Precision@10
          - type: cosine_recall@1
            value: 0.325
            name: Cosine Recall@1
          - type: cosine_recall@3
            value: 0.44
            name: Cosine Recall@3
          - type: cosine_recall@5
            value: 0.46
            name: Cosine Recall@5
          - type: cosine_recall@10
            value: 0.605
            name: Cosine Recall@10
          - type: cosine_ndcg@10
            value: 0.4619142281323308
            name: Cosine Ndcg@10
          - type: cosine_mrr@10
            value: 0.43007936507936506
            name: Cosine Mrr@10
          - type: cosine_map@100
            value: 0.42744465636932816
            name: Cosine Map@100
      - task:
          type: information-retrieval
          name: Information Retrieval
        dataset:
          name: NanoTouche2020
          type: NanoTouche2020
        metrics:
          - type: cosine_accuracy@1
            value: 0.5306122448979592
            name: Cosine Accuracy@1
          - type: cosine_accuracy@3
            value: 0.7142857142857143
            name: Cosine Accuracy@3
          - type: cosine_accuracy@5
            value: 0.8571428571428571
            name: Cosine Accuracy@5
          - type: cosine_accuracy@10
            value: 0.9591836734693877
            name: Cosine Accuracy@10
          - type: cosine_precision@1
            value: 0.5306122448979592
            name: Cosine Precision@1
          - type: cosine_precision@3
            value: 0.44217687074829926
            name: Cosine Precision@3
          - type: cosine_precision@5
            value: 0.4122448979591837
            name: Cosine Precision@5
          - type: cosine_precision@10
            value: 0.3530612244897959
            name: Cosine Precision@10
          - type: cosine_recall@1
            value: 0.03881638827876476
            name: Cosine Recall@1
          - type: cosine_recall@3
            value: 0.09899647775241191
            name: Cosine Recall@3
          - type: cosine_recall@5
            value: 0.14503016807403868
            name: Cosine Recall@5
          - type: cosine_recall@10
            value: 0.23921899286976872
            name: Cosine Recall@10
          - type: cosine_ndcg@10
            value: 0.40436873020254516
            name: Cosine Ndcg@10
          - type: cosine_mrr@10
            value: 0.6520651117589893
            name: Cosine Mrr@10
          - type: cosine_map@100
            value: 0.3151658515102787
            name: Cosine Map@100
      - task:
          type: nano-beir
          name: Nano BEIR
        dataset:
          name: NanoBEIR mean
          type: NanoBEIR_mean
        metrics:
          - type: cosine_accuracy@1
            value: 0.3469701726844584
            name: Cosine Accuracy@1
          - type: cosine_accuracy@3
            value: 0.5303296703296704
            name: Cosine Accuracy@3
          - type: cosine_accuracy@5
            value: 0.6074725274725276
            name: Cosine Accuracy@5
          - type: cosine_accuracy@10
            value: 0.7214756671899529
            name: Cosine Accuracy@10
          - type: cosine_precision@1
            value: 0.3469701726844584
            name: Cosine Precision@1
          - type: cosine_precision@3
            value: 0.23401360544217684
            name: Cosine Precision@3
          - type: cosine_precision@5
            value: 0.18648037676609108
            name: Cosine Precision@5
          - type: cosine_precision@10
            value: 0.13500470957613817
            name: Cosine Precision@10
          - type: cosine_recall@1
            value: 0.18488460123328906
            name: Cosine Recall@1
          - type: cosine_recall@3
            value: 0.32576774706513195
            name: Cosine Recall@3
          - type: cosine_recall@5
            value: 0.3876666429564824
            name: Cosine Recall@5
          - type: cosine_recall@10
            value: 0.48913834108187415
            name: Cosine Recall@10
          - type: cosine_ndcg@10
            value: 0.4103820562345277
            name: Cosine Ndcg@10
          - type: cosine_mrr@10
            value: 0.4603737509655877
            name: Cosine Mrr@10
          - type: cosine_map@100
            value: 0.3391583508465161
            name: Cosine Map@100

MPNet base trained on Natural Questions pairs

This is a sentence-transformers model finetuned from microsoft/mpnet-base on the gooaq-hard-negatives dataset. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

Model Type: Sentence Transformer
Base model: microsoft/mpnet-base
Maximum Sequence Length: 512 tokens
Output Dimensionality: 768 dimensions
Similarity Function: Cosine Similarity
Training Dataset:
- gooaq-hard-negatives
Language: en
License: apache-2.0

Model Sources

Documentation: Sentence Transformers Documentation
Repository: Sentence Transformers on GitHub
Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 512, 'do_lower_case': False}) with Transformer model: MPNetModel 
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("tomaarsen/mpnet-base-nq-cgist-triplet-gt")
# Run inference
sentences = [
    'what energy is released when coal is burned?',
    'When coal is burned, it reacts with the oxygen in the air. This chemical reaction converts the stored solar energy into thermal energy, which is released as heat. But it also produces carbon dioxide and methane.',
    'When coal is burned it releases a number of airborne toxins and pollutants. They include mercury, lead, sulfur dioxide, nitrogen oxides, particulates, and various other heavy metals.',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 768]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]

Evaluation

Metrics

Information Retrieval

Datasets: NanoClimateFEVER, NanoDBPedia, NanoFEVER, NanoFiQA2018, NanoHotpotQA, NanoMSMARCO, NanoNFCorpus, NanoNQ, NanoQuoraRetrieval, NanoSCIDOCS, NanoArguAna, NanoSciFact and NanoTouche2020
Evaluated with InformationRetrievalEvaluator

Metric	NanoClimateFEVER	NanoDBPedia	NanoFEVER	NanoFiQA2018	NanoHotpotQA	NanoMSMARCO	NanoNFCorpus	NanoNQ	NanoQuoraRetrieval	NanoSCIDOCS	NanoArguAna	NanoSciFact	NanoTouche2020
cosine_accuracy@1	0.22	0.46	0.38	0.28	0.34	0.12	0.32	0.16	0.8	0.36	0.18	0.36	0.5306
cosine_accuracy@3	0.44	0.64	0.54	0.5	0.58	0.3	0.4	0.38	0.9	0.48	0.56	0.46	0.7143
cosine_accuracy@5	0.5	0.76	0.58	0.52	0.64	0.56	0.44	0.46	0.92	0.54	0.64	0.48	0.8571
cosine_accuracy@10	0.72	0.84	0.7	0.58	0.74	0.66	0.5	0.58	0.96	0.68	0.84	0.62	0.9592
cosine_precision@1	0.22	0.46	0.38	0.28	0.34	0.12	0.32	0.16	0.8	0.36	0.18	0.36	0.5306
cosine_precision@3	0.1667	0.4	0.18	0.22	0.2133	0.1	0.2133	0.1267	0.3667	0.26	0.1867	0.1667	0.4422
cosine_precision@5	0.116	0.376	0.12	0.16	0.148	0.112	0.196	0.092	0.244	0.216	0.128	0.104	0.4122
cosine_precision@10	0.092	0.34	0.072	0.096	0.094	0.066	0.144	0.06	0.134	0.152	0.084	0.068	0.3531
cosine_recall@1	0.0933	0.0307	0.37	0.1372	0.17	0.12	0.0122	0.15	0.7007	0.0757	0.18	0.325	0.0388
cosine_recall@3	0.195	0.0782	0.52	0.3254	0.32	0.3	0.0204	0.36	0.8553	0.1617	0.56	0.44	0.099
cosine_recall@5	0.2267	0.1219	0.57	0.3654	0.37	0.56	0.0287	0.43	0.8993	0.2227	0.64	0.46	0.145
cosine_recall@10	0.3673	0.2165	0.68	0.4294	0.47	0.66	0.04	0.54	0.9567	0.3147	0.84	0.605	0.2392
cosine_ndcg@10	0.274	0.3892	0.5227	0.3441	0.3815	0.366	0.1735	0.3427	0.876	0.299	0.5001	0.4619	0.4044
cosine_mrr@10	0.3633	0.5769	0.479	0.3882	0.4696	0.2741	0.3729	0.2898	0.8545	0.442	0.3925	0.4301	0.6521
cosine_map@100	0.2038	0.2783	0.4781	0.2878	0.3064	0.287	0.0491	0.293	0.8464	0.2364	0.4001	0.4274	0.3152

Nano BEIR

Dataset: NanoBEIR_mean
Evaluated with NanoBEIREvaluator

Metric	Value
cosine_accuracy@1	0.347
cosine_accuracy@3	0.5303
cosine_accuracy@5	0.6075
cosine_accuracy@10	0.7215
cosine_precision@1	0.347
cosine_precision@3	0.234
cosine_precision@5	0.1865
cosine_precision@10	0.135
cosine_recall@1	0.1849
cosine_recall@3	0.3258
cosine_recall@5	0.3877
cosine_recall@10	0.4891
cosine_ndcg@10	0.4104
cosine_mrr@10	0.4604
cosine_map@100	0.3392

Training Details

Training Dataset

gooaq-hard-negatives

Dataset: gooaq-hard-negatives at 87594a1
Size: 50,000 training samples
Columns: question, answer, and negative

Approximate statistics based on the first 1000 samples:

	question	answer	negative
type	string	string	string
details	min: 8 tokens mean: 11.53 tokens max: 28 tokens	min: 14 tokens mean: 59.79 tokens max: 150 tokens	min: 15 tokens mean: 58.76 tokens max: 143 tokens

Samples:

question	answer	negative
`what is the difference between calories from fat and total fat?`	`Fat has more than twice as many calories per gram as carbohydrates and proteins. A gram of fat has about 9 calories, while a gram of carbohydrate or protein has about 4 calories. In other words, you could eat twice as much carbohydrates or proteins as fat for the same amount of calories.`	`Fat has more than twice as many calories per gram as carbohydrates and proteins. A gram of fat has about 9 calories, while a gram of carbohydrate or protein has about 4 calories. In other words, you could eat twice as much carbohydrates or proteins as fat for the same amount of calories.`
`what is the difference between return transcript and account transcript?`	`A tax return transcript usually meets the needs of lending institutions offering mortgages and student loans. ... Tax Account Transcript - shows basic data such as return type, marital status, adjusted gross income, taxable income and all payment types. It also shows changes made after you filed your original return.`	`Trial balance is not a financial statement whereas a balance sheet is a financial statement. Trial balance is solely used for internal purposes whereas a balance sheet is used for purposes other than internal i.e. external. In a trial balance, each and every account is divided into debit (dr.) and credit (cr.)`
`how long does my dog need to fast before sedation?`	`Now, guidelines are aimed towards 6-8 hours before surgery. This pre-op fasting time is much more beneficial for your pets because you have enough food in there to neutralize the stomach acid, preventing it from coming up the esophagus that causes regurgitation under anesthetic.`	`Try not to let your pooch rapidly wolf down his/her food! Do not let the dog play or exercise (e.g. go for a walk) for at least two hours after having a meal. Ensure continuous fresh water is available to avoid your pet gulping down a large amount after eating.`

Loss: CachedGISTEmbedLoss with these parameters:

{'guide': SentenceTransformer(
  (0): Transformer({'max_seq_length': 256, 'do_lower_case': False}) with Transformer model: BertModel 
  (1): Pooling({'word_embedding_dimension': 384, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Normalize()
), 'temperature': 0.01}

Evaluation Dataset

gooaq-hard-negatives

Dataset: gooaq-hard-negatives at 87594a1
Size: 10,048,700 evaluation samples
Columns: question, answer, and negative

Approximate statistics based on the first 1000 samples:

	question	answer	negative
type	string	string	string
details	min: 8 tokens mean: 11.61 tokens max: 21 tokens	min: 16 tokens mean: 58.16 tokens max: 131 tokens	min: 14 tokens mean: 57.98 tokens max: 157 tokens

Samples:

question	answer	negative
`how is height width and length written?`	`The Graphics' industry standard is width by height (width x height). Meaning that when you write your measurements, you write them from your point of view, beginning with the width.`	`The Graphics' industry standard is width by height (width x height). Meaning that when you write your measurements, you write them from your point of view, beginning with the width. That's important.`
`what is the difference between pork shoulder and loin?`	`All the recipes I've found for pulled pork recommends a shoulder/butt. Shoulders take longer to cook than a loin, because they're tougher. Loins are lean, while shoulders have marbled fat inside.`	`They are extracted from the loin, which runs from the hip to the shoulder, and it has a small strip of meat called the tenderloin. Unlike other pork, this pork chop is cut from four major sections, which are the shoulder, also known as the blade chops, ribs chops, loin chops, and the last, which is the sirloin chops.`
`is the yin yang symbol religious?`	`The ubiquitous yin-yang symbol holds its roots in Taoism/Daoism, a Chinese religion and philosophy. The yin, the dark swirl, is associated with shadows, femininity, and the trough of a wave; the yang, the light swirl, represents brightness, passion and growth.`	`Yin energy is in the calm colors around you, in the soft music, in the soothing sound of a water fountain, or the relaxing images of water. Yang (active energy) is the feng shui energy expressed in strong, vibrant sounds and colors, bright lights, upward moving energy, tall plants, etc.`

Loss: CachedGISTEmbedLoss with these parameters:

{'guide': SentenceTransformer(
  (0): Transformer({'max_seq_length': 256, 'do_lower_case': False}) with Transformer model: BertModel 
  (1): Pooling({'word_embedding_dimension': 384, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Normalize()
), 'temperature': 0.01}

Training Hyperparameters

Non-Default Hyperparameters

eval_strategy: steps
per_device_train_batch_size: 2048
per_device_eval_batch_size: 2048
learning_rate: 2e-05
num_train_epochs: 1
warmup_ratio: 0.1
seed: 12
bf16: True

All Hyperparameters

Click to expand

overwrite_output_dir: False
do_predict: False
eval_strategy: steps
prediction_loss_only: True
per_device_train_batch_size: 2048
per_device_eval_batch_size: 2048
per_gpu_train_batch_size: None
per_gpu_eval_batch_size: None
gradient_accumulation_steps: 1
eval_accumulation_steps: None
torch_empty_cache_steps: None
learning_rate: 2e-05
weight_decay: 0.0
adam_beta1: 0.9
adam_beta2: 0.999
adam_epsilon: 1e-08
max_grad_norm: 1.0
num_train_epochs: 1
max_steps: -1
lr_scheduler_type: linear
lr_scheduler_kwargs: {}
warmup_ratio: 0.1
warmup_steps: 0
log_level: passive
log_level_replica: warning
log_on_each_node: True
logging_nan_inf_filter: True
save_safetensors: True
save_on_each_node: False
save_only_model: False
restore_callback_states_from_checkpoint: False
no_cuda: False
use_cpu: False
use_mps_device: False
seed: 12
data_seed: None
jit_mode_eval: False
use_ipex: False
bf16: True
fp16: False
fp16_opt_level: O1
half_precision_backend: auto
bf16_full_eval: False
fp16_full_eval: False
tf32: None
local_rank: 0
ddp_backend: None
tpu_num_cores: None
tpu_metrics_debug: False
debug: []
dataloader_drop_last: False
dataloader_num_workers: 0
dataloader_prefetch_factor: None
past_index: -1
disable_tqdm: False
remove_unused_columns: True
label_names: None
load_best_model_at_end: False
ignore_data_skip: False
fsdp: []
fsdp_min_num_params: 0
fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
fsdp_transformer_layer_cls_to_wrap: None
accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
deepspeed: None
label_smoothing_factor: 0.0
optim: adamw_torch
optim_args: None
adafactor: False
group_by_length: False
length_column_name: length
ddp_find_unused_parameters: None
ddp_bucket_cap_mb: None
ddp_broadcast_buffers: False
dataloader_pin_memory: True
dataloader_persistent_workers: False
skip_memory_metrics: True
use_legacy_prediction_loop: False
push_to_hub: False
resume_from_checkpoint: None
hub_model_id: None
hub_strategy: every_save
hub_private_repo: False
hub_always_push: False
gradient_checkpointing: False
gradient_checkpointing_kwargs: None
include_inputs_for_metrics: False
include_for_metrics: []
eval_do_concat_batches: True
fp16_backend: auto
push_to_hub_model_id: None
push_to_hub_organization: None
mp_parameters:
auto_find_batch_size: False
full_determinism: False
torchdynamo: None
ray_scope: last
ddp_timeout: 1800
torch_compile: False
torch_compile_backend: None
torch_compile_mode: None
dispatch_batches: None
split_batches: None
include_tokens_per_second: False
include_num_input_tokens_seen: False
neftune_noise_alpha: None
optim_target_modules: None
batch_eval_metrics: False
eval_on_start: False
use_liger_kernel: False
eval_use_gather_object: False
average_tokens_across_devices: False
prompts: None
batch_sampler: batch_sampler
multi_dataset_batch_sampler: proportional

Training Logs

Epoch	Step	Training Loss	Validation Loss	NanoClimateFEVER_cosine_ndcg@10	NanoDBPedia_cosine_ndcg@10	NanoFEVER_cosine_ndcg@10	NanoFiQA2018_cosine_ndcg@10	NanoHotpotQA_cosine_ndcg@10	NanoMSMARCO_cosine_ndcg@10	NanoNFCorpus_cosine_ndcg@10	NanoNQ_cosine_ndcg@10	NanoQuoraRetrieval_cosine_ndcg@10	NanoSCIDOCS_cosine_ndcg@10	NanoArguAna_cosine_ndcg@10	NanoSciFact_cosine_ndcg@10	NanoTouche2020_cosine_ndcg@10	NanoBEIR_mean_cosine_ndcg@10
0.04	1	11.5142	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.2	5	9.438	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.4	10	5.5516	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.6	15	3.7045	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.8	20	2.7618	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
1.0	25	2.1657	1.3177	0.2740	0.3892	0.5227	0.3441	0.3815	0.3660	0.1735	0.3427	0.8760	0.2990	0.5001	0.4619	0.4044	0.4104

Environmental Impact

Carbon emissions were measured using CodeCarbon.

Energy Consumed: 0.104 kWh
Carbon Emitted: 0.040 kg of CO2
Hours Used: 0.297 hours

Training Hardware

On Cloud: No
GPU Model: 1 x NVIDIA GeForce RTX 3090
CPU Model: 13th Gen Intel(R) Core(TM) i7-13700K
RAM Size: 31.78 GB

Framework Versions

Python: 3.11.6
Sentence Transformers: 3.4.0.dev0
Transformers: 4.46.2
PyTorch: 2.5.0+cu121
Accelerate: 0.35.0.dev0
Datasets: 2.20.0
Tokenizers: 0.20.3

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}