|
--- |
|
base_model: Qwen/Qwen2.5-0.5B-Instruct |
|
library_name: sentence-transformers |
|
metrics: |
|
- cosine_accuracy@1 |
|
- cosine_accuracy@3 |
|
- cosine_accuracy@5 |
|
- cosine_accuracy@10 |
|
- cosine_precision@1 |
|
- cosine_precision@3 |
|
- cosine_precision@5 |
|
- cosine_precision@10 |
|
- cosine_recall@1 |
|
- cosine_recall@3 |
|
- cosine_recall@5 |
|
- cosine_recall@10 |
|
- cosine_ndcg@10 |
|
- cosine_mrr@10 |
|
- cosine_map@100 |
|
pipeline_tag: sentence-similarity |
|
tags: |
|
- sentence-transformers |
|
- sentence-similarity |
|
- feature-extraction |
|
- generated_from_trainer |
|
- dataset_size:1077240 |
|
- loss:MultipleNegativesRankingLoss |
|
widget: |
|
- source_sentence: When was the frigate the USS Alliance built? |
|
sentences: |
|
- 'Middle Stone Age |
|
|
|
The Middle Stone Age (or MSA) was a period of African prehistory between the Early |
|
Stone Age and the Later Stone Age. It is generally considered to have begun around |
|
280,000 years ago and ended around 50–25,000 years ago.[1] The beginnings of particular |
|
MSA stone tools have their origins as far back as 550–500,000 years ago and as |
|
such some researchers consider this to be the beginnings of the MSA.[2] The MSA |
|
is often mistakenly understood to be synonymous with the Middle Paleolithic of |
|
Europe, especially due to their roughly contemporaneous time span, however, the |
|
Middle Paleolithic of Europe represents an entirely different hominin population, |
|
Homo neanderthalensis, than the MSA of Africa, which did not have Neanderthal |
|
populations. Additionally, current archaeological research in Africa has yielded |
|
much evidence to suggest that modern human behavior and cognition was beginning |
|
to develop much earlier in Africa during the MSA than it was in Europe during |
|
the Middle Paleolithic.[3] The MSA is associated with both anatomically modern |
|
humans (Homo sapiens) as well as archaic Homo sapiens, sometimes referred to as |
|
Homo helmei. Early physical evidence comes from the Gademotta Formation in Ethiopia, |
|
the Kapthurin Formation in Kenya and Kathu Pan in South Africa.[2]' |
|
- 'USS Alliance (1778) |
|
|
|
Originally named Hancock, she was laid down in 1777 on the Merrimack River at |
|
Amesbury, Massachusetts, by the partners and cousins, William and James K. Hackett, |
|
launched on 28 April 1778, and renamed Alliance on 29 May 1778 by resolution of |
|
the Continental Congress. Her first commanding officer was Capt. Pierre Landais, |
|
a former officer of the French Navy who had come to the New World hoping to become |
|
a naval counterpart of Lafayette. The frigate''s first captain was widely accepted |
|
as such in America. Massachusetts made him an honorary citizen and the Continental |
|
Congress gave him command of Alliance, thought to be the finest warship built |
|
to that date on the western side of the Atlantic.' |
|
- 'USS Frigate Bird (AMS-191) |
|
|
|
The second ship in the Navy to be named "Frigate Bird", she was laid down 20 July |
|
1953, as AMS-191; launched 24 October 1953, by Quincy Adams Yacht Yard, Inc., |
|
Quincy, Massachusetts; sponsored by Mrs. Matthew Gushing; and commissioned 13 |
|
January 1955, Lieutenant (jg) George B. Shick, Jr., in command. She was reclassified |
|
MSC-191 on 7 February 1955.' |
|
- source_sentence: How many bananas do Americans consume each year? |
|
sentences: |
|
- 'Dust Bowl |
|
|
|
The Dust Bowl was a period of severe dust storms that greatly damaged the ecology |
|
and agriculture of the American and Canadian prairies during the 1930s; severe |
|
drought and a failure to apply dryland farming methods to prevent the aeolian |
|
processes (wind erosion) caused the phenomenon.[1][2] The drought came in three |
|
waves, 1934, 1936, and 1939–1940, but some regions of the high plains experienced |
|
drought conditions for as many as eight years.[3] With insufficient understanding |
|
of the ecology of the plains, farmers had conducted extensive deep plowing of |
|
the virgin topsoil of the Great Plains during the previous decade; this had displaced |
|
the native, deep-rooted grasses that normally trapped soil and moisture even during |
|
periods of drought and high winds. The rapid mechanization of farm equipment, |
|
especially small gasoline tractors, and widespread use of the combine harvester |
|
contributed to farmers'' decisions to convert arid grassland (much of which received |
|
no more than 10inches (~250mm) of precipitation per year) to cultivated cropland.[4]' |
|
- 'Banana production in the United States |
|
|
|
Commercial banana production in the United States is relatively limited in scale |
|
and economic impact. While Americans eat 26 pounds (12kg) of bananas per person |
|
per year, the vast majority of the fruit is imported from other countries, chiefly |
|
Central and South America, where the US has previously occupied areas containing |
|
banana plantations, and controlled the importation of bananas via various fruit |
|
companies, such as Dole and Chiquita. [1]' |
|
- 'Spinach in the United States |
|
|
|
Per capita spinach consumption is greatest in the Northeast and Western US. About |
|
80% of fresh-market spinach is purchased at retail and consumed at home, while |
|
91% of processed spinach is consumed at home. Per capita spinach use is strongest |
|
among Asians, highest among women 40 and older, and weakest among teenage girls.' |
|
- source_sentence: Which rapper has the most Grammy wins? |
|
sentences: |
|
- 'Rhineland |
|
|
|
The Rhineland (German: Rheinland, French: Rhénanie, Latinised name: Rhenania) |
|
is the name used for a loosely defined area of Western Germany along the Rhine, |
|
chiefly its middle section.' |
|
- 'Grammy Award for Best Rap Album |
|
|
|
6 wins Eminem 4 wins Kanye West 2 wins Outkast Kendrick Lamar' |
|
- 'Latin Grammy Award records |
|
|
|
René Pérez Joglar "Residente" and Eduardo Cabra "Visitante" with 24 awards, have |
|
won more than any other male artist.' |
|
- source_sentence: Is Iodine radioactive? |
|
sentences: |
|
- 'Double-decker bus |
|
|
|
With the exception of coaches, double-decker buses are uncommon in the United |
|
States. Many private operators, such as Megabus, run by Coach USA, employs double-decker |
|
buses on its busier intercity routes.' |
|
- 'Iodine-123 |
|
|
|
I is the most suitable isotope of iodine for the diagnostic study of thyroid diseases. |
|
The half-life of approximately 13.13 hours is ideal for the 24-hour iodine uptake |
|
test and I has other advantages for diagnostic imaging thyroid tissue and thyroid |
|
cancer metastasis. The energy of the photon, 159 keV, is ideal for the NaI (sodium |
|
iodide) crystal detector of current gamma cameras and also for the pinhole collimators. |
|
It has much greater photon flux than I. It gives approximately 20 times the counting |
|
rate of I for the same administered dose. The radiation burden to the thyroid |
|
is far less (1%) than that of I. Moreover, scanning a thyroid remnant or metastasis |
|
with I does not cause "stunning" of the tissue (with loss of uptake), because |
|
of the low radiation burden of this isotope. (For the same reasons, I is never |
|
used for thyroid cancer or Graves disease "treatment", and this role is reserved |
|
for I.)' |
|
- 'Iodine-131 |
|
|
|
Iodine-131 (131I) is an important radioisotope of iodine discovered by Glenn Seaborg |
|
and John Livingood in 1938 at the University of California, Berkeley.[1] It has |
|
a radioactive decay half-life of about eight days. It is associated with nuclear |
|
energy, medical diagnostic and treatment procedures, and natural gas production. |
|
It also plays a major role as a radioactive isotope present in nuclear fission |
|
products, and was a significant contributor to the health hazards from open-air |
|
atomic bomb testing in the 1950s, and from the Chernobyl disaster, as well as |
|
being a large fraction of the contamination hazard in the first weeks in the Fukushima |
|
nuclear crisis. This is because I-131 is a major fission product of uranium and |
|
plutonium, comprising nearly 3% of the total products of fission (by weight). |
|
See fission product yield for a comparison with other radioactive fission products. |
|
I-131 is also a major fission product of uranium-233, produced from thorium.' |
|
- source_sentence: Is a birth certificate a form of ID? |
|
sentences: |
|
- 'Identity documents in the United States |
|
|
|
The birth certificate is the initial identification document issued to parents |
|
shortly after the birth of their child. The birth certificate is typically issued |
|
by local governments, usually the city or county where a child is born. It is |
|
an important record, often called a "feeder document," because it establishes |
|
U.S. citizenship through birthright citizenship, which is then used to obtain, |
|
or is the basis for, all other identity documents.[2] By itself, the birth certificate |
|
is usually only considered proof of citizenship but not proof of identity, since |
|
it is issued without a photograph at birth, containing no identifying features. |
|
A birth certificate is normally produced along with proof of identity, such as |
|
a driver''s license or the testimony of a third party (such as a parent), to establish |
|
identity or entitlement to a service.' |
|
- 'Identity document |
|
|
|
In Canada, different forms of identification documentation are used, but there |
|
is no de jure national identity card. The Canadian passport is issued by the federal |
|
(national) government, and the provinces and territories issue various documents |
|
which can be used for identification purposes. The most commonly used forms of |
|
identification within Canada are the health card and driver''s licence issued |
|
by provincial and territorial governments. The widespread usage of these two documents |
|
for identification purposes has made them de facto identity cards.' |
|
- 'International Ladies'' Garment Workers'' Union |
|
|
|
The ILGWU was founded on June 3, 1900[2] in New York City by seven local unions, |
|
with a few thousand members between them. The union grew rapidly in the next few |
|
years but began to stagnate as the conservative leadership favored the interests |
|
of skilled workers, such as cutters. This did not sit well with the majority of |
|
immigrant workers, particularly Jewish workers with a background in Bundist activities |
|
in Tsarist Russia, or with Polish and Italian workers, many of whom had strong |
|
socialist and anarchist leanings.' |
|
model-index: |
|
- name: SentenceTransformer based on Qwen/Qwen2.5-0.5B-Instruct |
|
results: |
|
- task: |
|
type: information-retrieval |
|
name: Information Retrieval |
|
dataset: |
|
name: NanoClimateFEVER |
|
type: NanoClimateFEVER |
|
metrics: |
|
- type: cosine_accuracy@1 |
|
value: 0.18 |
|
name: Cosine Accuracy@1 |
|
- type: cosine_accuracy@3 |
|
value: 0.28 |
|
name: Cosine Accuracy@3 |
|
- type: cosine_accuracy@5 |
|
value: 0.32 |
|
name: Cosine Accuracy@5 |
|
- type: cosine_accuracy@10 |
|
value: 0.4 |
|
name: Cosine Accuracy@10 |
|
- type: cosine_precision@1 |
|
value: 0.18 |
|
name: Cosine Precision@1 |
|
- type: cosine_precision@3 |
|
value: 0.1 |
|
name: Cosine Precision@3 |
|
- type: cosine_precision@5 |
|
value: 0.07200000000000001 |
|
name: Cosine Precision@5 |
|
- type: cosine_precision@10 |
|
value: 0.052000000000000005 |
|
name: Cosine Precision@10 |
|
- type: cosine_recall@1 |
|
value: 0.07 |
|
name: Cosine Recall@1 |
|
- type: cosine_recall@3 |
|
value: 0.12833333333333333 |
|
name: Cosine Recall@3 |
|
- type: cosine_recall@5 |
|
value: 0.145 |
|
name: Cosine Recall@5 |
|
- type: cosine_recall@10 |
|
value: 0.19333333333333333 |
|
name: Cosine Recall@10 |
|
- type: cosine_ndcg@10 |
|
value: 0.16942887258019743 |
|
name: Cosine Ndcg@10 |
|
- type: cosine_mrr@10 |
|
value: 0.24757142857142853 |
|
name: Cosine Mrr@10 |
|
- type: cosine_map@100 |
|
value: 0.1355707100998398 |
|
name: Cosine Map@100 |
|
- task: |
|
type: information-retrieval |
|
name: Information Retrieval |
|
dataset: |
|
name: NanoDBPedia |
|
type: NanoDBPedia |
|
metrics: |
|
- type: cosine_accuracy@1 |
|
value: 0.6 |
|
name: Cosine Accuracy@1 |
|
- type: cosine_accuracy@3 |
|
value: 0.72 |
|
name: Cosine Accuracy@3 |
|
- type: cosine_accuracy@5 |
|
value: 0.82 |
|
name: Cosine Accuracy@5 |
|
- type: cosine_accuracy@10 |
|
value: 0.88 |
|
name: Cosine Accuracy@10 |
|
- type: cosine_precision@1 |
|
value: 0.6 |
|
name: Cosine Precision@1 |
|
- type: cosine_precision@3 |
|
value: 0.4333333333333333 |
|
name: Cosine Precision@3 |
|
- type: cosine_precision@5 |
|
value: 0.4159999999999999 |
|
name: Cosine Precision@5 |
|
- type: cosine_precision@10 |
|
value: 0.34 |
|
name: Cosine Precision@10 |
|
- type: cosine_recall@1 |
|
value: 0.05876239296513622 |
|
name: Cosine Recall@1 |
|
- type: cosine_recall@3 |
|
value: 0.12033972099857915 |
|
name: Cosine Recall@3 |
|
- type: cosine_recall@5 |
|
value: 0.1722048755781874 |
|
name: Cosine Recall@5 |
|
- type: cosine_recall@10 |
|
value: 0.23956661282775613 |
|
name: Cosine Recall@10 |
|
- type: cosine_ndcg@10 |
|
value: 0.43285369033010385 |
|
name: Cosine Ndcg@10 |
|
- type: cosine_mrr@10 |
|
value: 0.6865000000000001 |
|
name: Cosine Mrr@10 |
|
- type: cosine_map@100 |
|
value: 0.26928358503732847 |
|
name: Cosine Map@100 |
|
- task: |
|
type: information-retrieval |
|
name: Information Retrieval |
|
dataset: |
|
name: NanoFEVER |
|
type: NanoFEVER |
|
metrics: |
|
- type: cosine_accuracy@1 |
|
value: 0.28 |
|
name: Cosine Accuracy@1 |
|
- type: cosine_accuracy@3 |
|
value: 0.56 |
|
name: Cosine Accuracy@3 |
|
- type: cosine_accuracy@5 |
|
value: 0.66 |
|
name: Cosine Accuracy@5 |
|
- type: cosine_accuracy@10 |
|
value: 0.82 |
|
name: Cosine Accuracy@10 |
|
- type: cosine_precision@1 |
|
value: 0.28 |
|
name: Cosine Precision@1 |
|
- type: cosine_precision@3 |
|
value: 0.18666666666666668 |
|
name: Cosine Precision@3 |
|
- type: cosine_precision@5 |
|
value: 0.136 |
|
name: Cosine Precision@5 |
|
- type: cosine_precision@10 |
|
value: 0.08599999999999998 |
|
name: Cosine Precision@10 |
|
- type: cosine_recall@1 |
|
value: 0.26 |
|
name: Cosine Recall@1 |
|
- type: cosine_recall@3 |
|
value: 0.52 |
|
name: Cosine Recall@3 |
|
- type: cosine_recall@5 |
|
value: 0.63 |
|
name: Cosine Recall@5 |
|
- type: cosine_recall@10 |
|
value: 0.79 |
|
name: Cosine Recall@10 |
|
- type: cosine_ndcg@10 |
|
value: 0.5142272491063538 |
|
name: Cosine Ndcg@10 |
|
- type: cosine_mrr@10 |
|
value: 0.4404126984126984 |
|
name: Cosine Mrr@10 |
|
- type: cosine_map@100 |
|
value: 0.4252817429020436 |
|
name: Cosine Map@100 |
|
- task: |
|
type: information-retrieval |
|
name: Information Retrieval |
|
dataset: |
|
name: NanoFiQA2018 |
|
type: NanoFiQA2018 |
|
metrics: |
|
- type: cosine_accuracy@1 |
|
value: 0.18 |
|
name: Cosine Accuracy@1 |
|
- type: cosine_accuracy@3 |
|
value: 0.28 |
|
name: Cosine Accuracy@3 |
|
- type: cosine_accuracy@5 |
|
value: 0.4 |
|
name: Cosine Accuracy@5 |
|
- type: cosine_accuracy@10 |
|
value: 0.5 |
|
name: Cosine Accuracy@10 |
|
- type: cosine_precision@1 |
|
value: 0.18 |
|
name: Cosine Precision@1 |
|
- type: cosine_precision@3 |
|
value: 0.11333333333333333 |
|
name: Cosine Precision@3 |
|
- type: cosine_precision@5 |
|
value: 0.096 |
|
name: Cosine Precision@5 |
|
- type: cosine_precision@10 |
|
value: 0.062 |
|
name: Cosine Precision@10 |
|
- type: cosine_recall@1 |
|
value: 0.0968888888888889 |
|
name: Cosine Recall@1 |
|
- type: cosine_recall@3 |
|
value: 0.1971111111111111 |
|
name: Cosine Recall@3 |
|
- type: cosine_recall@5 |
|
value: 0.274968253968254 |
|
name: Cosine Recall@5 |
|
- type: cosine_recall@10 |
|
value: 0.3351904761904762 |
|
name: Cosine Recall@10 |
|
- type: cosine_ndcg@10 |
|
value: 0.23653831229880148 |
|
name: Cosine Ndcg@10 |
|
- type: cosine_mrr@10 |
|
value: 0.2642460317460318 |
|
name: Cosine Mrr@10 |
|
- type: cosine_map@100 |
|
value: 0.19230802555548754 |
|
name: Cosine Map@100 |
|
- task: |
|
type: information-retrieval |
|
name: Information Retrieval |
|
dataset: |
|
name: NanoHotpotQA |
|
type: NanoHotpotQA |
|
metrics: |
|
- type: cosine_accuracy@1 |
|
value: 0.46 |
|
name: Cosine Accuracy@1 |
|
- type: cosine_accuracy@3 |
|
value: 0.56 |
|
name: Cosine Accuracy@3 |
|
- type: cosine_accuracy@5 |
|
value: 0.6 |
|
name: Cosine Accuracy@5 |
|
- type: cosine_accuracy@10 |
|
value: 0.68 |
|
name: Cosine Accuracy@10 |
|
- type: cosine_precision@1 |
|
value: 0.46 |
|
name: Cosine Precision@1 |
|
- type: cosine_precision@3 |
|
value: 0.23333333333333336 |
|
name: Cosine Precision@3 |
|
- type: cosine_precision@5 |
|
value: 0.14800000000000002 |
|
name: Cosine Precision@5 |
|
- type: cosine_precision@10 |
|
value: 0.08999999999999998 |
|
name: Cosine Precision@10 |
|
- type: cosine_recall@1 |
|
value: 0.23 |
|
name: Cosine Recall@1 |
|
- type: cosine_recall@3 |
|
value: 0.35 |
|
name: Cosine Recall@3 |
|
- type: cosine_recall@5 |
|
value: 0.37 |
|
name: Cosine Recall@5 |
|
- type: cosine_recall@10 |
|
value: 0.45 |
|
name: Cosine Recall@10 |
|
- type: cosine_ndcg@10 |
|
value: 0.4053306177663136 |
|
name: Cosine Ndcg@10 |
|
- type: cosine_mrr@10 |
|
value: 0.512126984126984 |
|
name: Cosine Mrr@10 |
|
- type: cosine_map@100 |
|
value: 0.34910258985509013 |
|
name: Cosine Map@100 |
|
- task: |
|
type: information-retrieval |
|
name: Information Retrieval |
|
dataset: |
|
name: NanoMSMARCO |
|
type: NanoMSMARCO |
|
metrics: |
|
- type: cosine_accuracy@1 |
|
value: 0.22 |
|
name: Cosine Accuracy@1 |
|
- type: cosine_accuracy@3 |
|
value: 0.34 |
|
name: Cosine Accuracy@3 |
|
- type: cosine_accuracy@5 |
|
value: 0.34 |
|
name: Cosine Accuracy@5 |
|
- type: cosine_accuracy@10 |
|
value: 0.44 |
|
name: Cosine Accuracy@10 |
|
- type: cosine_precision@1 |
|
value: 0.22 |
|
name: Cosine Precision@1 |
|
- type: cosine_precision@3 |
|
value: 0.11333333333333333 |
|
name: Cosine Precision@3 |
|
- type: cosine_precision@5 |
|
value: 0.068 |
|
name: Cosine Precision@5 |
|
- type: cosine_precision@10 |
|
value: 0.044000000000000004 |
|
name: Cosine Precision@10 |
|
- type: cosine_recall@1 |
|
value: 0.22 |
|
name: Cosine Recall@1 |
|
- type: cosine_recall@3 |
|
value: 0.34 |
|
name: Cosine Recall@3 |
|
- type: cosine_recall@5 |
|
value: 0.34 |
|
name: Cosine Recall@5 |
|
- type: cosine_recall@10 |
|
value: 0.44 |
|
name: Cosine Recall@10 |
|
- type: cosine_ndcg@10 |
|
value: 0.3192297885891097 |
|
name: Cosine Ndcg@10 |
|
- type: cosine_mrr@10 |
|
value: 0.2824126984126984 |
|
name: Cosine Mrr@10 |
|
- type: cosine_map@100 |
|
value: 0.2997920687164519 |
|
name: Cosine Map@100 |
|
- task: |
|
type: information-retrieval |
|
name: Information Retrieval |
|
dataset: |
|
name: NanoNFCorpus |
|
type: NanoNFCorpus |
|
metrics: |
|
- type: cosine_accuracy@1 |
|
value: 0.32 |
|
name: Cosine Accuracy@1 |
|
- type: cosine_accuracy@3 |
|
value: 0.42 |
|
name: Cosine Accuracy@3 |
|
- type: cosine_accuracy@5 |
|
value: 0.46 |
|
name: Cosine Accuracy@5 |
|
- type: cosine_accuracy@10 |
|
value: 0.52 |
|
name: Cosine Accuracy@10 |
|
- type: cosine_precision@1 |
|
value: 0.32 |
|
name: Cosine Precision@1 |
|
- type: cosine_precision@3 |
|
value: 0.28 |
|
name: Cosine Precision@3 |
|
- type: cosine_precision@5 |
|
value: 0.24 |
|
name: Cosine Precision@5 |
|
- type: cosine_precision@10 |
|
value: 0.18 |
|
name: Cosine Precision@10 |
|
- type: cosine_recall@1 |
|
value: 0.012493999137489852 |
|
name: Cosine Recall@1 |
|
- type: cosine_recall@3 |
|
value: 0.03666681098754983 |
|
name: Cosine Recall@3 |
|
- type: cosine_recall@5 |
|
value: 0.05620504095265686 |
|
name: Cosine Recall@5 |
|
- type: cosine_recall@10 |
|
value: 0.06687360120479266 |
|
name: Cosine Recall@10 |
|
- type: cosine_ndcg@10 |
|
value: 0.22113733731375582 |
|
name: Cosine Ndcg@10 |
|
- type: cosine_mrr@10 |
|
value: 0.3847142857142857 |
|
name: Cosine Mrr@10 |
|
- type: cosine_map@100 |
|
value: 0.08151129221772184 |
|
name: Cosine Map@100 |
|
- task: |
|
type: information-retrieval |
|
name: Information Retrieval |
|
dataset: |
|
name: NanoNQ |
|
type: NanoNQ |
|
metrics: |
|
- type: cosine_accuracy@1 |
|
value: 0.28 |
|
name: Cosine Accuracy@1 |
|
- type: cosine_accuracy@3 |
|
value: 0.4 |
|
name: Cosine Accuracy@3 |
|
- type: cosine_accuracy@5 |
|
value: 0.5 |
|
name: Cosine Accuracy@5 |
|
- type: cosine_accuracy@10 |
|
value: 0.58 |
|
name: Cosine Accuracy@10 |
|
- type: cosine_precision@1 |
|
value: 0.28 |
|
name: Cosine Precision@1 |
|
- type: cosine_precision@3 |
|
value: 0.13333333333333333 |
|
name: Cosine Precision@3 |
|
- type: cosine_precision@5 |
|
value: 0.10000000000000002 |
|
name: Cosine Precision@5 |
|
- type: cosine_precision@10 |
|
value: 0.05800000000000001 |
|
name: Cosine Precision@10 |
|
- type: cosine_recall@1 |
|
value: 0.27 |
|
name: Cosine Recall@1 |
|
- type: cosine_recall@3 |
|
value: 0.37 |
|
name: Cosine Recall@3 |
|
- type: cosine_recall@5 |
|
value: 0.47 |
|
name: Cosine Recall@5 |
|
- type: cosine_recall@10 |
|
value: 0.53 |
|
name: Cosine Recall@10 |
|
- type: cosine_ndcg@10 |
|
value: 0.397781398000215 |
|
name: Cosine Ndcg@10 |
|
- type: cosine_mrr@10 |
|
value: 0.3704682539682539 |
|
name: Cosine Mrr@10 |
|
- type: cosine_map@100 |
|
value: 0.36349470659883915 |
|
name: Cosine Map@100 |
|
- task: |
|
type: information-retrieval |
|
name: Information Retrieval |
|
dataset: |
|
name: NanoQuoraRetrieval |
|
type: NanoQuoraRetrieval |
|
metrics: |
|
- type: cosine_accuracy@1 |
|
value: 0.84 |
|
name: Cosine Accuracy@1 |
|
- type: cosine_accuracy@3 |
|
value: 0.92 |
|
name: Cosine Accuracy@3 |
|
- type: cosine_accuracy@5 |
|
value: 0.92 |
|
name: Cosine Accuracy@5 |
|
- type: cosine_accuracy@10 |
|
value: 0.98 |
|
name: Cosine Accuracy@10 |
|
- type: cosine_precision@1 |
|
value: 0.84 |
|
name: Cosine Precision@1 |
|
- type: cosine_precision@3 |
|
value: 0.3666666666666666 |
|
name: Cosine Precision@3 |
|
- type: cosine_precision@5 |
|
value: 0.236 |
|
name: Cosine Precision@5 |
|
- type: cosine_precision@10 |
|
value: 0.128 |
|
name: Cosine Precision@10 |
|
- type: cosine_recall@1 |
|
value: 0.7406666666666666 |
|
name: Cosine Recall@1 |
|
- type: cosine_recall@3 |
|
value: 0.8686666666666667 |
|
name: Cosine Recall@3 |
|
- type: cosine_recall@5 |
|
value: 0.8859999999999999 |
|
name: Cosine Recall@5 |
|
- type: cosine_recall@10 |
|
value: 0.956 |
|
name: Cosine Recall@10 |
|
- type: cosine_ndcg@10 |
|
value: 0.8890148621063009 |
|
name: Cosine Ndcg@10 |
|
- type: cosine_mrr@10 |
|
value: 0.8828571428571428 |
|
name: Cosine Mrr@10 |
|
- type: cosine_map@100 |
|
value: 0.8629278499278499 |
|
name: Cosine Map@100 |
|
- task: |
|
type: information-retrieval |
|
name: Information Retrieval |
|
dataset: |
|
name: NanoSCIDOCS |
|
type: NanoSCIDOCS |
|
metrics: |
|
- type: cosine_accuracy@1 |
|
value: 0.26 |
|
name: Cosine Accuracy@1 |
|
- type: cosine_accuracy@3 |
|
value: 0.38 |
|
name: Cosine Accuracy@3 |
|
- type: cosine_accuracy@5 |
|
value: 0.48 |
|
name: Cosine Accuracy@5 |
|
- type: cosine_accuracy@10 |
|
value: 0.62 |
|
name: Cosine Accuracy@10 |
|
- type: cosine_precision@1 |
|
value: 0.26 |
|
name: Cosine Precision@1 |
|
- type: cosine_precision@3 |
|
value: 0.15333333333333332 |
|
name: Cosine Precision@3 |
|
- type: cosine_precision@5 |
|
value: 0.14800000000000002 |
|
name: Cosine Precision@5 |
|
- type: cosine_precision@10 |
|
value: 0.118 |
|
name: Cosine Precision@10 |
|
- type: cosine_recall@1 |
|
value: 0.053000000000000005 |
|
name: Cosine Recall@1 |
|
- type: cosine_recall@3 |
|
value: 0.09300000000000001 |
|
name: Cosine Recall@3 |
|
- type: cosine_recall@5 |
|
value: 0.15 |
|
name: Cosine Recall@5 |
|
- type: cosine_recall@10 |
|
value: 0.24066666666666667 |
|
name: Cosine Recall@10 |
|
- type: cosine_ndcg@10 |
|
value: 0.21606068741193518 |
|
name: Cosine Ndcg@10 |
|
- type: cosine_mrr@10 |
|
value: 0.35269047619047617 |
|
name: Cosine Mrr@10 |
|
- type: cosine_map@100 |
|
value: 0.15911734324501478 |
|
name: Cosine Map@100 |
|
- task: |
|
type: information-retrieval |
|
name: Information Retrieval |
|
dataset: |
|
name: NanoArguAna |
|
type: NanoArguAna |
|
metrics: |
|
- type: cosine_accuracy@1 |
|
value: 0.06 |
|
name: Cosine Accuracy@1 |
|
- type: cosine_accuracy@3 |
|
value: 0.42 |
|
name: Cosine Accuracy@3 |
|
- type: cosine_accuracy@5 |
|
value: 0.52 |
|
name: Cosine Accuracy@5 |
|
- type: cosine_accuracy@10 |
|
value: 0.68 |
|
name: Cosine Accuracy@10 |
|
- type: cosine_precision@1 |
|
value: 0.06 |
|
name: Cosine Precision@1 |
|
- type: cosine_precision@3 |
|
value: 0.13999999999999999 |
|
name: Cosine Precision@3 |
|
- type: cosine_precision@5 |
|
value: 0.10400000000000002 |
|
name: Cosine Precision@5 |
|
- type: cosine_precision@10 |
|
value: 0.068 |
|
name: Cosine Precision@10 |
|
- type: cosine_recall@1 |
|
value: 0.06 |
|
name: Cosine Recall@1 |
|
- type: cosine_recall@3 |
|
value: 0.42 |
|
name: Cosine Recall@3 |
|
- type: cosine_recall@5 |
|
value: 0.52 |
|
name: Cosine Recall@5 |
|
- type: cosine_recall@10 |
|
value: 0.68 |
|
name: Cosine Recall@10 |
|
- type: cosine_ndcg@10 |
|
value: 0.37083936956670244 |
|
name: Cosine Ndcg@10 |
|
- type: cosine_mrr@10 |
|
value: 0.2719126984126984 |
|
name: Cosine Mrr@10 |
|
- type: cosine_map@100 |
|
value: 0.28287447040897273 |
|
name: Cosine Map@100 |
|
- task: |
|
type: information-retrieval |
|
name: Information Retrieval |
|
dataset: |
|
name: NanoSciFact |
|
type: NanoSciFact |
|
metrics: |
|
- type: cosine_accuracy@1 |
|
value: 0.26 |
|
name: Cosine Accuracy@1 |
|
- type: cosine_accuracy@3 |
|
value: 0.38 |
|
name: Cosine Accuracy@3 |
|
- type: cosine_accuracy@5 |
|
value: 0.5 |
|
name: Cosine Accuracy@5 |
|
- type: cosine_accuracy@10 |
|
value: 0.6 |
|
name: Cosine Accuracy@10 |
|
- type: cosine_precision@1 |
|
value: 0.26 |
|
name: Cosine Precision@1 |
|
- type: cosine_precision@3 |
|
value: 0.1333333333333333 |
|
name: Cosine Precision@3 |
|
- type: cosine_precision@5 |
|
value: 0.10400000000000001 |
|
name: Cosine Precision@5 |
|
- type: cosine_precision@10 |
|
value: 0.066 |
|
name: Cosine Precision@10 |
|
- type: cosine_recall@1 |
|
value: 0.235 |
|
name: Cosine Recall@1 |
|
- type: cosine_recall@3 |
|
value: 0.355 |
|
name: Cosine Recall@3 |
|
- type: cosine_recall@5 |
|
value: 0.465 |
|
name: Cosine Recall@5 |
|
- type: cosine_recall@10 |
|
value: 0.58 |
|
name: Cosine Recall@10 |
|
- type: cosine_ndcg@10 |
|
value: 0.39672106927390305 |
|
name: Cosine Ndcg@10 |
|
- type: cosine_mrr@10 |
|
value: 0.35040476190476183 |
|
name: Cosine Mrr@10 |
|
- type: cosine_map@100 |
|
value: 0.3458399624509464 |
|
name: Cosine Map@100 |
|
- task: |
|
type: information-retrieval |
|
name: Information Retrieval |
|
dataset: |
|
name: NanoTouche2020 |
|
type: NanoTouche2020 |
|
metrics: |
|
- type: cosine_accuracy@1 |
|
value: 0.4489795918367347 |
|
name: Cosine Accuracy@1 |
|
- type: cosine_accuracy@3 |
|
value: 0.7142857142857143 |
|
name: Cosine Accuracy@3 |
|
- type: cosine_accuracy@5 |
|
value: 0.7959183673469388 |
|
name: Cosine Accuracy@5 |
|
- type: cosine_accuracy@10 |
|
value: 0.9795918367346939 |
|
name: Cosine Accuracy@10 |
|
- type: cosine_precision@1 |
|
value: 0.4489795918367347 |
|
name: Cosine Precision@1 |
|
- type: cosine_precision@3 |
|
value: 0.3945578231292517 |
|
name: Cosine Precision@3 |
|
- type: cosine_precision@5 |
|
value: 0.3673469387755102 |
|
name: Cosine Precision@5 |
|
- type: cosine_precision@10 |
|
value: 0.3346938775510204 |
|
name: Cosine Precision@10 |
|
- type: cosine_recall@1 |
|
value: 0.0346506610646935 |
|
name: Cosine Recall@1 |
|
- type: cosine_recall@3 |
|
value: 0.09241187721057745 |
|
name: Cosine Recall@3 |
|
- type: cosine_recall@5 |
|
value: 0.13780926782570618 |
|
name: Cosine Recall@5 |
|
- type: cosine_recall@10 |
|
value: 0.23921204048524203 |
|
name: Cosine Recall@10 |
|
- type: cosine_ndcg@10 |
|
value: 0.37383535385863614 |
|
name: Cosine Ndcg@10 |
|
- type: cosine_mrr@10 |
|
value: 0.6103984450923227 |
|
name: Cosine Mrr@10 |
|
- type: cosine_map@100 |
|
value: 0.3108247074769811 |
|
name: Cosine Map@100 |
|
- task: |
|
type: nano-beir |
|
name: Nano BEIR |
|
dataset: |
|
name: NanoBEIR mean |
|
type: NanoBEIR_mean |
|
metrics: |
|
- type: cosine_accuracy@1 |
|
value: 0.33761381475667196 |
|
name: Cosine Accuracy@1 |
|
- type: cosine_accuracy@3 |
|
value: 0.49032967032967034 |
|
name: Cosine Accuracy@3 |
|
- type: cosine_accuracy@5 |
|
value: 0.5627629513343799 |
|
name: Cosine Accuracy@5 |
|
- type: cosine_accuracy@10 |
|
value: 0.6676609105180534 |
|
name: Cosine Accuracy@10 |
|
- type: cosine_precision@1 |
|
value: 0.33761381475667196 |
|
name: Cosine Precision@1 |
|
- type: cosine_precision@3 |
|
value: 0.2139403453689168 |
|
name: Cosine Precision@3 |
|
- type: cosine_precision@5 |
|
value: 0.17194976452119312 |
|
name: Cosine Precision@5 |
|
- type: cosine_precision@10 |
|
value: 0.12513029827315542 |
|
name: Cosine Precision@10 |
|
- type: cosine_recall@1 |
|
value: 0.18011250836329812 |
|
name: Cosine Recall@1 |
|
- type: cosine_recall@3 |
|
value: 0.29934842463906286 |
|
name: Cosine Recall@3 |
|
- type: cosine_recall@5 |
|
value: 0.3551682644865234 |
|
name: Cosine Recall@5 |
|
- type: cosine_recall@10 |
|
value: 0.441603286977559 |
|
name: Cosine Recall@10 |
|
- type: cosine_ndcg@10 |
|
value: 0.3802306621694099 |
|
name: Cosine Ndcg@10 |
|
- type: cosine_mrr@10 |
|
value: 0.43513199272382946 |
|
name: Cosine Mrr@10 |
|
- type: cosine_map@100 |
|
value: 0.3136868503455821 |
|
name: Cosine Map@100 |
|
--- |
|
|
|
# SentenceTransformer based on Qwen/Qwen2.5-0.5B-Instruct |
|
|
|
This is a [sentence-transformers](https://www.SBERT.net) model finetuned from [Qwen/Qwen2.5-0.5B-Instruct](https://huggingface.co/Qwen/Qwen2.5-0.5B-Instruct). It maps sentences & paragraphs to a 896-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more. |
|
|
|
## Model Details |
|
|
|
### Model Description |
|
- **Model Type:** Sentence Transformer |
|
- **Base model:** [Qwen/Qwen2.5-0.5B-Instruct](https://huggingface.co/Qwen/Qwen2.5-0.5B-Instruct) <!-- at revision 7ae557604adf67be50417f59c2c2f167def9a775 --> |
|
- **Maximum Sequence Length:** 1024 tokens |
|
- **Output Dimensionality:** 896 dimensions |
|
- **Similarity Function:** Cosine Similarity |
|
<!-- - **Training Dataset:** Unknown --> |
|
<!-- - **Language:** Unknown --> |
|
<!-- - **License:** Unknown --> |
|
|
|
### Model Sources |
|
|
|
- **Documentation:** [Sentence Transformers Documentation](https://sbert.net) |
|
- **Repository:** [Sentence Transformers on GitHub](https://github.com/UKPLab/sentence-transformers) |
|
- **Hugging Face:** [Sentence Transformers on Hugging Face](https://huggingface.co/models?library=sentence-transformers) |
|
|
|
### Full Model Architecture |
|
|
|
``` |
|
SentenceTransformer( |
|
(0): Transformer({'max_seq_length': 1024, 'do_lower_case': False}) with Transformer model: Qwen2Model |
|
(1): Pooling({'word_embedding_dimension': 896, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True}) |
|
) |
|
``` |
|
|
|
## Usage |
|
|
|
### Direct Usage (Sentence Transformers) |
|
|
|
First install the Sentence Transformers library: |
|
|
|
```bash |
|
pip install -U sentence-transformers |
|
``` |
|
|
|
Then you can load this model and run inference. |
|
```python |
|
from sentence_transformers import SentenceTransformer |
|
|
|
# Download from the 🤗 Hub |
|
model = SentenceTransformer("AlexWortega/qwen_emb_600_best_21.11") |
|
# Run inference |
|
sentences = [ |
|
'Is a birth certificate a form of ID?', |
|
'Identity documents in the United States\nThe birth certificate is the initial identification document issued to parents shortly after the birth of their child. The birth certificate is typically issued by local governments, usually the city or county where a child is born. It is an important record, often called a "feeder document," because it establishes U.S. citizenship through birthright citizenship, which is then used to obtain, or is the basis for, all other identity documents.[2] By itself, the birth certificate is usually only considered proof of citizenship but not proof of identity, since it is issued without a photograph at birth, containing no identifying features. A birth certificate is normally produced along with proof of identity, such as a driver\'s license or the testimony of a third party (such as a parent), to establish identity or entitlement to a service.', |
|
"Identity document\nIn Canada, different forms of identification documentation are used, but there is no de jure national identity card. The Canadian passport is issued by the federal (national) government, and the provinces and territories issue various documents which can be used for identification purposes. The most commonly used forms of identification within Canada are the health card and driver's licence issued by provincial and territorial governments. The widespread usage of these two documents for identification purposes has made them de facto identity cards.", |
|
] |
|
embeddings = model.encode(sentences) |
|
print(embeddings.shape) |
|
# [3, 896] |
|
|
|
# Get the similarity scores for the embeddings |
|
similarities = model.similarity(embeddings, embeddings) |
|
print(similarities.shape) |
|
# [3, 3] |
|
``` |
|
|
|
<!-- |
|
### Direct Usage (Transformers) |
|
|
|
<details><summary>Click to see the direct usage in Transformers</summary> |
|
|
|
</details> |
|
--> |
|
|
|
<!-- |
|
### Downstream Usage (Sentence Transformers) |
|
|
|
You can finetune this model on your own dataset. |
|
|
|
<details><summary>Click to expand</summary> |
|
|
|
</details> |
|
--> |
|
|
|
<!-- |
|
### Out-of-Scope Use |
|
|
|
*List how the model may foreseeably be misused and address what users ought not to do with the model.* |
|
--> |
|
|
|
## Evaluation |
|
|
|
### Metrics |
|
|
|
#### Information Retrieval |
|
|
|
* Datasets: `NanoClimateFEVER`, `NanoDBPedia`, `NanoFEVER`, `NanoFiQA2018`, `NanoHotpotQA`, `NanoMSMARCO`, `NanoNFCorpus`, `NanoNQ`, `NanoQuoraRetrieval`, `NanoSCIDOCS`, `NanoArguAna`, `NanoSciFact` and `NanoTouche2020` |
|
* Evaluated with [<code>InformationRetrievalEvaluator</code>](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.InformationRetrievalEvaluator) |
|
|
|
| Metric | NanoClimateFEVER | NanoDBPedia | NanoFEVER | NanoFiQA2018 | NanoHotpotQA | NanoMSMARCO | NanoNFCorpus | NanoNQ | NanoQuoraRetrieval | NanoSCIDOCS | NanoArguAna | NanoSciFact | NanoTouche2020 | |
|
|:--------------------|:-----------------|:------------|:-----------|:-------------|:-------------|:------------|:-------------|:-----------|:-------------------|:------------|:------------|:------------|:---------------| |
|
| cosine_accuracy@1 | 0.18 | 0.6 | 0.28 | 0.18 | 0.46 | 0.22 | 0.32 | 0.28 | 0.84 | 0.26 | 0.06 | 0.26 | 0.449 | |
|
| cosine_accuracy@3 | 0.28 | 0.72 | 0.56 | 0.28 | 0.56 | 0.34 | 0.42 | 0.4 | 0.92 | 0.38 | 0.42 | 0.38 | 0.7143 | |
|
| cosine_accuracy@5 | 0.32 | 0.82 | 0.66 | 0.4 | 0.6 | 0.34 | 0.46 | 0.5 | 0.92 | 0.48 | 0.52 | 0.5 | 0.7959 | |
|
| cosine_accuracy@10 | 0.4 | 0.88 | 0.82 | 0.5 | 0.68 | 0.44 | 0.52 | 0.58 | 0.98 | 0.62 | 0.68 | 0.6 | 0.9796 | |
|
| cosine_precision@1 | 0.18 | 0.6 | 0.28 | 0.18 | 0.46 | 0.22 | 0.32 | 0.28 | 0.84 | 0.26 | 0.06 | 0.26 | 0.449 | |
|
| cosine_precision@3 | 0.1 | 0.4333 | 0.1867 | 0.1133 | 0.2333 | 0.1133 | 0.28 | 0.1333 | 0.3667 | 0.1533 | 0.14 | 0.1333 | 0.3946 | |
|
| cosine_precision@5 | 0.072 | 0.416 | 0.136 | 0.096 | 0.148 | 0.068 | 0.24 | 0.1 | 0.236 | 0.148 | 0.104 | 0.104 | 0.3673 | |
|
| cosine_precision@10 | 0.052 | 0.34 | 0.086 | 0.062 | 0.09 | 0.044 | 0.18 | 0.058 | 0.128 | 0.118 | 0.068 | 0.066 | 0.3347 | |
|
| cosine_recall@1 | 0.07 | 0.0588 | 0.26 | 0.0969 | 0.23 | 0.22 | 0.0125 | 0.27 | 0.7407 | 0.053 | 0.06 | 0.235 | 0.0347 | |
|
| cosine_recall@3 | 0.1283 | 0.1203 | 0.52 | 0.1971 | 0.35 | 0.34 | 0.0367 | 0.37 | 0.8687 | 0.093 | 0.42 | 0.355 | 0.0924 | |
|
| cosine_recall@5 | 0.145 | 0.1722 | 0.63 | 0.275 | 0.37 | 0.34 | 0.0562 | 0.47 | 0.886 | 0.15 | 0.52 | 0.465 | 0.1378 | |
|
| cosine_recall@10 | 0.1933 | 0.2396 | 0.79 | 0.3352 | 0.45 | 0.44 | 0.0669 | 0.53 | 0.956 | 0.2407 | 0.68 | 0.58 | 0.2392 | |
|
| **cosine_ndcg@10** | **0.1694** | **0.4329** | **0.5142** | **0.2365** | **0.4053** | **0.3192** | **0.2211** | **0.3978** | **0.889** | **0.2161** | **0.3708** | **0.3967** | **0.3738** | |
|
| cosine_mrr@10 | 0.2476 | 0.6865 | 0.4404 | 0.2642 | 0.5121 | 0.2824 | 0.3847 | 0.3705 | 0.8829 | 0.3527 | 0.2719 | 0.3504 | 0.6104 | |
|
| cosine_map@100 | 0.1356 | 0.2693 | 0.4253 | 0.1923 | 0.3491 | 0.2998 | 0.0815 | 0.3635 | 0.8629 | 0.1591 | 0.2829 | 0.3458 | 0.3108 | |
|
|
|
#### Nano BEIR |
|
|
|
* Dataset: `NanoBEIR_mean` |
|
* Evaluated with [<code>NanoBEIREvaluator</code>](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.NanoBEIREvaluator) |
|
|
|
| Metric | Value | |
|
|:--------------------|:-----------| |
|
| cosine_accuracy@1 | 0.3376 | |
|
| cosine_accuracy@3 | 0.4903 | |
|
| cosine_accuracy@5 | 0.5628 | |
|
| cosine_accuracy@10 | 0.6677 | |
|
| cosine_precision@1 | 0.3376 | |
|
| cosine_precision@3 | 0.2139 | |
|
| cosine_precision@5 | 0.1719 | |
|
| cosine_precision@10 | 0.1251 | |
|
| cosine_recall@1 | 0.1801 | |
|
| cosine_recall@3 | 0.2993 | |
|
| cosine_recall@5 | 0.3552 | |
|
| cosine_recall@10 | 0.4416 | |
|
| **cosine_ndcg@10** | **0.3802** | |
|
| cosine_mrr@10 | 0.4351 | |
|
| cosine_map@100 | 0.3137 | |
|
|
|
<!-- |
|
## Bias, Risks and Limitations |
|
|
|
*What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.* |
|
--> |
|
|
|
<!-- |
|
### Recommendations |
|
|
|
*What are recommendations with respect to the foreseeable issues? For example, filtering explicit content.* |
|
--> |
|
|
|
## Training Details |
|
|
|
### Training Dataset |
|
|
|
#### Unnamed Dataset |
|
|
|
|
|
* Size: 1,077,240 training samples |
|
* Columns: <code>query</code>, <code>response</code>, and <code>negative</code> |
|
* Approximate statistics based on the first 1000 samples: |
|
| | query | response | negative | |
|
|:--------|:---------------------------------------------------------------------------------|:-------------------------------------------------------------------------------------|:------------------------------------------------------------------------------------| |
|
| type | string | string | string | |
|
| details | <ul><li>min: 4 tokens</li><li>mean: 8.76 tokens</li><li>max: 26 tokens</li></ul> | <ul><li>min: 23 tokens</li><li>mean: 141.88 tokens</li><li>max: 532 tokens</li></ul> | <ul><li>min: 4 tokens</li><li>mean: 134.02 tokens</li><li>max: 472 tokens</li></ul> | |
|
* Samples: |
|
| query | response | negative | |
|
|:--------------------------------------------------|:---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| |
|
| <code>Was there a year 0?</code> | <code>Year zero<br>Year zero does not exist in the anno Domini system usually used to number years in the Gregorian calendar and in its predecessor, the Julian calendar. In this system, the year 1 BC is followed by AD 1. However, there is a year zero in astronomical year numbering (where it coincides with the Julian year 1 BC) and in ISO 8601:2004 (where it coincides with the Gregorian year 1 BC) as well as in all Buddhist and Hindu calendars.</code> | <code>504<br>Year 504 (DIV) was a leap year starting on Thursday (link will display the full calendar) of the Julian calendar. At the time, it was known as the Year of the Consulship of Nicomachus without colleague (or, less frequently, year 1257 "Ab urbe condita"). The denomination 504 for this year has been used since the early medieval period, when the Anno Domini calendar era became the prevalent method in Europe for naming years.</code> | |
|
| <code>When is the dialectical method used?</code> | <code>Dialectic<br>Dialectic or dialectics (Greek: διαλεκτική, dialektikḗ; related to dialogue), also known as the dialectical method, is at base a discourse between two or more people holding different points of view about a subject but wishing to establish the truth through reasoned arguments. Dialectic resembles debate, but the concept excludes subjective elements such as emotional appeal and the modern pejorative sense of rhetoric.[1][2] Dialectic may be contrasted with the didactic method, wherein one side of the conversation teaches the other. Dialectic is alternatively known as minor logic, as opposed to major logic or critique.</code> | <code>Derek Bentley case<br>Another factor in the posthumous defence was that a "confession" recorded by Bentley, which was claimed by the prosecution to be a "verbatim record of dictated monologue", was shown by forensic linguistics methods to have been largely edited by policemen. Linguist Malcolm Coulthard showed that certain patterns, such as the frequency of the word "then" and the grammatical use of "then" after the grammatical subject ("I then" rather than "then I"), were not consistent with Bentley's use of language (his idiolect), as evidenced in court testimony. These patterns fit better the recorded testimony of the policemen involved. This is one of the earliest uses of forensic linguistics on record.</code> | |
|
| <code>What do Grasshoppers eat?</code> | <code>Grasshopper<br>Grasshoppers are plant-eaters, with a few species at times becoming serious pests of cereals, vegetables and pasture, especially when they swarm in their millions as locusts and destroy crops over wide areas. They protect themselves from predators by camouflage; when detected, many species attempt to startle the predator with a brilliantly-coloured wing-flash while jumping and (if adult) launching themselves into the air, usually flying for only a short distance. Other species such as the rainbow grasshopper have warning coloration which deters predators. Grasshoppers are affected by parasites and various diseases, and many predatory creatures feed on both nymphs and adults. The eggs are the subject of attack by parasitoids and predators.</code> | <code>Groundhog<br>Very often the dens of groundhogs provide homes for other animals including skunks, red foxes, and cottontail rabbits. The fox and skunk feed upon field mice, grasshoppers, beetles and other creatures that destroy farm crops. In aiding these animals, the groundhog indirectly helps the farmer. In addition to providing homes for itself and other animals, the groundhog aids in soil improvement by bringing subsoil to the surface. The groundhog is also a valuable game animal and is considered a difficult sport when hunted in a fair manner. In some parts of Appalachia, they are eaten.</code> | |
|
* Loss: [<code>MultipleNegativesRankingLoss</code>](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#multiplenegativesrankingloss) with these parameters: |
|
```json |
|
{ |
|
"scale": 20.0, |
|
"similarity_fct": "cos_sim" |
|
} |
|
``` |
|
|
|
### Training Hyperparameters |
|
#### Non-Default Hyperparameters |
|
|
|
- `eval_strategy`: steps |
|
- `gradient_accumulation_steps`: 8 |
|
- `learning_rate`: 0.0001 |
|
- `max_grad_norm`: 0.01 |
|
- `num_train_epochs`: 2 |
|
- `warmup_ratio`: 0.4 |
|
- `bf16`: True |
|
- `dataloader_num_workers`: 8 |
|
- `batch_sampler`: no_duplicates |
|
|
|
#### All Hyperparameters |
|
<details><summary>Click to expand</summary> |
|
|
|
- `overwrite_output_dir`: False |
|
- `do_predict`: False |
|
- `eval_strategy`: steps |
|
- `prediction_loss_only`: True |
|
- `per_device_train_batch_size`: 8 |
|
- `per_device_eval_batch_size`: 8 |
|
- `per_gpu_train_batch_size`: None |
|
- `per_gpu_eval_batch_size`: None |
|
- `gradient_accumulation_steps`: 8 |
|
- `eval_accumulation_steps`: None |
|
- `torch_empty_cache_steps`: None |
|
- `learning_rate`: 0.0001 |
|
- `weight_decay`: 0.0 |
|
- `adam_beta1`: 0.9 |
|
- `adam_beta2`: 0.999 |
|
- `adam_epsilon`: 1e-08 |
|
- `max_grad_norm`: 0.01 |
|
- `num_train_epochs`: 2 |
|
- `max_steps`: -1 |
|
- `lr_scheduler_type`: linear |
|
- `lr_scheduler_kwargs`: {} |
|
- `warmup_ratio`: 0.4 |
|
- `warmup_steps`: 0 |
|
- `log_level`: passive |
|
- `log_level_replica`: warning |
|
- `log_on_each_node`: True |
|
- `logging_nan_inf_filter`: True |
|
- `save_safetensors`: True |
|
- `save_on_each_node`: False |
|
- `save_only_model`: False |
|
- `restore_callback_states_from_checkpoint`: False |
|
- `no_cuda`: False |
|
- `use_cpu`: False |
|
- `use_mps_device`: False |
|
- `seed`: 42 |
|
- `data_seed`: None |
|
- `jit_mode_eval`: False |
|
- `use_ipex`: False |
|
- `bf16`: True |
|
- `fp16`: False |
|
- `fp16_opt_level`: O1 |
|
- `half_precision_backend`: auto |
|
- `bf16_full_eval`: False |
|
- `fp16_full_eval`: False |
|
- `tf32`: None |
|
- `local_rank`: 0 |
|
- `ddp_backend`: None |
|
- `tpu_num_cores`: None |
|
- `tpu_metrics_debug`: False |
|
- `debug`: [] |
|
- `dataloader_drop_last`: True |
|
- `dataloader_num_workers`: 8 |
|
- `dataloader_prefetch_factor`: None |
|
- `past_index`: -1 |
|
- `disable_tqdm`: False |
|
- `remove_unused_columns`: True |
|
- `label_names`: None |
|
- `load_best_model_at_end`: False |
|
- `ignore_data_skip`: False |
|
- `fsdp`: [] |
|
- `fsdp_min_num_params`: 0 |
|
- `fsdp_config`: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False} |
|
- `fsdp_transformer_layer_cls_to_wrap`: None |
|
- `accelerator_config`: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None} |
|
- `deepspeed`: None |
|
- `label_smoothing_factor`: 0.0 |
|
- `optim`: adamw_torch |
|
- `optim_args`: None |
|
- `adafactor`: False |
|
- `group_by_length`: False |
|
- `length_column_name`: length |
|
- `ddp_find_unused_parameters`: None |
|
- `ddp_bucket_cap_mb`: None |
|
- `ddp_broadcast_buffers`: False |
|
- `dataloader_pin_memory`: True |
|
- `dataloader_persistent_workers`: False |
|
- `skip_memory_metrics`: True |
|
- `use_legacy_prediction_loop`: False |
|
- `push_to_hub`: False |
|
- `resume_from_checkpoint`: None |
|
- `hub_model_id`: None |
|
- `hub_strategy`: every_save |
|
- `hub_private_repo`: False |
|
- `hub_always_push`: False |
|
- `gradient_checkpointing`: False |
|
- `gradient_checkpointing_kwargs`: None |
|
- `include_inputs_for_metrics`: False |
|
- `include_for_metrics`: [] |
|
- `eval_do_concat_batches`: True |
|
- `fp16_backend`: auto |
|
- `push_to_hub_model_id`: None |
|
- `push_to_hub_organization`: None |
|
- `mp_parameters`: |
|
- `auto_find_batch_size`: False |
|
- `full_determinism`: False |
|
- `torchdynamo`: None |
|
- `ray_scope`: last |
|
- `ddp_timeout`: 1800 |
|
- `torch_compile`: False |
|
- `torch_compile_backend`: None |
|
- `torch_compile_mode`: None |
|
- `dispatch_batches`: None |
|
- `split_batches`: None |
|
- `include_tokens_per_second`: False |
|
- `include_num_input_tokens_seen`: False |
|
- `neftune_noise_alpha`: None |
|
- `optim_target_modules`: None |
|
- `batch_eval_metrics`: False |
|
- `eval_on_start`: False |
|
- `use_liger_kernel`: False |
|
- `eval_use_gather_object`: False |
|
- `average_tokens_across_devices`: False |
|
- `prompts`: None |
|
- `batch_sampler`: no_duplicates |
|
- `multi_dataset_batch_sampler`: proportional |
|
|
|
</details> |
|
|
|
### Training Logs |
|
<details><summary>Click to expand</summary> |
|
|
|
| Epoch | Step | Training Loss | NanoClimateFEVER_cosine_ndcg@10 | NanoDBPedia_cosine_ndcg@10 | NanoFEVER_cosine_ndcg@10 | NanoFiQA2018_cosine_ndcg@10 | NanoHotpotQA_cosine_ndcg@10 | NanoMSMARCO_cosine_ndcg@10 | NanoNFCorpus_cosine_ndcg@10 | NanoNQ_cosine_ndcg@10 | NanoQuoraRetrieval_cosine_ndcg@10 | NanoSCIDOCS_cosine_ndcg@10 | NanoArguAna_cosine_ndcg@10 | NanoSciFact_cosine_ndcg@10 | NanoTouche2020_cosine_ndcg@10 | NanoBEIR_mean_cosine_ndcg@10 | |
|
|:------:|:----:|:-------------:|:-------------------------------:|:--------------------------:|:------------------------:|:---------------------------:|:---------------------------:|:--------------------------:|:---------------------------:|:---------------------:|:---------------------------------:|:--------------------------:|:--------------------------:|:--------------------------:|:-----------------------------:|:----------------------------:| |
|
| 0.0002 | 2 | 1.8808 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0005 | 4 | 1.9239 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0007 | 6 | 2.0324 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0010 | 8 | 2.0387 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0012 | 10 | 2.0336 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0014 | 12 | 1.9943 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0017 | 14 | 1.971 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0019 | 16 | 1.9206 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0021 | 18 | 1.8157 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0024 | 20 | 1.8605 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0026 | 22 | 1.862 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0029 | 24 | 1.9313 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0031 | 26 | 1.8326 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0033 | 28 | 1.9208 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0036 | 30 | 2.4718 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0038 | 32 | 2.4819 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0040 | 34 | 2.4956 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0043 | 36 | 2.4335 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0045 | 38 | 2.4694 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0048 | 40 | 2.5719 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0050 | 42 | 2.4666 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0052 | 44 | 2.4919 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0055 | 46 | 2.4179 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0057 | 48 | 2.4022 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0059 | 50 | 2.39 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0062 | 52 | 2.4682 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0064 | 54 | 2.3442 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0067 | 56 | 2.3157 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0069 | 58 | 2.2665 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0071 | 60 | 2.2969 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0074 | 62 | 2.1652 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0076 | 64 | 2.1243 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0078 | 66 | 2.0499 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0081 | 68 | 2.0115 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0083 | 70 | 1.8372 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0086 | 72 | 1.6257 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0088 | 74 | 1.6398 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0090 | 76 | 1.4927 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0093 | 78 | 1.3491 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0095 | 80 | 1.3303 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0097 | 82 | 1.3846 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0100 | 84 | 1.2647 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0102 | 86 | 1.1579 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0105 | 88 | 1.0146 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0107 | 90 | 0.9201 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0109 | 92 | 0.8631 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0112 | 94 | 0.7801 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0114 | 96 | 0.7813 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0116 | 98 | 0.7898 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0119 | 100 | 0.722 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0121 | 102 | 0.7595 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0124 | 104 | 0.6245 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0126 | 106 | 0.6036 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0128 | 108 | 0.7248 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0131 | 110 | 0.637 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0133 | 112 | 0.6205 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0135 | 114 | 0.5956 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0138 | 116 | 0.6126 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0140 | 118 | 0.547 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0143 | 120 | 0.5414 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0145 | 122 | 0.4896 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0147 | 124 | 0.5351 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0150 | 126 | 0.5404 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0152 | 128 | 0.4479 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0154 | 130 | 1.1779 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0157 | 132 | 1.4533 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0159 | 134 | 1.5042 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0162 | 136 | 1.2167 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0164 | 138 | 1.2484 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0166 | 140 | 1.1236 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0169 | 142 | 1.1729 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0171 | 144 | 1.0076 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0173 | 146 | 1.0314 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0176 | 148 | 0.9106 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0178 | 150 | 0.8994 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0181 | 152 | 0.9679 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0183 | 154 | 0.8576 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0185 | 156 | 0.777 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0188 | 158 | 0.8527 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0190 | 160 | 0.864 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0192 | 162 | 0.807 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0195 | 164 | 0.9083 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0197 | 166 | 0.7705 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0200 | 168 | 0.7179 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0202 | 170 | 0.7485 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0204 | 172 | 0.7198 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0207 | 174 | 0.7712 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0209 | 176 | 0.82 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0212 | 178 | 0.7744 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0214 | 180 | 0.7668 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0216 | 182 | 0.6501 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0219 | 184 | 0.6327 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0221 | 186 | 0.752 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0223 | 188 | 0.6204 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0226 | 190 | 0.6258 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0228 | 192 | 0.607 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0231 | 194 | 0.5688 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0233 | 196 | 0.6831 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0235 | 198 | 0.5653 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0238 | 200 | 0.5966 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0240 | 202 | 0.5798 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0242 | 204 | 0.5991 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0245 | 206 | 0.5856 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0247 | 208 | 0.5935 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0250 | 210 | 0.5624 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0252 | 212 | 0.6188 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0254 | 214 | 0.5497 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0257 | 216 | 0.582 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0259 | 218 | 0.5912 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0261 | 220 | 0.4818 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0264 | 222 | 0.5686 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0266 | 224 | 0.5174 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0269 | 226 | 0.523 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0271 | 228 | 0.5337 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0273 | 230 | 0.5253 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0276 | 232 | 0.5434 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0278 | 234 | 0.5351 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0280 | 236 | 0.5202 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0283 | 238 | 0.4611 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0285 | 240 | 0.4509 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0288 | 242 | 0.5217 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0290 | 244 | 0.5256 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0292 | 246 | 0.5529 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0295 | 248 | 0.4944 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0297 | 250 | 0.568 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0299 | 252 | 0.5024 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0302 | 254 | 0.5094 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0304 | 256 | 0.5057 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0307 | 258 | 0.5424 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0309 | 260 | 0.6485 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0311 | 262 | 0.4823 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0314 | 264 | 0.475 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0316 | 266 | 0.3753 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0318 | 268 | 0.5117 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0321 | 270 | 0.4067 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0323 | 272 | 0.4706 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0326 | 274 | 0.4099 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0328 | 276 | 0.4251 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0330 | 278 | 0.4392 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0333 | 280 | 0.5373 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0335 | 282 | 0.4259 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0337 | 284 | 0.4227 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0340 | 286 | 0.4774 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0342 | 288 | 0.4878 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0345 | 290 | 0.5619 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0347 | 292 | 0.5061 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0349 | 294 | 0.5434 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0352 | 296 | 0.5115 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0354 | 298 | 0.4281 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0356 | 300 | 0.4287 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0359 | 302 | 0.4864 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0361 | 304 | 0.4724 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0364 | 306 | 0.4607 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0366 | 308 | 0.3978 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0368 | 310 | 0.4851 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0371 | 312 | 0.3466 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0373 | 314 | 0.565 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0375 | 316 | 0.4122 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0378 | 318 | 0.3757 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0380 | 320 | 0.4673 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0383 | 322 | 0.4358 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0385 | 324 | 0.4423 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0387 | 326 | 0.3754 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0390 | 328 | 0.4358 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0392 | 330 | 0.408 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0394 | 332 | 0.3901 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0397 | 334 | 0.4155 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0399 | 336 | 0.379 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0402 | 338 | 0.373 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0404 | 340 | 0.2917 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0406 | 342 | 0.3755 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0409 | 344 | 0.3262 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0411 | 346 | 0.4975 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0414 | 348 | 0.3469 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0416 | 350 | 0.3895 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0418 | 352 | 0.4424 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0421 | 354 | 0.3609 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0423 | 356 | 0.434 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0425 | 358 | 0.4474 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0428 | 360 | 0.3514 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0430 | 362 | 0.4029 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0433 | 364 | 0.4438 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0435 | 366 | 0.4271 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0437 | 368 | 0.3825 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0440 | 370 | 0.3848 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0442 | 372 | 0.4088 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0444 | 374 | 0.4188 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0447 | 376 | 0.4333 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0449 | 378 | 0.3784 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0452 | 380 | 0.4509 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0454 | 382 | 0.4084 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0456 | 384 | 0.371 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0459 | 386 | 0.3965 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0461 | 388 | 0.375 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0463 | 390 | 0.4098 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0466 | 392 | 0.4198 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0468 | 394 | 0.3854 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0471 | 396 | 0.3146 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0473 | 398 | 0.3892 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0475 | 400 | 0.3295 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0478 | 402 | 0.4124 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0480 | 404 | 0.3039 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0482 | 406 | 0.3353 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0485 | 408 | 0.4382 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0487 | 410 | 0.4013 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0490 | 412 | 0.3283 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0492 | 414 | 0.4264 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0494 | 416 | 0.4295 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0497 | 418 | 0.3451 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0499 | 420 | 0.2973 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0501 | 422 | 0.3734 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0504 | 424 | 0.3992 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0506 | 426 | 0.3234 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0509 | 428 | 0.4007 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0511 | 430 | 0.4446 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0513 | 432 | 0.282 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0516 | 434 | 0.3922 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0518 | 436 | 0.4224 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0520 | 438 | 0.3362 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0523 | 440 | 0.3461 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0525 | 442 | 0.344 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0528 | 444 | 0.4355 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0530 | 446 | 0.3443 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0532 | 448 | 0.4363 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0535 | 450 | 0.3282 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0537 | 452 | 0.3761 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0539 | 454 | 0.3279 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0542 | 456 | 0.3774 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0544 | 458 | 0.3888 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0547 | 460 | 0.5149 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0549 | 462 | 0.343 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0551 | 464 | 0.3943 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0554 | 466 | 0.366 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0556 | 468 | 0.344 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0558 | 470 | 0.3681 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0561 | 472 | 0.3041 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0563 | 474 | 0.3857 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0566 | 476 | 0.3665 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0568 | 478 | 0.3871 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0570 | 480 | 0.4707 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0573 | 482 | 0.4031 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0575 | 484 | 0.385 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0577 | 486 | 0.2868 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0580 | 488 | 0.3637 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0582 | 490 | 0.4484 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0585 | 492 | 0.4984 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0587 | 494 | 0.3725 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0589 | 496 | 0.3102 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0592 | 498 | 0.3529 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0594 | 500 | 0.3929 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0596 | 502 | 0.3012 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0599 | 504 | 0.4137 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0601 | 506 | 0.3987 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0604 | 508 | 0.3724 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0606 | 510 | 0.3761 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0608 | 512 | 0.389 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0611 | 514 | 0.3775 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0613 | 516 | 0.3429 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0616 | 518 | 0.348 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0618 | 520 | 0.3706 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0620 | 522 | 0.3563 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0623 | 524 | 0.3029 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0625 | 526 | 0.4227 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0627 | 528 | 0.3457 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0630 | 530 | 0.3666 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0632 | 532 | 0.3331 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0635 | 534 | 0.3362 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0637 | 536 | 0.3974 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0639 | 538 | 0.3841 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0642 | 540 | 0.3318 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0644 | 542 | 0.3349 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0646 | 544 | 0.461 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0649 | 546 | 0.3271 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0651 | 548 | 0.3901 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0654 | 550 | 0.3292 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0656 | 552 | 0.3291 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0658 | 554 | 0.374 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0661 | 556 | 0.3432 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0663 | 558 | 0.2994 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0665 | 560 | 0.3391 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0668 | 562 | 0.3764 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0670 | 564 | 0.2555 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0673 | 566 | 0.3553 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0675 | 568 | 0.3436 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0677 | 570 | 0.4347 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0680 | 572 | 0.3271 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0682 | 574 | 0.2988 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0684 | 576 | 0.3698 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0687 | 578 | 0.3309 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0689 | 580 | 0.3529 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0692 | 582 | 0.3685 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0694 | 584 | 0.333 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0696 | 586 | 0.3344 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0699 | 588 | 0.3496 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0701 | 590 | 0.3616 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0703 | 592 | 0.3637 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0706 | 594 | 0.3745 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0708 | 596 | 0.3465 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0711 | 598 | 0.4128 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | |
|
| 0.0713 | 600 | 0.3674 | 0.1694 | 0.4329 | 0.5142 | 0.2365 | 0.4053 | 0.3192 | 0.2211 | 0.3978 | 0.8890 | 0.2161 | 0.3708 | 0.3967 | 0.3738 | 0.3802 | |
|
|
|
</details> |
|
|
|
### Framework Versions |
|
- Python: 3.10.12 |
|
- Sentence Transformers: 3.3.0 |
|
- Transformers: 4.46.2 |
|
- PyTorch: 2.1.1+cu121 |
|
- Accelerate: 0.34.2 |
|
- Datasets: 3.1.0 |
|
- Tokenizers: 0.20.3 |
|
|
|
## Citation |
|
|
|
### BibTeX |
|
|
|
#### Sentence Transformers |
|
```bibtex |
|
@inproceedings{reimers-2019-sentence-bert, |
|
title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks", |
|
author = "Reimers, Nils and Gurevych, Iryna", |
|
booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing", |
|
month = "11", |
|
year = "2019", |
|
publisher = "Association for Computational Linguistics", |
|
url = "https://arxiv.org/abs/1908.10084", |
|
} |
|
``` |
|
|
|
#### MultipleNegativesRankingLoss |
|
```bibtex |
|
@misc{henderson2017efficient, |
|
title={Efficient Natural Language Response Suggestion for Smart Reply}, |
|
author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil}, |
|
year={2017}, |
|
eprint={1705.00652}, |
|
archivePrefix={arXiv}, |
|
primaryClass={cs.CL} |
|
} |
|
``` |
|
|
|
<!-- |
|
## Glossary |
|
|
|
*Clearly define terms in order to be accessible across audiences.* |
|
--> |
|
|
|
<!-- |
|
## Model Card Authors |
|
|
|
*Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction.* |
|
--> |
|
|
|
<!-- |
|
## Model Card Contact |
|
|
|
*Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.* |
|
--> |