---
base_model: Snowflake/snowflake-arctic-embed-m
library_name: sentence-transformers
metrics:
- cosine_accuracy@1
- cosine_accuracy@3
- cosine_accuracy@5
- cosine_accuracy@10
- cosine_precision@1
- cosine_precision@3
- cosine_precision@5
- cosine_precision@10
- cosine_recall@1
- cosine_recall@3
- cosine_recall@5
- cosine_recall@10
- cosine_ndcg@10
- cosine_mrr@10
- cosine_map@100
- dot_accuracy@1
- dot_accuracy@3
- dot_accuracy@5
- dot_accuracy@10
- dot_precision@1
- dot_precision@3
- dot_precision@5
- dot_precision@10
- dot_recall@1
- dot_recall@3
- dot_recall@5
- dot_recall@10
- dot_ndcg@10
- dot_mrr@10
- dot_map@100
pipeline_tag: sentence-similarity
tags:
- sentence-transformers
- sentence-similarity
- feature-extraction
- generated_from_trainer
- dataset_size:522
- loss:MatryoshkaLoss
- loss:MultipleNegativesRankingLoss
widget:
- source_sentence: How did the hiring tool's design contribute to the rejection of
women applicants?
sentences:
- "legal protections. Throughout this framework the term “algorithmic discrimination”\
\ takes this meaning (and \nnot a technical understanding of discrimination as\
\ distinguishing between items). \nAUTOMATED SYSTEM: An \"automated system\" is\
\ any system, software, or process that uses computation as \nwhole or part of\
\ a system to determine outcomes, make or aid decisions, inform policy implementation,\
\ collect \ndata or observations, or otherwise interact with individuals and/or\
\ communities. Automated systems \ninclude, but are not limited to, systems derived\
\ from machine learning, statistics, or other data processing \nor artificial\
\ intelligence techniques, and exclude passive computing infrastructure. “Passive\
\ computing"
- "communities. \n• An automated system using nontraditional factors such as educational\
\ attainment and employment history as\npart of its loan underwriting and pricing\
\ model was found to be much more likely to charge an applicant whoattended a\
\ Historically Black College or University (HBCU) higher loan prices for refinancing\
\ a student loanthan an applicant who did not attend an HBCU. This was found to\
\ be true even when controlling for\nother credit-related factors.32\n•A hiring\
\ tool that learned the features of a company's employees (predominantly men)\
\ rejected women appli -\ncants for spurious and discriminatory reasons; resumes\
\ with the word “women’s,” such as “women’s\nchess club captain,” were penalized\
\ in the candidate ranking.33"
- dures before deploying the system, as well as responsibility of specific individuals
or entities to oversee ongoing assessment and mitigation. Organizational stakeholders
including those with oversight of the business process or operation being automated,
as well as other organizational divisions that may be affected due to the use
of the system, should be involved in establishing governance procedures. Responsibility
should rest high enough in the organization that decisions about resources, mitigation,
incident response, and potential rollback can be made promptly, with sufficient
weight given to risk mitigation objectives against competing concerns. Those holding
this responsibility should be made aware of any use cases with the
- source_sentence: How are companies using individual profiles based on tracked behavior
to impact the American public?
sentences:
- "requests should be used so that users understand for what use contexts, time\
\ span, and entities they are providing data and metadata consent. User experience\
\ research should be performed to ensure these consent requests meet performance\
\ standards for readability and comprehension. This includes ensuring that consent\
\ requests are accessible to users with disabilities and are available in the\
\ language(s) and reading level appro\n-\npriate for the audience. User experience\
\ design choices that intentionally obfuscate or manipulate user choice (i.e.,\
\ “dark patterns”) should be not be used. \n34\n DATA PRIVACY \nWHAT SHOULD\
\ BE EXPECTED OF AUTOMATED SYSTEMS"
- with more and more companies tracking the behavior of the American public, building
individual profiles based on this data, and using this granular-level information
as input into automated systems that further track, profile, and impact the American
public. Government agencies, particularly law enforcement agencies, also use and
help develop a variety of technologies that enhance and expand surveillance capabilities,
which similarly collect data used as input into other automated systems that directly
impact people’s lives. Federal law has not grown to address the expanding scale
of private data collection, or of the ability of governments at all levels to
access that data and leverage the means of private collection.
- "ways that threaten the rights of the American public. Too often, these tools\
\ are used to limit our opportunities and \nprevent our access to critical resources\
\ or services. These problems are well documented. In America and around \nthe\
\ world, systems supposed to help with patient care have proven unsafe, ineffective,\
\ or biased. Algorithms used \nin hiring and credit decisions have been found\
\ to reflect and reproduce existing unwanted inequities or embed \nnew harmful\
\ bias and discrimination. Unchecked social media data collection has been used\
\ to threaten people’s \nopportunities, undermine their privac y, or pervasively\
\ track their activity—often without their knowledge or \nconsent."
- source_sentence: What should entities developing technologies related to sensitive
data regularly report on?
sentences:
- "concerns that may limit their effectiveness. The results of assessments of the\
\ efficacy and potential bias of such human-based systems should be overseen by\
\ governance structures that have the potential to update the operation of the\
\ human-based system in order to mitigate these effects. \n50\n \n HUMAN\
\ ALTERNATIVES, \nCONSIDERATION, AND \nFALLBACK \nWHAT SHOULD BE EXPECTED OF AUTOMATED\
\ SYSTEMS\nThe expectations for automated systems are meant to serve as a blueprint\
\ for the development of additional \ntechnical standards and practices that are\
\ tailored for particular sectors and contexts. \nImplement additional human oversight\
\ and safeguards for automated systems related to \nsensitive domains"
- "performance testing including, but not limited to, accuracy, differential demographic\
\ impact, resulting \nerror rates (overall and per demographic group), and comparisons\
\ to previously deployed systems; \nongoing monitoring procedures and regular\
\ performance testing reports, including monitoring frequency, \nresults, and\
\ actions taken; and the procedures for and results from independent evaluations.\
\ Reporting \nshould be provided in a plain language and machine-readable manner.\
\ \n20\n \n \n \n \n \n \n SAFE AND EFFECTIVE \nSYSTEMS \nHOW THESE PRINCIPLES\
\ CAN MOVE INTO PRACTICE\nReal-life examples of how these principles can become\
\ reality, through laws, policies, and practical"
- "those who are less proximate do not (e.g., a teacher has access to their students’\
\ daily progress data while a \nsuperintendent does not). \nReporting. In addition\
\ to the reporting on data privacy (as listed above for non-sensitive data), entities\
\ devel-\noping technologies related to a sensitive domain and those collecting,\
\ using, storing, or sharing sensitive data \nshould, whenever appropriate, regularly\
\ provide public reports describing: any data security lapses or breaches \nthat\
\ resulted in sensitive data leaks; the numbe r, type, and outcomes of ethical\
\ pre-reviews undertaken; a \ndescription of any data sold, shared, or made public,\
\ and how that data was assessed to determine it did not pres-"
- source_sentence: What are the expectations for automated systems intended to serve
as a blueprint for?
sentences:
- 'Clear organizational oversight. Entities responsible for the development or use
of automated systems should lay out clear governance structures and procedures. This
includes clearly-stated governance proce
-'
- "critical resources or services. These rights, opportunities, and access to critical\
\ resources of services should \nbe enjoyed equally and be fully protected, regardless\
\ of the changing role that automated systems may play in \nour lives. \nThis\
\ framework describes protections that should be applied with respect to all automated\
\ systems that \nhave the potential to meaningfully impact individuals' or communities'\
\ exercise of: \nRIGHTS, OPPORTUNITIES, OR ACCESS\nCivil rights, civil liberties,\
\ and privacy, including freedom of speech, voting, and protections from discrimi\
\ -\nnation, excessive punishment, unlawful surveillance, and violations of privacy\
\ and other freedoms in both \npublic and private sector contexts;"
- "19\n \n \n SAFE AND EFFECTIVE \nSYSTEMS \nWHAT SHOULD BE EXPECTED OF AUTOMATED\
\ SYSTEMS\nThe expectations for automated systems are meant to serve as a blueprint\
\ for the development of additional \ntechnical standards and practices that are\
\ tailored for particular sectors and contexts. \nDerived data sources tracked\
\ and reviewed carefully. Data that is derived from other data through \nthe use\
\ of algorithms, such as data derived or inferred from prior model outputs, should\
\ be identified and tracked, e.g., via a specialized type in a data schema. Derived\
\ data should be viewed as potentially high-risk inputs that may lead to feedback\
\ loops, compounded harm, or inaccurate results. Such sources should be care\n\
-"
- source_sentence: What types of systems are considered time-critical according to
the context?
sentences:
- "Equity includes a commitment from the agencies that oversee mortgage lending\
\ to include a \nnondiscrimination standard in the proposed rules for Automated\
\ Valuation Models.52\nThe Equal Employment Opportunity Commission and the Department\
\ of Justice have clearly \nlaid out how employers’ use of AI and other automated\
\ systems can result in discrimination \nagainst job applicants and employees\
\ with disabilities.53 The documents explain \nhow employers’ use of software\
\ that relies on algorithmic decision-making may violate existing requirements\
\ \nunder Title I of the Americans with Disabilities Act (“ADA”). This technical\
\ assistance also provides practical"
- "Discrimination \nProtections \n \n WHAT SHOULD BE EXPECTED OF AUTOMATED\
\ SYSTEMS\nThe expectations for automated systems are meant to serve as a blueprint\
\ for the development of additional \ntechnical standards and practices that are\
\ tailored for particular sectors and contexts. \nDemonstrate that the system\
\ protects against algorithmic discrimination \nIndependent evaluation. As described\
\ in the section on Safe and Effective Systems, entities should allow \nindependent\
\ evaluation of potential algorithmic discrimination caused by automated systems\
\ they use or"
- "where possible, available before the harm occurs. Time-critical systems include,\
\ but are not limited to, \nvoting-related systems, automated building access\
\ and other access systems, systems that form a critical \ncomponent of healthcare,\
\ and systems that have the ability to withhold wages or otherwise cause \nimmediate\
\ financial penalties. \nEffective. The organizational structure surrounding processes\
\ for consideration and fallback should \nbe designed so that if the human decision-maker\
\ charged with reassessing a decision determines that it \nshould be overruled,\
\ the new decision will be effectively enacted. This includes ensuring that the\
\ new"
model-index:
- name: SentenceTransformer based on Snowflake/snowflake-arctic-embed-m
results:
- task:
type: information-retrieval
name: Information Retrieval
dataset:
name: Unknown
type: unknown
metrics:
- type: cosine_accuracy@1
value: 0.8448275862068966
name: Cosine Accuracy@1
- type: cosine_accuracy@3
value: 0.9482758620689655
name: Cosine Accuracy@3
- type: cosine_accuracy@5
value: 0.9770114942528736
name: Cosine Accuracy@5
- type: cosine_accuracy@10
value: 0.9942528735632183
name: Cosine Accuracy@10
- type: cosine_precision@1
value: 0.8448275862068966
name: Cosine Precision@1
- type: cosine_precision@3
value: 0.3160919540229885
name: Cosine Precision@3
- type: cosine_precision@5
value: 0.19540229885057464
name: Cosine Precision@5
- type: cosine_precision@10
value: 0.09942528735632182
name: Cosine Precision@10
- type: cosine_recall@1
value: 0.8448275862068966
name: Cosine Recall@1
- type: cosine_recall@3
value: 0.9482758620689655
name: Cosine Recall@3
- type: cosine_recall@5
value: 0.9770114942528736
name: Cosine Recall@5
- type: cosine_recall@10
value: 0.9942528735632183
name: Cosine Recall@10
- type: cosine_ndcg@10
value: 0.924865695917767
name: Cosine Ndcg@10
- type: cosine_mrr@10
value: 0.901963601532567
name: Cosine Mrr@10
- type: cosine_map@100
value: 0.9021617783062492
name: Cosine Map@100
- type: dot_accuracy@1
value: 0.8448275862068966
name: Dot Accuracy@1
- type: dot_accuracy@3
value: 0.9482758620689655
name: Dot Accuracy@3
- type: dot_accuracy@5
value: 0.9770114942528736
name: Dot Accuracy@5
- type: dot_accuracy@10
value: 0.9942528735632183
name: Dot Accuracy@10
- type: dot_precision@1
value: 0.8448275862068966
name: Dot Precision@1
- type: dot_precision@3
value: 0.3160919540229885
name: Dot Precision@3
- type: dot_precision@5
value: 0.19540229885057464
name: Dot Precision@5
- type: dot_precision@10
value: 0.09942528735632182
name: Dot Precision@10
- type: dot_recall@1
value: 0.8448275862068966
name: Dot Recall@1
- type: dot_recall@3
value: 0.9482758620689655
name: Dot Recall@3
- type: dot_recall@5
value: 0.9770114942528736
name: Dot Recall@5
- type: dot_recall@10
value: 0.9942528735632183
name: Dot Recall@10
- type: dot_ndcg@10
value: 0.924865695917767
name: Dot Ndcg@10
- type: dot_mrr@10
value: 0.901963601532567
name: Dot Mrr@10
- type: dot_map@100
value: 0.9021617783062492
name: Dot Map@100
---
# SentenceTransformer based on Snowflake/snowflake-arctic-embed-m
This is a [sentence-transformers](https://www.SBERT.net) model finetuned from [Snowflake/snowflake-arctic-embed-m](https://huggingface.co/Snowflake/snowflake-arctic-embed-m). It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.
## Model Details
### Model Description
- **Model Type:** Sentence Transformer
- **Base model:** [Snowflake/snowflake-arctic-embed-m](https://huggingface.co/Snowflake/snowflake-arctic-embed-m)
- **Maximum Sequence Length:** 512 tokens
- **Output Dimensionality:** 768 tokens
- **Similarity Function:** Cosine Similarity
### Model Sources
- **Documentation:** [Sentence Transformers Documentation](https://sbert.net)
- **Repository:** [Sentence Transformers on GitHub](https://github.com/UKPLab/sentence-transformers)
- **Hugging Face:** [Sentence Transformers on Hugging Face](https://huggingface.co/models?library=sentence-transformers)
### Full Model Architecture
```
SentenceTransformer(
(0): Transformer({'max_seq_length': 512, 'do_lower_case': False}) with Transformer model: BertModel
(1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': True, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
(2): Normalize()
)
```
## Usage
### Direct Usage (Sentence Transformers)
First install the Sentence Transformers library:
```bash
pip install -U sentence-transformers
```
Then you can load this model and run inference.
```python
from sentence_transformers import SentenceTransformer
# Download from the 🤗 Hub
model = SentenceTransformer("sentence_transformers_model_id")
# Run inference
sentences = [
'What types of systems are considered time-critical according to the context?',
'where possible, available before the harm occurs. Time-critical systems include, but are not limited to, \nvoting-related systems, automated building access and other access systems, systems that form a critical \ncomponent of healthcare, and systems that have the ability to withhold wages or otherwise cause \nimmediate financial penalties. \nEffective. The organizational structure surrounding processes for consideration and fallback should \nbe designed so that if the human decision-maker charged with reassessing a decision determines that it \nshould be overruled, the new decision will be effectively enacted. This includes ensuring that the new',
'Discrimination \nProtections \n \n WHAT SHOULD BE EXPECTED OF AUTOMATED SYSTEMS\nThe expectations for automated systems are meant to serve as a blueprint for the development of additional \ntechnical standards and practices that are tailored for particular sectors and contexts. \nDemonstrate that the system protects against algorithmic discrimination \nIndependent evaluation. As described in the section on Safe and Effective Systems, entities should allow \nindependent evaluation of potential algorithmic discrimination caused by automated systems they use or',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 768]
# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]
```
## Evaluation
### Metrics
#### Information Retrieval
* Evaluated with [InformationRetrievalEvaluator](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.InformationRetrievalEvaluator)
| Metric | Value |
|:--------------------|:-----------|
| cosine_accuracy@1 | 0.8448 |
| cosine_accuracy@3 | 0.9483 |
| cosine_accuracy@5 | 0.977 |
| cosine_accuracy@10 | 0.9943 |
| cosine_precision@1 | 0.8448 |
| cosine_precision@3 | 0.3161 |
| cosine_precision@5 | 0.1954 |
| cosine_precision@10 | 0.0994 |
| cosine_recall@1 | 0.8448 |
| cosine_recall@3 | 0.9483 |
| cosine_recall@5 | 0.977 |
| cosine_recall@10 | 0.9943 |
| cosine_ndcg@10 | 0.9249 |
| cosine_mrr@10 | 0.902 |
| **cosine_map@100** | **0.9022** |
| dot_accuracy@1 | 0.8448 |
| dot_accuracy@3 | 0.9483 |
| dot_accuracy@5 | 0.977 |
| dot_accuracy@10 | 0.9943 |
| dot_precision@1 | 0.8448 |
| dot_precision@3 | 0.3161 |
| dot_precision@5 | 0.1954 |
| dot_precision@10 | 0.0994 |
| dot_recall@1 | 0.8448 |
| dot_recall@3 | 0.9483 |
| dot_recall@5 | 0.977 |
| dot_recall@10 | 0.9943 |
| dot_ndcg@10 | 0.9249 |
| dot_mrr@10 | 0.902 |
| dot_map@100 | 0.9022 |
## Training Details
### Training Dataset
#### Unnamed Dataset
* Size: 522 training samples
* Columns: sentence_0 and sentence_1
* Approximate statistics based on the first 522 samples:
| | sentence_0 | sentence_1 |
|:--------|:-----------------------------------------------------------------------------------|:-------------------------------------------------------------------------------------|
| type | string | string |
| details | - min: 11 tokens
- mean: 19.05 tokens
- max: 35 tokens
| - min: 10 tokens
- mean: 116.38 tokens
- max: 161 tokens
|
* Samples:
| sentence_0 | sentence_1 |
|:-------------------------------------------------------------------------------------------------------------------------------------------------|:-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| What is the purpose of the AI Bill of Rights mentioned in the context? | BLUEPRINT FOR AN
AI B ILL OF
RIGHTS
MAKING AUTOMATED
SYSTEMS WORK FOR
THE AMERICAN PEOPLE
OCTOBER 2022 |
| When was the Blueprint for an AI Bill of Rights published? | BLUEPRINT FOR AN
AI B ILL OF
RIGHTS
MAKING AUTOMATED
SYSTEMS WORK FOR
THE AMERICAN PEOPLE
OCTOBER 2022 |
| What is the purpose of the Blueprint for an AI Bill of Rights published by the White House Office of Science and Technology Policy? | About this Document
The Blueprint for an AI Bill of Rights: Making Automated Systems Work for the American People was
published by the White House Office of Science and Technology Policy in October 2022. This framework was
released one year after OSTP announced the launch of a process to develop “a bill of rights for an AI-powered
world.” Its release follows a year of public engagement to inform this initiative. The framework is available
online at: https://www.whitehouse.gov/ostp/ai-bill-of-rights
About the Office of Science and Technology Policy
The Office of Science and Technology Policy (OSTP) was established by the National Science and Technology |
* Loss: [MatryoshkaLoss](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#matryoshkaloss) with these parameters:
```json
{
"loss": "MultipleNegativesRankingLoss",
"matryoshka_dims": [
768,
512,
256,
128,
64
],
"matryoshka_weights": [
1,
1,
1,
1,
1
],
"n_dims_per_step": -1
}
```
### Training Hyperparameters
#### Non-Default Hyperparameters
- `eval_strategy`: steps
- `per_device_train_batch_size`: 20
- `per_device_eval_batch_size`: 20
- `num_train_epochs`: 5
- `multi_dataset_batch_sampler`: round_robin
#### All Hyperparameters
Click to expand
- `overwrite_output_dir`: False
- `do_predict`: False
- `eval_strategy`: steps
- `prediction_loss_only`: True
- `per_device_train_batch_size`: 20
- `per_device_eval_batch_size`: 20
- `per_gpu_train_batch_size`: None
- `per_gpu_eval_batch_size`: None
- `gradient_accumulation_steps`: 1
- `eval_accumulation_steps`: None
- `torch_empty_cache_steps`: None
- `learning_rate`: 5e-05
- `weight_decay`: 0.0
- `adam_beta1`: 0.9
- `adam_beta2`: 0.999
- `adam_epsilon`: 1e-08
- `max_grad_norm`: 1
- `num_train_epochs`: 5
- `max_steps`: -1
- `lr_scheduler_type`: linear
- `lr_scheduler_kwargs`: {}
- `warmup_ratio`: 0.0
- `warmup_steps`: 0
- `log_level`: passive
- `log_level_replica`: warning
- `log_on_each_node`: True
- `logging_nan_inf_filter`: True
- `save_safetensors`: True
- `save_on_each_node`: False
- `save_only_model`: False
- `restore_callback_states_from_checkpoint`: False
- `no_cuda`: False
- `use_cpu`: False
- `use_mps_device`: False
- `seed`: 42
- `data_seed`: None
- `jit_mode_eval`: False
- `use_ipex`: False
- `bf16`: False
- `fp16`: False
- `fp16_opt_level`: O1
- `half_precision_backend`: auto
- `bf16_full_eval`: False
- `fp16_full_eval`: False
- `tf32`: None
- `local_rank`: 0
- `ddp_backend`: None
- `tpu_num_cores`: None
- `tpu_metrics_debug`: False
- `debug`: []
- `dataloader_drop_last`: False
- `dataloader_num_workers`: 0
- `dataloader_prefetch_factor`: None
- `past_index`: -1
- `disable_tqdm`: False
- `remove_unused_columns`: True
- `label_names`: None
- `load_best_model_at_end`: False
- `ignore_data_skip`: False
- `fsdp`: []
- `fsdp_min_num_params`: 0
- `fsdp_config`: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
- `fsdp_transformer_layer_cls_to_wrap`: None
- `accelerator_config`: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
- `deepspeed`: None
- `label_smoothing_factor`: 0.0
- `optim`: adamw_torch
- `optim_args`: None
- `adafactor`: False
- `group_by_length`: False
- `length_column_name`: length
- `ddp_find_unused_parameters`: None
- `ddp_bucket_cap_mb`: None
- `ddp_broadcast_buffers`: False
- `dataloader_pin_memory`: True
- `dataloader_persistent_workers`: False
- `skip_memory_metrics`: True
- `use_legacy_prediction_loop`: False
- `push_to_hub`: False
- `resume_from_checkpoint`: None
- `hub_model_id`: None
- `hub_strategy`: every_save
- `hub_private_repo`: False
- `hub_always_push`: False
- `gradient_checkpointing`: False
- `gradient_checkpointing_kwargs`: None
- `include_inputs_for_metrics`: False
- `eval_do_concat_batches`: True
- `fp16_backend`: auto
- `push_to_hub_model_id`: None
- `push_to_hub_organization`: None
- `mp_parameters`:
- `auto_find_batch_size`: False
- `full_determinism`: False
- `torchdynamo`: None
- `ray_scope`: last
- `ddp_timeout`: 1800
- `torch_compile`: False
- `torch_compile_backend`: None
- `torch_compile_mode`: None
- `dispatch_batches`: None
- `split_batches`: None
- `include_tokens_per_second`: False
- `include_num_input_tokens_seen`: False
- `neftune_noise_alpha`: None
- `optim_target_modules`: None
- `batch_eval_metrics`: False
- `eval_on_start`: False
- `eval_use_gather_object`: False
- `batch_sampler`: batch_sampler
- `multi_dataset_batch_sampler`: round_robin