Update README.md

README.md

@@ -1,130 +1,130 @@
----
-base_model: baai/bge-base-en-v1.5
-language:
-- en
-library_name: model2vec
-license: mit
-model_name: brown-fairy-base-v0
-tags:
-- embeddings
-- static-embeddings
-- sentence-transformers
----
-# 🪲 brown-fairy-base-v0 Model Card
-
-<div align="center">
-  <img width="50%" alt="Fairy logo" src="./assets/fairy_logo.png">
-</div>
-
-> [!TIP]
-> Fairies are among the most enchanting and magical beings in folklore and mythology. They appear across countless cultures and stories, from ancient forests to modern gardens. They are celebrated for their ability to bridge the mundane and magical realms, known for their ethereal grace and transformative powers. Fairies are tiny, higher-dimensional beings that can interact with the world in ways that are beyond our understanding.
-
-The fairy series of models are an attempt to tune the beetle series of models to be more suitable for downstream tasks. These models are meant to fully open experiments at making state-of-the-art static embeddings. 
-
-The brown-fairy-base-v0 model is a distillation of the `baai/bge-base-en-v1.5` model into the `brown-beetle-base-v0` model. There was no PCA or Zipf applied to this model.
-
-## Installation
-
-Install model2vec using pip:
-
-```bash
-pip install model2vec
-```
-
-## Usage
-
-Load this model using the `from_pretrained` method:
-
-```python
-from model2vec import StaticModel
-
-# Load a pretrained Model2Vec model
-model = StaticModel.from_pretrained("bhavnicksm/brown-fairy-base-v0")
-
-# Compute text embeddings
-embeddings = model.encode(["Example sentence"])
-```
-
-Read more about the Model2Vec library [here](https://github.com/MinishLab/model2vec).
-
-## Reproduce this model
-
-This model was trained on a subset of the 2 Million texts from the [FineWeb-Edu](https://huggingface.co/datasets/mixedbread-ai/fineweb-edu) dataset, which was labeled by the `baai/bge-base-en-v1.5` model.
-
-<details>
-<summary>Training Code</summary>
-
-Note: The datasets need to me made seperately and loaded with the `datasets` library. 
-
-```python
-static_embedding = StaticEmbedding.from_model2vec("bhavnicksm/brown-beetle-base-v0")
-model = SentenceTransformer(
-    modules=[static_embedding]
-)
-
-loss = MSELoss(model)
-
-run_name = "brown-fairy-base-v0"
-args = SentenceTransformerTrainingArguments(
-    # Required parameter:
-    output_dir=f"output/{run_name}",
-    # Optional training parameters:
-    num_train_epochs=1,
-    per_device_train_batch_size=2048,
-    per_device_eval_batch_size=2048,
-    learning_rate=1e-1,
-    warmup_ratio=0.1,
-    fp16=False,  # Set to False if you get an error that your GPU can't run on FP16
-    bf16=True,  # Set to True if you have a GPU that supports BF16
-    batch_sampler=BatchSamplers.NO_DUPLICATES,  
-    # Optional tracking/debugging parameters:
-    eval_strategy="steps",
-    eval_steps=50,
-    save_strategy="steps",
-    save_steps=50,
-    save_total_limit=5,
-    logging_steps=50,
-    logging_first_step=True,
-    run_name=run_name, 
-)
-
-evaluator = NanoBEIREvaluator()
-evaluator(model)
-
-trainer = SentenceTransformerTrainer(
-    model=model,
-    args=args,
-    train_dataset=train_dataset,
-    eval_dataset=eval_dataset,
-    loss=loss,
-    evaluator=evaluator,
-)
-trainer.train()
-
-evaluator(model)
-
-model.save_pretrained(f"output/{run_name}")
-```
-
-</details>
-
-## Comparison with other models
-
-Coming soon...
-
-## Acknowledgements
-
-This model is based on the [Model2Vec](https://github.com/MinishLab/model2vec) library. Credit goes to the [Minish Lab](https://github.com/MinishLab) team for developing this library.
-
-## Citation
-
-This model builds on work done by Minish Lab. Please cite the [Model2Vec repository](https://github.com/MinishLab/model2vec) if you use this model in your work.
-
-```bibtex
-@software{minishlab2024model2vec,
-  authors = {Stephan Tulkens, Thomas van Dongen},
-  title = {Model2Vec: Turn any Sentence Transformer into a Small Fast Model},
-  year = {2024},
-  url = {https://github.com/MinishLab/model2vec},
-}
-```
+---
+base_model: baai/bge-base-en-v1.5
+language:
+- en
+library_name: model2vec
+license: mit
+model_name: brown-fairy-base-v0
+tags:
+- embeddings
+- static-embeddings
+- sentence-transformers
+---
+# 🧚🏻‍♀️ brown-fairy-base-v0 Model Card
+
+<div align="center">
+  <img width="50%" alt="Fairy logo" src="./assets/fairy_logo.png">
+</div>
+
+> [!TIP]
+> Fairies are among the most enchanting and magical beings in folklore and mythology. They appear across countless cultures and stories, from ancient forests to modern gardens. They are celebrated for their ability to bridge the mundane and magical realms, known for their ethereal grace and transformative powers. Fairies are tiny, higher-dimensional beings that can interact with the world in ways that are beyond our understanding.
+
+The fairy series of models are an attempt to tune the beetle series of models to be more suitable for downstream tasks. These models are meant to fully open experiments at making state-of-the-art static embeddings. 
+
+The brown-fairy-base-v0 model is a distillation of the `baai/bge-base-en-v1.5` model into the `brown-beetle-base-v0` model. There was no PCA or Zipf applied to this model.
+
+## Installation
+
+Install model2vec using pip:
+
+```bash
+pip install model2vec
+```
+
+## Usage
+
+Load this model using the `from_pretrained` method:
+
+```python
+from model2vec import StaticModel
+
+# Load a pretrained Model2Vec model
+model = StaticModel.from_pretrained("bhavnicksm/brown-fairy-base-v0")
+
+# Compute text embeddings
+embeddings = model.encode(["Example sentence"])
+```
+
+Read more about the Model2Vec library [here](https://github.com/MinishLab/model2vec).
+
+## Reproduce this model
+
+This model was trained on a subset of the 2 Million texts from the [FineWeb-Edu](https://huggingface.co/datasets/mixedbread-ai/fineweb-edu) dataset, which was labeled by the `baai/bge-base-en-v1.5` model.
+
+<details>
+<summary>Training Code</summary>
+
+Note: The datasets need to me made seperately and loaded with the `datasets` library. 
+
+```python
+static_embedding = StaticEmbedding.from_model2vec("bhavnicksm/brown-beetle-base-v0")
+model = SentenceTransformer(
+    modules=[static_embedding]
+)
+
+loss = MSELoss(model)
+
+run_name = "brown-fairy-base-v0"
+args = SentenceTransformerTrainingArguments(
+    # Required parameter:
+    output_dir=f"output/{run_name}",
+    # Optional training parameters:
+    num_train_epochs=1,
+    per_device_train_batch_size=2048,
+    per_device_eval_batch_size=2048,
+    learning_rate=1e-1,
+    warmup_ratio=0.1,
+    fp16=False,  # Set to False if you get an error that your GPU can't run on FP16
+    bf16=True,  # Set to True if you have a GPU that supports BF16
+    batch_sampler=BatchSamplers.NO_DUPLICATES,  
+    # Optional tracking/debugging parameters:
+    eval_strategy="steps",
+    eval_steps=50,
+    save_strategy="steps",
+    save_steps=50,
+    save_total_limit=5,
+    logging_steps=50,
+    logging_first_step=True,
+    run_name=run_name, 
+)
+
+evaluator = NanoBEIREvaluator()
+evaluator(model)
+
+trainer = SentenceTransformerTrainer(
+    model=model,
+    args=args,
+    train_dataset=train_dataset,
+    eval_dataset=eval_dataset,
+    loss=loss,
+    evaluator=evaluator,
+)
+trainer.train()
+
+evaluator(model)
+
+model.save_pretrained(f"output/{run_name}")
+```
+
+</details>
+
+## Comparison with other models
+
+Coming soon...
+
+## Acknowledgements
+
+This model is based on the [Model2Vec](https://github.com/MinishLab/model2vec) library. Credit goes to the [Minish Lab](https://github.com/MinishLab) team for developing this library.
+
+## Citation
+
+This model builds on work done by Minish Lab. Please cite the [Model2Vec repository](https://github.com/MinishLab/model2vec) if you use this model in your work.
+
+```bibtex
+@software{minishlab2024model2vec,
+  authors = {Stephan Tulkens, Thomas van Dongen},
+  title = {Model2Vec: Turn any Sentence Transformer into a Small Fast Model},
+  year = {2024},
+  url = {https://github.com/MinishLab/model2vec},
+}
+```