Add SetFit model

README.md

@@ -8,11 +8,13 @@ tags:
 metrics:
 - accuracy
 widget:
-- text: What is the capital of France?
-- text: What do you call a group of lions?
-- text: Who wrote 'To Kill a Mockingbird'?
-- text: What is the largest ocean on Earth?
-- text: How many sides does a square have?
+- text: If you have 20 marbles and you give 5 of them to your friend, how many marbles
+    do you have left?
+- text: If a dozen eggs cost $3, how much do 3 dozen eggs cost?
+- text: If a train leaves the station at 9:00 AM and arrives at its destination at
+    11:30 AM, how long is the journey?
+- text: What is the chemical symbol for water?
+- text: Who painted the Mona Lisa?
 pipeline_tag: text-classification
 inference: true
 base_model: sentence-transformers/all-MiniLM-L6-v2
@@ -60,10 +62,10 @@ The model has been trained using an efficient few-shot learning technique that i
 - **Blogpost:** [SetFit: Efficient Few-Shot Learning Without Prompts](https://huggingface.co/blog/setfit)
 
 ### Model Labels
-| Label | Examples                                                                                                                                                                                                                                                                                       |
-|:------|:-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
-| 0     | <ul><li>'What is the tallest animal in the world?'</li><li>'What is the main ingredient in guacamole?'</li><li>'What do you call a baby kangaroo?'</li></ul>                                                                                                                                   |
-| 1     | <ul><li>'What is the sum of 7 and 5?'</li><li>'If there are 3 pencils in each pack and you have 7 packs, how many pencils do you have altogether?'</li><li>'If a pencil is 15 centimeters long and another pencil is 10 centimeters long, what is the total length of both pencils?'</li></ul> |
+| Label    | Examples                                                                                                                                                                                                     |
+|:---------|:-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+| non_math | <ul><li>'What do you call a baby kangaroo?'</li><li>'What is the currency used in Japan?'</li><li>'What is the smallest planet in our solar system?'</li></ul>                                               |
+| math     | <ul><li>'What is the number that comes after 15?'</li><li>'What is the sum of 18 and 27?'</li><li>'There are 8 apples on the table. If you add 3 more apples, how many apples are there in total?'</li></ul> |
 
 ## Evaluation
 
@@ -90,7 +92,7 @@ from setfit import SetFitModel
 # Download from the 🤗 Hub
 model = SetFitModel.from_pretrained("serdarcaglar/primary-school-math-question")
 # Run inference
-preds = model("What is the capital of France?")
+preds = model("Who painted the Mona Lisa?")
 ```
 
 <!--
@@ -122,12 +124,12 @@ preds = model("What is the capital of France?")
 ### Training Set Metrics
 | Training set | Min | Median  | Max |
 |:-------------|:----|:--------|:----|
-| Word count   | 3   | 10.9388 | 23  |
+| Word count   | 3   | 10.8406 | 23  |
 
-| Label | Training Sample Count |
-|:------|:----------------------|
-| 0     | 29                    |
-| 1     | 20                    |
+| Label    | Training Sample Count |
+|:---------|:----------------------|
+| math     | 40                    |
+| non_math | 29                    |
 
 ### Training Hyperparameters
 - batch_size: (16, 16)
@@ -150,9 +152,10 @@ preds = model("What is the capital of France?")
 ### Training Results
 | Epoch  | Step | Training Loss | Validation Loss |
 |:------:|:----:|:-------------:|:---------------:|
-| 0.0081 | 1    | 0.2707        | -               |
-| 0.4065 | 50   | 0.0183        | -               |
-| 0.8130 | 100  | 0.0032        | -               |
+| 0.0058 | 1    | 0.2737        | -               |
+| 0.2890 | 50   | 0.0212        | -               |
+| 0.5780 | 100  | 0.0037        | -               |
+| 0.8671 | 150  | 0.0023        | -               |
 
 ### Framework Versions
 - Python: 3.10.12

config_setfit.json

@@ -1,4 +1,7 @@
 {
-  "normalize_embeddings": false,
-  "labels": null
+  "labels": [
+    "math",
+    "non_math"
+  ],
+  "normalize_embeddings": false
 }

model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:1f05693d3ce3b6d406824e39bfc738827982da4795d791035c208187cefc827b
+oid sha256:b23a7b11036a5a3ed6d7fa78520930502934b3c0fdf69c69a7dae06e24a31106
 size 90864192

model_head.pkl

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:2c0f268519c4bebe519f6cdf0b832121d8059e6baf32a395860ef0e9b7e0f4da
-size 3935
+oid sha256:d0cae6363f78c05e9e530147c0a21ae4a56abfac59b6f4fb1578ee8daf34680e
+size 3967