Add SetFit ABSA model

README.md

@@ -13,24 +13,26 @@ datasets:
 metrics:
 - accuracy
 widget:
-- text: people:Regardless of whether there are two people or two hundred people ahead
-    of you the hostess will take your name and tell you Five minutes.
-- text: dish:This dish is my favorite and I always get it when I go there and never
-    get tired of it.
-- text: food:Get your food to go, find a bench, and kick back with a plate of dumplings.
-- text: crabmeat lasagna:You must have the crabmeat lasagna which is out of this world
-    and the chocolate bread pudding for dessert.
-- text: plate:Get your food to go, find a bench, and kick back with a plate of dumplings.
+- text: bottles of wine:bottles of wine are cheap and good.
+- text: world:I also ordered the Change Mojito, which was out of this world.
+- text: bar:We were still sitting at the bar while we drank the sangria, but facing
+    away from the bar when we turned back around, the $2 was gone the people next
+    to us said the bartender took it.
+- text: word:word of advice, save room for pasta dishes and never leave until you've
+    had the tiramisu.
+- text: bartender:We were still sitting at the bar while we drank the sangria, but
+    facing away from the bar when we turned back around, the $2 was gone the people
+    next to us said the bartender took it.
 pipeline_tag: text-classification
 inference: false
 co2_eq_emissions:
-  emissions: 12.371061343498498
+  emissions: 18.322516829847984
   source: codecarbon
   training_type: fine-tuning
   on_cloud: false
   cpu_model: 13th Gen Intel(R) Core(TM) i7-13700K
   ram_total_size: 31.777088165283203
-  hours_used: 0.206
+  hours_used: 0.303
   hardware_used: 1 x NVIDIA GeForce RTX 3090
 base_model: BAAI/bge-small-en-v1.5
 model-index:
@@ -45,7 +47,7 @@ model-index:
       split: test
     metrics:
     - type: accuracy
-      value: 0.7871243108660857
+      value: 0.8623188405797102
       name: Accuracy
 ---
 
@@ -70,6 +72,7 @@ This model was trained within the context of a larger system for ABSA, which loo
 - **Model Type:** SetFit
 - **Sentence Transformer body:** [BAAI/bge-small-en-v1.5](https://huggingface.co/BAAI/bge-small-en-v1.5)
 - **Classification head:** a [LogisticRegression](https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LogisticRegression.html) instance
+- **spaCy Model:** en_core_web_lg
 - **SetFitABSA Aspect Model:** [tomaarsen/setfit-absa-bge-small-en-v1.5-restaurants-aspect](https://huggingface.co/tomaarsen/setfit-absa-bge-small-en-v1.5-restaurants-aspect)
 - **SetFitABSA Polarity Model:** [tomaarsen/setfit-absa-bge-small-en-v1.5-restaurants-polarity](https://huggingface.co/tomaarsen/setfit-absa-bge-small-en-v1.5-restaurants-polarity)
 - **Maximum Sequence Length:** 512 tokens
@@ -95,7 +98,7 @@ This model was trained within the context of a larger system for ABSA, which loo
 ### Metrics
 | Label   | Accuracy |
 |:--------|:---------|
-| **all** | 0.7871   |
+| **all** | 0.8623   |
 
 ## Uses
 
@@ -150,12 +153,12 @@ preds = model("The food was great, but the venue is just way too busy.")
 ### Training Set Metrics
 | Training set | Min | Median  | Max |
 |:-------------|:----|:--------|:----|
-| Word count   | 4   | 19.3034 | 45  |
+| Word count   | 4   | 19.3576 | 45  |
 
 | Label     | Training Sample Count |
 |:----------|:----------------------|
-| no aspect | 231                   |
-| aspect    | 204                   |
+| no aspect | 170                   |
+| aspect    | 255                   |
 
 ### Training Hyperparameters
 - batch_size: (256, 256)
@@ -171,34 +174,43 @@ preds = model("The food was great, but the venue is just way too busy.")
 - use_amp: True
 - warmup_proportion: 0.1
 - seed: 42
+- eval_max_steps: -1
 - load_best_model_at_end: True
 
 ### Training Results
 | Epoch      | Step    | Training Loss | Validation Loss |
 |:----------:|:-------:|:-------------:|:---------------:|
-| 0.0027     | 1       | 0.2574        | -               |
-| 0.1340     | 50      | 0.2561        | -               |
-| 0.2681     | 100     | 0.251         | 0.2543          |
-| 0.4021     | 150     | 0.2451        | -               |
-| 0.5362     | 200     | 0.242         | 0.2506          |
-| 0.6702     | 250     | 0.2239        | -               |
-| **0.8043** | **300** | **0.0473**    | **0.2499**      |
-| 0.9383     | 350     | 0.0098        | -               |
-| 1.0724     | 400     | 0.0097        | 0.2734          |
-| 1.2064     | 450     | 0.0047        | -               |
-| 1.3405     | 500     | 0.0071        | 0.2834          |
-| 1.4745     | 550     | 0.0089        | -               |
-| 1.6086     | 600     | 0.005         | 0.273           |
-| 1.7426     | 650     | 0.0041        | -               |
-| 1.8767     | 700     | 0.0042        | 0.2942          |
-| 2.0107     | 750     | 0.0053        | -               |
-| 2.1448     | 800     | 0.0073        | 0.2898          |
+| 0.0027     | 1       | 0.2498        | -               |
+| 0.1355     | 50      | 0.2442        | -               |
+| 0.2710     | 100     | 0.2462        | 0.2496          |
+| 0.4065     | 150     | 0.2282        | -               |
+| 0.5420     | 200     | 0.0752        | 0.1686          |
+| 0.6775     | 250     | 0.0124        | -               |
+| 0.8130     | 300     | 0.0128        | 0.1884          |
+| 0.9485     | 350     | 0.0062        | -               |
+| 1.0840     | 400     | 0.0012        | 0.183           |
+| 1.2195     | 450     | 0.0009        | -               |
+| 1.3550     | 500     | 0.0008        | 0.2072          |
+| 1.4905     | 550     | 0.0031        | -               |
+| 1.6260     | 600     | 0.0006        | 0.1716          |
+| 1.7615     | 650     | 0.0005        | -               |
+| **1.8970** | **700** | **0.0005**    | **0.1666**      |
+| 2.0325     | 750     | 0.0005        | -               |
+| 2.1680     | 800     | 0.0004        | 0.2086          |
+| 2.3035     | 850     | 0.0005        | -               |
+| 2.4390     | 900     | 0.0004        | 0.183           |
+| 2.5745     | 950     | 0.0004        | -               |
+| 2.7100     | 1000    | 0.0036        | 0.1725          |
+| 2.8455     | 1050    | 0.0004        | -               |
+| 2.9810     | 1100    | 0.0003        | 0.1816          |
+| 3.1165     | 1150    | 0.0004        | -               |
+| 3.2520     | 1200    | 0.0003        | 0.1802          |
 
 * The bold row denotes the saved checkpoint.
 ### Environmental Impact
 Carbon emissions were measured using [CodeCarbon](https://github.com/mlco2/codecarbon).
-- **Carbon Emitted**: 0.012 kg of CO2
-- **Hours Used**: 0.206 hours
+- **Carbon Emitted**: 0.018 kg of CO2
+- **Hours Used**: 0.303 hours
 
 ### Training Hardware
 - **On Cloud**: No
@@ -210,6 +222,7 @@ Carbon emissions were measured using [CodeCarbon](https://github.com/mlco2/codec
 - Python: 3.9.16
 - SetFit: 1.0.0.dev0
 - Sentence Transformers: 2.2.2
+- spaCy: 3.7.2
 - Transformers: 4.29.0
 - PyTorch: 1.13.1+cu117
 - Datasets: 2.15.0

config.json

@@ -1,5 +1,5 @@
 {
-  "_name_or_path": "models\\step_300\\",
+  "_name_or_path": "models\\step_700\\",
   "architectures": [
     "BertModel"
   ],

config_setfit.json

@@ -1,4 +1,5 @@
 {
+  "spacy_model": "en_core_web_lg",
   "normalize_embeddings": false,
   "labels": [
     "no aspect",

model_head.pkl

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:40981c12f3dc9655afbabd43a518ca9aaeb02bca44eb5812e3d98e8f04b90761
+oid sha256:410891858f59f504ec87489b123ebaef75277ab06357a08cdab676c7f0e0a4c4
 size 3919

pytorch_model.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:053fcaf18552044f9fab2b6c2eccfceff2b5c353804ac31e4688befd443f7be5
+oid sha256:1822a3ac45126bf5d760c1302f760b0b71999da32a45d06858acc5317b6d3c15
 size 133511213