Hugging Face's logo

tomaarsen
/
mpnet-base-allnli

Sentence Similarity
sentence-transformers
TensorBoard
Safetensors
English
mpnet
feature-extraction
generated_with_trainer
dataset_size:100K<n<1M
loss:SoftmaxLoss
loss:CosineSimilarityLoss
Eval Results
Carbon Emissions
Inference Endpoints
Model card Files Metrics Community
tomaarsen HF staff commited on
Commit
5fa33a7
•
1 Parent(s): 919982b

Model save

Browse files
Files changed (3) hide show
  1. README.md +74 -74
  2. emissions.csv +2 -1
  3. runs/Jun05_14-24-25_Tom/events.out.tfevents.1717590271.Tom.22032.0 +2 -2
README.md CHANGED
@@ -63,14 +63,14 @@ widget:
63
  - I don't know how cold it got last night.
64
  pipeline_tag: sentence-similarity
65
  co2_eq_emissions:
66
- emissions: 6.543912203095872
67
- energy_consumed: 0.01683529336894555
68
  source: codecarbon
69
  training_type: fine-tuning
70
  on_cloud: false
71
  cpu_model: 13th Gen Intel(R) Core(TM) i7-13700K
72
  ram_total_size: 31.777088165283203
73
- hours_used: 0.067
74
  hardware_used: 1 x NVIDIA GeForce RTX 3090
75
  model-index:
76
  - name: SentenceTransformer based on microsoft/mpnet-base
@@ -83,34 +83,34 @@ model-index:
83
  type: sts-dev
84
  metrics:
85
  - type: pearson_cosine
86
- value: 0.8625771940364872
87
  name: Pearson Cosine
88
  - type: spearman_cosine
89
- value: 0.8606717551154308
90
  name: Spearman Cosine
91
  - type: pearson_manhattan
92
- value: 0.8638967614504363
93
  name: Pearson Manhattan
94
  - type: spearman_manhattan
95
- value: 0.8633946128639698
96
  name: Spearman Manhattan
97
  - type: pearson_euclidean
98
- value: 0.8611337271100419
99
  name: Pearson Euclidean
100
  - type: spearman_euclidean
101
- value: 0.8606717551154308
102
  name: Spearman Euclidean
103
  - type: pearson_dot
104
- value: 0.862577202108671
105
  name: Pearson Dot
106
  - type: spearman_dot
107
- value: 0.8606717551154308
108
  name: Spearman Dot
109
  - type: pearson_max
110
- value: 0.8638967614504363
111
  name: Pearson Max
112
  - type: spearman_max
113
- value: 0.8633946128639698
114
  name: Spearman Max
115
  - task:
116
  type: semantic-similarity
@@ -120,34 +120,34 @@ model-index:
120
  type: sts-test
121
  metrics:
122
  - type: pearson_cosine
123
- value: 0.8121966861722953
124
  name: Pearson Cosine
125
  - type: spearman_cosine
126
- value: 0.8064524624275264
127
  name: Spearman Cosine
128
  - type: pearson_manhattan
129
- value: 0.8164566762295066
130
  name: Pearson Manhattan
131
  - type: spearman_manhattan
132
- value: 0.8087376581901532
133
  name: Spearman Manhattan
134
  - type: pearson_euclidean
135
- value: 0.8146700964672056
136
  name: Pearson Euclidean
137
  - type: spearman_euclidean
138
- value: 0.8064524624275264
139
  name: Spearman Euclidean
140
  - type: pearson_dot
141
- value: 0.8121966895185604
142
  name: Pearson Dot
143
  - type: spearman_dot
144
- value: 0.8064524624275264
145
  name: Spearman Dot
146
  - type: pearson_max
147
- value: 0.8164566762295066
148
  name: Pearson Max
149
  - type: spearman_max
150
- value: 0.8087376581901532
151
  name: Spearman Max
152
  ---
153
 
@@ -252,16 +252,16 @@ You can finetune this model on your own dataset.
252
 
253
  | Metric | Value |
254
  |:-------------------|:-----------|
255
- | pearson_cosine | 0.8626 |
256
- | spearman_cosine | 0.8607 |
257
- | pearson_manhattan | 0.8639 |
258
- | spearman_manhattan | 0.8634 |
259
- | pearson_euclidean | 0.8611 |
260
- | spearman_euclidean | 0.8607 |
261
- | pearson_dot | 0.8626 |
262
- | **spearman_dot** | **0.8607** |
263
- | pearson_max | 0.8639 |
264
- | spearman_max | 0.8634 |
265
 
266
  #### Semantic Similarity
267
  * Dataset: `sts-test`
@@ -269,16 +269,16 @@ You can finetune this model on your own dataset.
269
 
270
  | Metric | Value |
271
  |:--------------------|:-----------|
272
- | pearson_cosine | 0.8122 |
273
- | **spearman_cosine** | **0.8065** |
274
- | pearson_manhattan | 0.8165 |
275
- | spearman_manhattan | 0.8087 |
276
- | pearson_euclidean | 0.8147 |
277
- | spearman_euclidean | 0.8065 |
278
- | pearson_dot | 0.8122 |
279
- | spearman_dot | 0.8065 |
280
- | pearson_max | 0.8165 |
281
- | spearman_max | 0.8087 |
282
 
283
  <!--
284
  ## Bias, Risks and Limitations
@@ -548,43 +548,43 @@ You can finetune this model on your own dataset.
548
  </details>
549
 
550
  ### Training Logs
551
- | Epoch | Step | Training Loss | stsb loss | snli loss | multi nli loss | sts-dev_spearman_dot | sts-test_spearman_cosine |
552
- |:----------:|:-------:|:-------------:|:----------:|:----------:|:--------------:|:--------------------:|:------------------------:|
553
- | 0.0370 | 10 | 0.8336 | - | - | - | - | - |
554
- | 0.0741 | 20 | 0.8257 | - | - | - | - | - |
555
- | 0.1111 | 30 | 0.6998 | 0.0736 | 1.0978 | 1.0961 | 0.6791 | - |
556
- | 0.1481 | 40 | 0.7878 | - | - | - | - | - |
557
- | 0.1852 | 50 | 0.7868 | - | - | - | - | - |
558
- | 0.2222 | 60 | 0.6761 | 0.0528 | 1.0958 | 1.0963 | 0.8035 | - |
559
- | 0.2593 | 70 | 0.7804 | - | - | - | - | - |
560
- | 0.2963 | 80 | 0.7789 | - | - | - | - | - |
561
- | 0.3333 | 90 | 0.6756 | 0.0390 | 1.0940 | 1.0962 | 0.8341 | - |
562
- | 0.3704 | 100 | 0.7811 | - | - | - | - | - |
563
- | 0.4074 | 110 | 0.775 | - | - | - | - | - |
564
- | 0.4444 | 120 | 0.6721 | 0.0351 | 1.0932 | 1.0981 | 0.8413 | - |
565
- | 0.4815 | 130 | 0.7794 | - | - | - | - | - |
566
- | 0.5185 | 140 | 0.7764 | - | - | - | - | - |
567
- | 0.5556 | 150 | 0.6705 | 0.0343 | 1.0906 | 1.0950 | 0.8485 | - |
568
- | 0.5926 | 160 | 0.776 | - | - | - | - | - |
569
- | 0.6296 | 170 | 0.7742 | - | - | - | - | - |
570
- | 0.6667 | 180 | 0.6643 | 0.0326 | 1.0887 | 1.0927 | 0.8547 | - |
571
- | 0.7037 | 190 | 0.7732 | - | - | - | - | - |
572
- | 0.7407 | 200 | 0.7733 | - | - | - | - | - |
573
- | 0.7778 | 210 | 0.6676 | 0.0318 | 1.0867 | 1.0912 | 0.8591 | - |
574
- | 0.8148 | 220 | 0.7706 | - | - | - | - | - |
575
- | 0.8519 | 230 | 0.7716 | - | - | - | - | - |
576
- | **0.8889** | **240** | **0.6633** | **0.0302** | **1.0855** | **1.0889** | **0.8607** | **-** |
577
- | 0.9259 | 250 | 0.7711 | - | - | - | - | - |
578
- | 0.9630 | 260 | 0.7716 | - | - | - | - | - |
579
- | 1.0 | 270 | 0.6644 | 0.0316 | 1.0852 | 1.0890 | 0.8607 | 0.8065 |
580
 
581
  * The bold row denotes the saved checkpoint.
582
 
583
  ### Environmental Impact
584
  Carbon emissions were measured using [CodeCarbon](https://github.com/mlco2/codecarbon).
585
- - **Energy Consumed**: 0.017 kWh
586
  - **Carbon Emitted**: 0.007 kg of CO2
587
- - **Hours Used**: 0.067 hours
588
 
589
  ### Training Hardware
590
  - **On Cloud**: No
 
63
  - I don't know how cold it got last night.
64
  pipeline_tag: sentence-similarity
65
  co2_eq_emissions:
66
+ emissions: 6.863209894681815
67
+ energy_consumed: 0.017656739339344318
68
  source: codecarbon
69
  training_type: fine-tuning
70
  on_cloud: false
71
  cpu_model: 13th Gen Intel(R) Core(TM) i7-13700K
72
  ram_total_size: 31.777088165283203
73
+ hours_used: 0.068
74
  hardware_used: 1 x NVIDIA GeForce RTX 3090
75
  model-index:
76
  - name: SentenceTransformer based on microsoft/mpnet-base
 
83
  type: sts-dev
84
  metrics:
85
  - type: pearson_cosine
86
+ value: 0.8344104750902503
87
  name: Pearson Cosine
88
  - type: spearman_cosine
89
+ value: 0.8294923795333993
90
  name: Spearman Cosine
91
  - type: pearson_manhattan
92
+ value: 0.8316959259914674
93
  name: Pearson Manhattan
94
  - type: spearman_manhattan
95
+ value: 0.8331844817222047
96
  name: Spearman Manhattan
97
  - type: pearson_euclidean
98
+ value: 0.8272941934077804
99
  name: Pearson Euclidean
100
  - type: spearman_euclidean
101
+ value: 0.8294923795333993
102
  name: Spearman Euclidean
103
  - type: pearson_dot
104
+ value: 0.8344104825648291
105
  name: Pearson Dot
106
  - type: spearman_dot
107
+ value: 0.8294923795333993
108
  name: Spearman Dot
109
  - type: pearson_max
110
+ value: 0.8344104825648291
111
  name: Pearson Max
112
  - type: spearman_max
113
+ value: 0.8331844817222047
114
  name: Spearman Max
115
  - task:
116
  type: semantic-similarity
 
120
  type: sts-test
121
  metrics:
122
  - type: pearson_cosine
123
+ value: 0.7776062173443514
124
  name: Pearson Cosine
125
  - type: spearman_cosine
126
+ value: 0.7642518713703523
127
  name: Spearman Cosine
128
  - type: pearson_manhattan
129
+ value: 0.7788269653910183
130
  name: Pearson Manhattan
131
  - type: spearman_manhattan
132
+ value: 0.7659203139768728
133
  name: Spearman Manhattan
134
  - type: pearson_euclidean
135
+ value: 0.7763456809736229
136
  name: Pearson Euclidean
137
  - type: spearman_euclidean
138
+ value: 0.7642518713703523
139
  name: Spearman Euclidean
140
  - type: pearson_dot
141
+ value: 0.7776062158976489
142
  name: Pearson Dot
143
  - type: spearman_dot
144
+ value: 0.7642518713703523
145
  name: Spearman Dot
146
  - type: pearson_max
147
+ value: 0.7788269653910183
148
  name: Pearson Max
149
  - type: spearman_max
150
+ value: 0.7659203139768728
151
  name: Spearman Max
152
  ---
153
 
 
252
 
253
  | Metric | Value |
254
  |:-------------------|:-----------|
255
+ | pearson_cosine | 0.8344 |
256
+ | spearman_cosine | 0.8295 |
257
+ | pearson_manhattan | 0.8317 |
258
+ | spearman_manhattan | 0.8332 |
259
+ | pearson_euclidean | 0.8273 |
260
+ | spearman_euclidean | 0.8295 |
261
+ | pearson_dot | 0.8344 |
262
+ | **spearman_dot** | **0.8295** |
263
+ | pearson_max | 0.8344 |
264
+ | spearman_max | 0.8332 |
265
 
266
  #### Semantic Similarity
267
  * Dataset: `sts-test`
 
269
 
270
  | Metric | Value |
271
  |:--------------------|:-----------|
272
+ | pearson_cosine | 0.7776 |
273
+ | **spearman_cosine** | **0.7643** |
274
+ | pearson_manhattan | 0.7788 |
275
+ | spearman_manhattan | 0.7659 |
276
+ | pearson_euclidean | 0.7763 |
277
+ | spearman_euclidean | 0.7643 |
278
+ | pearson_dot | 0.7776 |
279
+ | spearman_dot | 0.7643 |
280
+ | pearson_max | 0.7788 |
281
+ | spearman_max | 0.7659 |
282
 
283
  <!--
284
  ## Bias, Risks and Limitations
 
548
  </details>
549
 
550
  ### Training Logs
551
+ | Epoch | Step | Training Loss | multi nli loss | snli loss | stsb loss | sts-dev_spearman_dot | sts-test_spearman_cosine |
552
+ |:----------:|:-------:|:-------------:|:--------------:|:----------:|:----------:|:--------------------:|:------------------------:|
553
+ | 0.0370 | 10 | 0.8347 | - | - | - | - | - |
554
+ | 0.0741 | 20 | 0.8269 | - | - | - | - | - |
555
+ | 0.1111 | 30 | 0.7036 | 1.0978 | 1.0984 | 0.0830 | 0.6636 | - |
556
+ | 0.1481 | 40 | 0.7889 | - | - | - | - | - |
557
+ | 0.1852 | 50 | 0.7948 | - | - | - | - | - |
558
+ | 0.2222 | 60 | 0.688 | 1.0976 | 1.0961 | 0.0679 | 0.7124 | - |
559
+ | 0.2593 | 70 | 0.7911 | - | - | - | - | - |
560
+ | 0.2963 | 80 | 0.7847 | - | - | - | - | - |
561
+ | 0.3333 | 90 | 0.6801 | 1.0950 | 1.0942 | 0.0522 | 0.7810 | - |
562
+ | 0.3704 | 100 | 0.7837 | - | - | - | - | - |
563
+ | 0.4074 | 110 | 0.7803 | - | - | - | - | - |
564
+ | 0.4444 | 120 | 0.6756 | 1.0978 | 1.0929 | 0.0441 | 0.8157 | - |
565
+ | 0.4815 | 130 | 0.7829 | - | - | - | - | - |
566
+ | 0.5185 | 140 | 0.7789 | - | - | - | - | - |
567
+ | 0.5556 | 150 | 0.6756 | 1.0954 | 1.0911 | 0.0433 | 0.8215 | - |
568
+ | 0.5926 | 160 | 0.7802 | - | - | - | - | - |
569
+ | 0.6296 | 170 | 0.7751 | - | - | - | - | - |
570
+ | 0.6667 | 180 | 0.6679 | 1.0934 | 1.0885 | 0.0401 | 0.8235 | - |
571
+ | 0.7037 | 190 | 0.7755 | - | - | - | - | - |
572
+ | 0.7407 | 200 | 0.775 | - | - | - | - | - |
573
+ | **0.7778** | **210** | **0.6694** | **1.0919** | **1.0859** | **0.0377** | **0.8295** | **-** |
574
+ | 0.8148 | 220 | 0.7733 | - | - | - | - | - |
575
+ | 0.8519 | 230 | 0.772 | - | - | - | - | - |
576
+ | 0.8889 | 240 | 0.6656 | 1.0891 | 1.0838 | 0.0365 | 0.8292 | - |
577
+ | 0.9259 | 250 | 0.7726 | - | - | - | - | - |
578
+ | 0.9630 | 260 | 0.7731 | - | - | - | - | - |
579
+ | 1.0 | 270 | 0.6674 | 1.0888 | 1.0833 | 0.0372 | 0.8295 | 0.7643 |
580
 
581
  * The bold row denotes the saved checkpoint.
582
 
583
  ### Environmental Impact
584
  Carbon emissions were measured using [CodeCarbon](https://github.com/mlco2/codecarbon).
585
+ - **Energy Consumed**: 0.018 kWh
586
  - **Carbon Emitted**: 0.007 kg of CO2
587
+ - **Hours Used**: 0.068 hours
588
 
589
  ### Training Hardware
590
  - **On Cloud**: No
emissions.csv CHANGED
@@ -1,3 +1,4 @@
1
  timestamp,project_name,run_id,duration,emissions,emissions_rate,cpu_power,gpu_power,ram_power,cpu_energy,gpu_energy,ram_energy,energy_consumed,country_name,country_iso_code,region,cloud_provider,cloud_region,os,python_version,codecarbon_version,cpu_count,cpu_model,gpu_count,gpu_model,longitude,latitude,ram_total_size,tracking_mode,on_cloud,pue
2
  2024-06-05T13:25:31,codecarbon,8f78a832-cb8b-4255-90f9-87291ab5a2c5,203.90490198135376,0.0066664767107362,3.269404828406667e-05,42.5,262.78315247135714,11.9164080619812,0.0024072106483909,0.0140688801439834,0.0006745199858985,0.0171506107782729,The Netherlands,NLD,utrecht,,,Windows-10-10.0.22631-SP0,3.11.6,2.3.4,24,13th Gen Intel(R) Core(TM) i7-13700K,1,1 x NVIDIA GeForce RTX 3090,5.0582,52.0756,31.777088165283203,machine,N,1.0
3
- 2024-06-05T14:02:28,codecarbon,c19a4b76-5dda-443e-a1eb-73550f818127,200.78387594223022,0.006543912203095872,3.259182129235663e-05,42.5,248.66806443178965,11.916408061981201,0.002370365202095774,0.013800701318329445,0.0006642268485203281,0.01683529336894555,The Netherlands,NLD,utrecht,,,Windows-10-10.0.22631-SP0,3.11.6,2.3.4,24,13th Gen Intel(R) Core(TM) i7-13700K,1,1 x NVIDIA GeForce RTX 3090,5.0582,52.0756,31.777088165283203,machine,N,1.0
 
 
1
  timestamp,project_name,run_id,duration,emissions,emissions_rate,cpu_power,gpu_power,ram_power,cpu_energy,gpu_energy,ram_energy,energy_consumed,country_name,country_iso_code,region,cloud_provider,cloud_region,os,python_version,codecarbon_version,cpu_count,cpu_model,gpu_count,gpu_model,longitude,latitude,ram_total_size,tracking_mode,on_cloud,pue
2
  2024-06-05T13:25:31,codecarbon,8f78a832-cb8b-4255-90f9-87291ab5a2c5,203.90490198135376,0.0066664767107362,3.269404828406667e-05,42.5,262.78315247135714,11.9164080619812,0.0024072106483909,0.0140688801439834,0.0006745199858985,0.0171506107782729,The Netherlands,NLD,utrecht,,,Windows-10-10.0.22631-SP0,3.11.6,2.3.4,24,13th Gen Intel(R) Core(TM) i7-13700K,1,1 x NVIDIA GeForce RTX 3090,5.0582,52.0756,31.777088165283203,machine,N,1.0
3
+ 2024-06-05T14:02:28,codecarbon,c19a4b76-5dda-443e-a1eb-73550f818127,200.78387594223025,0.0065439122030958,3.259182129235663e-05,42.5,248.66806443178965,11.9164080619812,0.0023703652020957,0.0138007013183294,0.0006642268485203,0.0168352933689455,The Netherlands,NLD,utrecht,,,Windows-10-10.0.22631-SP0,3.11.6,2.3.4,24,13th Gen Intel(R) Core(TM) i7-13700K,1,1 x NVIDIA GeForce RTX 3090,5.0582,52.0756,31.777088165283203,machine,N,1.0
4
+ 2024-06-05T14:28:07,codecarbon,8f089da1-33f1-4fca-8f92-14f942b5d735,210.7226061820984,0.006863209894681815,3.256987951615828e-05,42.5,170.86763354628195,11.916408061981201,0.0024876974340942173,0.014472137411034147,0.0006969044942159526,0.017656739339344318,The Netherlands,NLD,utrecht,,,Windows-10-10.0.22631-SP0,3.11.6,2.3.4,24,13th Gen Intel(R) Core(TM) i7-13700K,1,1 x NVIDIA GeForce RTX 3090,5.0582,52.0756,31.777088165283203,machine,N,1.0
runs/Jun05_14-24-25_Tom/events.out.tfevents.1717590271.Tom.22032.0 CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:1c4283ba9381ee2728d31d202c1bd893ec457d4fb41463325b379322e4e5901a
3
- size 19454
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:5daa10416f13952f48a2b55db8df31a9bebe5122191c678dd1a01461154db825
3
+ size 24190