|
--- |
|
language: |
|
- en |
|
thumbnail: null |
|
tags: |
|
- text generation |
|
- instruct |
|
pipeline_tag: text-generation |
|
inference: false |
|
--- |
|
<h1 style="text-align: center">Metharme 7B 4bit</h1> |
|
<h2 style="text-align: center">An instruction-tuned LLaMA biased towards fiction writing and conversation.</h2> |
|
|
|
## Model Details |
|
|
|
Converted from the XORs weights from PygmalionAI's release https://huggingface.co/PygmalionAI/metharme-7b |
|
|
|
Quantized for KoboldAI use using https://github.com/0cc4m/GPTQ-for-LLaMa |
|
|
|
I created several dozen quantized variations of this model and believe this variation to be "best." <br> |
|
<html> |
|
<head> |
|
<style> |
|
table { |
|
border:1px solid #b3adad; |
|
border-collapse:collapse; |
|
padding:5px; |
|
} |
|
table th { |
|
border:1px solid #b3adad; |
|
padding:5px; |
|
background: #f0f0f0; |
|
color: #313030; |
|
} |
|
table td { |
|
border:1px solid #b3adad; |
|
text-align:center; |
|
padding:5px; |
|
background: #ffffff; |
|
color: #313030; |
|
} |
|
</style> |
|
</head> |
|
<body> |
|
<table> |
|
<thead> |
|
<tr> |
|
<th>Model:</th> |
|
<th>Wikitext2</th> |
|
<th>Ptb-New</th> |
|
<th>C4-New</th> |
|
</tr> |
|
</thead> |
|
<tbody> |
|
<tr> |
|
<td>Metharme 7b - 16bit</td> |
|
<td>5.7208476066589355</td> |
|
<td>41.61103439331055</td> |
|
<td>7.512405872344971</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-AO-TS-Trits-damp0.1</td> |
|
<td>6.172733783721924</td> |
|
<td>45.19890213012695</td> |
|
<td>7.872506141662598</td> |
|
</tr> |
|
</tbody> |
|
</table> |
|
</body> |
|
</html> |
|
<br><b>Other benchmark scores at the bottom of readme.</b> |
|
<hr> |
|
Metharme 7B is an instruct model based on Meta's LLaMA-7B. |
|
|
|
This is an experiment to try and get a model that is usable for conversation, roleplaying and storywriting, but which can be guided using natural language like other instruct models. See the [prompting](#prompting) section below for examples. |
|
|
|
It was trained by doing supervised fine-tuning over a mixture of regular instruction data alongside roleplay, fictional stories and conversations with synthetically generated instructions attached. |
|
|
|
## Prompting |
|
|
|
The current model version has been trained on prompts using three different roles, which are denoted by the following tokens: `<|system|>`, `<|user|>` and `<|model|>`. |
|
|
|
The `<|system|>` prompt can be used to inject out-of-channel information behind the scenes, while the `<|user|>` prompt should be used to indicate user input. The `<|model|>` token should then be used to indicate that the model should generate a response. These tokens can happen multiple times and be chained up to form a conversation history. |
|
|
|
### Prompting example |
|
|
|
Here's a concrete example. Let's say you have the following system prompt: |
|
|
|
> This is a text adventure game. Describe the scenario to the user and give him three options to pick from on each turn. |
|
|
|
And the user has typed in the following message: |
|
|
|
> Start! |
|
|
|
To perform inference against the model, the prompt should be formatted like this: |
|
|
|
``` |
|
<|system|>This is a text adventure game. Describe the scenario to the user and give him three options to pick from on each turn.<|user|>Start!<|model|> |
|
``` |
|
|
|
The model might generate something like: |
|
|
|
> You are standing in front of an old, abandoned house. The windows are boarded up, and there's no sign of life around it. As you approach, you notice a strange feeling emanating from within. Suddenly, you hear a voice calling out to you... 'Come inside!' |
|
> |
|
> - Go inside the house. |
|
> - Ignore the call and move away. |
|
> - Run as fast as you can. |
|
|
|
It will automatically emit an end-of-text token (`</s>`) when it judges that the response is complete. |
|
|
|
Pretend then that the user has replied with `go inside`. The next prompt should then be: |
|
|
|
``` |
|
<|system|>This is a text adventure game. Describe the scenario to the user and give him three options to pick from on each turn.<|user|>Start!<|model|>You are standing in front of an old, abandoned house. The windows are boarded up, and there's no sign of life around it. As you approach, you notice a strange feeling emanating from within. Suddenly, you hear a voice calling out to you... 'Come inside!' |
|
|
|
- Go inside the house. |
|
- Ignore the call and move away. |
|
- Run as fast as you can.<|user|>go inside<|model|> |
|
``` |
|
|
|
Which might generate something like: |
|
|
|
> You enter the house, and immediately, a wave of unsettling energy washes over you. Your senses seem heightened, but your mind feels foggy. A figure appears before you - a tall man with piercing eyes. He speaks to you in a language you don't understand. |
|
> |
|
> - Ask him what he wants. |
|
> - Attack him. |
|
> - Run away. |
|
|
|
Same process applies. Usually, it is best to do a sliding window over the user and model turns, but keep the system prompt fixed at the start of the context window. |
|
|
|
## Limitations and biases |
|
|
|
The intended use-case for this model is fictional writing for entertainment purposes. Any other sort of usage is out of scope. |
|
|
|
As such, it was **not** fine-tuned to be safe and harmless: the base model _and_ this fine-tune have been trained on data known to contain profanity and texts that are lewd or otherwise offensive. It may produce socially unacceptable or undesirable text, even if the prompt itself does not include anything explicitly offensive. Outputs might often be factually wrong or misleading. |
|
|
|
<hr> |
|
<p><strong><font size="5">Click to Expand Benchmarks of different quantized variations</font></strong></p> |
|
<strong><font size="4">The lower the number, the better the score.</font></strong> |
|
<html> |
|
<body> |
|
<details> |
|
<summary>Benchmarks Sorted by C4-New score</summary> |
|
<table> |
|
<thead> |
|
<tr> |
|
<th>GPTQ Variation:</th> |
|
<th>Wikitext2</th> |
|
<th>Ptb-New</th> |
|
<th>C4-New</th> |
|
</tr> |
|
</thead> |
|
<tbody> |
|
<tr> |
|
<td>Metharme-7b-16bit</td> |
|
<td>5.7208476066589355</td> |
|
<td>41.61103439331055</td> |
|
<td>7.512405872344971</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-ao-ts-trits-damp0.1</td> |
|
<td>6.172733783721924</td> |
|
<td>45.19890213012695</td> |
|
<td>7.872506141662598</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-ao-trits-damp0.1</td> |
|
<td>6.163661956787109</td> |
|
<td>46.50249099731445</td> |
|
<td>7.877425193786621</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-ao-ts-damp0.1</td> |
|
<td>6.184001445770264</td> |
|
<td>46.17180633544922</td> |
|
<td>7.880400657653809</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-7b-4bit-ao-damp0.1</td> |
|
<td>6.220707893371582</td> |
|
<td>47.82929611206055</td> |
|
<td>7.884565353393555</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-ao-ts-trits</td> |
|
<td>6.310682773590088</td> |
|
<td>46.4483757019043</td> |
|
<td>7.898126602172852</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-7b-4bit-ao</td> |
|
<td>6.281311511993408</td> |
|
<td>46.79158401489258</td> |
|
<td>7.906069755554199</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-ao-trits</td> |
|
<td>6.283935546875</td> |
|
<td>46.57590103149414</td> |
|
<td>7.907411575317383</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-ao-ts</td> |
|
<td>6.329496383666992</td> |
|
<td>46.88129806518555</td> |
|
<td>7.910323143005371</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-ao-ts-sym-trits-damp0.1</td> |
|
<td>6.232576370239258</td> |
|
<td>48.081459045410156</td> |
|
<td>7.95023250579834</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-ao-ts-sym-damp0.1</td> |
|
<td>6.210323333740234</td> |
|
<td>47.66789245605469</td> |
|
<td>7.952476978302002</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-ao-sym-trits-damp0.1</td> |
|
<td>6.329384803771973</td> |
|
<td>48.06882858276367</td> |
|
<td>7.959168910980225</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-ao-ts-sym-trits</td> |
|
<td>6.471063137054443</td> |
|
<td>49.650611877441406</td> |
|
<td>7.969552040100098</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-ao-ts-sym</td> |
|
<td>6.460526943206787</td> |
|
<td>47.190460205078125</td> |
|
<td>7.9732160568237305</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-ao-sym-trits</td> |
|
<td>6.390106678009033</td> |
|
<td>48.15375900268555</td> |
|
<td>7.9804582595825195</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-7b-4bit-ao-sym</td> |
|
<td>6.477842807769775</td> |
|
<td>48.53507614135742</td> |
|
<td>7.993765354156494</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-ao-sym</td> |
|
<td>6.477842807769775</td> |
|
<td>48.53507614135742</td> |
|
<td>7.993765354156494</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-7b-4bit-ts-32g-trits</td> |
|
<td>6.632943153381348</td> |
|
<td>47.973228454589844</td> |
|
<td>8.013848304748535</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-7b-4bit-ts-32g-sym-damp0.1</td> |
|
<td>6.274552822113037</td> |
|
<td>47.35737228393555</td> |
|
<td>8.06270980834961</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-7b-4bit-ts-32g-sym-trits-damp0.1</td> |
|
<td>6.266031265258789</td> |
|
<td>47.346702575683594</td> |
|
<td>8.068148612976074</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-7b-4bit-32g-damp0.1</td> |
|
<td>6.107605934143066</td> |
|
<td>47.91380310058594</td> |
|
<td>8.068695068359375</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-7b-4bit-32g-trits-damp0.1</td> |
|
<td>6.128157138824463</td> |
|
<td>48.04175567626953</td> |
|
<td>8.0708646774292</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-7b-4bit-ts-32g-damp0.1</td> |
|
<td>6.219024658203125</td> |
|
<td>45.834869384765625</td> |
|
<td>8.071272850036621</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-7b-4bit-32g-trits</td> |
|
<td>7.017086029052734</td> |
|
<td>45.04129409790039</td> |
|
<td>8.074845314025879</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-7b-4bit-32g-sym-trits-damp0.1</td> |
|
<td>6.109438896179199</td> |
|
<td>47.35737228393555</td> |
|
<td>8.075060844421387</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-7b-4bit-ts-32g-trits-damp0.1</td> |
|
<td>6.118431568145752</td> |
|
<td>45.67333221435547</td> |
|
<td>8.077078819274902</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-7b-4bit-32g</td> |
|
<td>6.902080535888672</td> |
|
<td>50.237754821777344</td> |
|
<td>8.081602096557617</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-7b-4bit-ts-32g</td> |
|
<td>6.424218654632568</td> |
|
<td>48.48588943481445</td> |
|
<td>8.089512825012207</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-7b-4bit-ts-32g-sym-trits</td> |
|
<td>6.82415771484375</td> |
|
<td>48.82029724121094</td> |
|
<td>8.090987205505371</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-7b-4bit-32g-sym-damp0.1</td> |
|
<td>6.566899299621582</td> |
|
<td>48.0670166015625</td> |
|
<td>8.095841407775879</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-7b-4bit-ts-128g-trits-damp0.1</td> |
|
<td>6.289113521575928</td> |
|
<td>46.06787109375</td> |
|
<td>8.122251510620117</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-7b-4bit-32g-sym</td> |
|
<td>6.518134117126465</td> |
|
<td>49.66925811767578</td> |
|
<td>8.13516616821289</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-7b-4bit-32g-sym-trits</td> |
|
<td>6.206963539123535</td> |
|
<td>46.88833999633789</td> |
|
<td>8.13610553741455</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-7b-4bit-128g-damp0.1</td> |
|
<td>6.242006301879883</td> |
|
<td>45.30938720703125</td> |
|
<td>8.14249324798584</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-7b-4bit-ts-32g-sym</td> |
|
<td>6.387663841247559</td> |
|
<td>48.07244110107422</td> |
|
<td>8.173730850219727</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-7b-4bit-128g-trits-damp0.1</td> |
|
<td>6.262309551239014</td> |
|
<td>47.80055618286133</td> |
|
<td>8.192194938659668</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-7b-4bit-128g</td> |
|
<td>10.206376075744629</td> |
|
<td>49.00401306152344</td> |
|
<td>8.198845863342285</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-7b-4bit-ts-128g-damp0.1</td> |
|
<td>6.17774772644043</td> |
|
<td>46.47630310058594</td> |
|
<td>8.20170783996582</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-7b-4bit-ts-128g-sym-trits-damp0.1</td> |
|
<td>6.225503921508789</td> |
|
<td>53.12746047973633</td> |
|
<td>8.240595817565918</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-7b-4bit-128g-trits</td> |
|
<td>8.68796443939209</td> |
|
<td>49.73833465576172</td> |
|
<td>8.2406587600708</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-7b-4bit-ts-128g-sym-damp0.1</td> |
|
<td>6.584965705871582</td> |
|
<td>55.20026397705078</td> |
|
<td>8.268644332885742</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-7b-4bit-ts-128g-trits</td> |
|
<td>7.350858688354492</td> |
|
<td>44.25314712524414</td> |
|
<td>8.274221420288086</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-7b-4bit-128g-sym-trits-damp0.1</td> |
|
<td>6.585468769073486</td> |
|
<td>51.55869674682617</td> |
|
<td>8.2803316116333</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-7b-4bit-ts-128g-sym-trits</td> |
|
<td>6.756448745727539</td> |
|
<td>51.510311126708984</td> |
|
<td>8.292160987854004</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-7b-4bit-128g-sym-damp0.1</td> |
|
<td>6.379064083099365</td> |
|
<td>52.17233657836914</td> |
|
<td>8.316649436950684</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-7b-4bit-128g-sym-trits</td> |
|
<td>7.056288242340088</td> |
|
<td>48.983768463134766</td> |
|
<td>8.339276313781738</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-7b-4bit-ts-128g</td> |
|
<td>9.475017547607422</td> |
|
<td>52.358829498291016</td> |
|
<td>8.340700149536133</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-7b-4bit-128g-sym</td> |
|
<td>6.9575653076171875</td> |
|
<td>49.356834411621094</td> |
|
<td>8.35644817352295</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-7b-4bit-ts-128g-sym</td> |
|
<td>6.819341659545898</td> |
|
<td>55.28740310668945</td> |
|
<td>8.377721786499023</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-7b-4bit-ts-damp0.1</td> |
|
<td>6.7783050537109375</td> |
|
<td>51.81301498413086</td> |
|
<td>8.621373176574707</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-7b-4bit-ts-trits-damp0.1</td> |
|
<td>6.631694793701172</td> |
|
<td>51.7371711730957</td> |
|
<td>8.656966209411621</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-7b-4bit-damp0.1</td> |
|
<td>6.495014190673828</td> |
|
<td>49.39763641357422</td> |
|
<td>8.68167781829834</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-sym-damp0.1</td> |
|
<td>6.896804332733154</td> |
|
<td>57.4250602722168</td> |
|
<td>8.703770637512207</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-7b-4bit-ts-sym</td> |
|
<td>7.270263671875</td> |
|
<td>54.35262680053711</td> |
|
<td>8.787986755371094</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-7b-4bit-trits</td> |
|
<td>7.832409858703613</td> |
|
<td>55.383026123046875</td> |
|
<td>8.806737899780273</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-trits</td> |
|
<td>7.832409858703613</td> |
|
<td>55.383026123046875</td> |
|
<td>8.806737899780273</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-7b-4bit-ts-sym-damp0.1</td> |
|
<td>6.7517595291137695</td> |
|
<td>54.06147384643555</td> |
|
<td>8.821818351745605</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-7b-4bit-alone</td> |
|
<td>6.997134685516357</td> |
|
<td>58.87525177001953</td> |
|
<td>8.824191093444824</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-7b-4bit-ts-trits</td> |
|
<td>7.2306809425354</td> |
|
<td>66.78710174560547</td> |
|
<td>8.879831314086914</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-7b-4bit-ts-sym-trits-damp0.1</td> |
|
<td>6.886506080627441</td> |
|
<td>64.72743225097656</td> |
|
<td>8.880627632141113</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-7b-4bit-ts</td> |
|
<td>7.735969543457031</td> |
|
<td>62.92238235473633</td> |
|
<td>8.913650512695312</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-sym-trits-damp0.1</td> |
|
<td>7.075908184051514</td> |
|
<td>59.13897705078125</td> |
|
<td>8.919178009033203</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-7b-4bit-ts-sym-trits</td> |
|
<td>7.599876403808594</td> |
|
<td>55.75454330444336</td> |
|
<td>8.932201385498047</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-sym-trits</td> |
|
<td>7.494253635406494</td> |
|
<td>63.320709228515625</td> |
|
<td>8.969240188598633</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-sym</td> |
|
<td>7.585672855377197</td> |
|
<td>61.01168441772461</td> |
|
<td>9.032520294189453</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-7b-4bit-ao-128g</td> |
|
<td>251321.265625</td> |
|
<td>250117.859375</td> |
|
<td>232929.234375</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-7b-4bit-ao-32g</td> |
|
<td>275425.5</td> |
|
<td>267733.25</td> |
|
<td>254506.71875</td> |
|
</tr> |
|
</tbody> |
|
</table> |
|
</details> |
|
<details> |
|
<summary>Benchmarks Sorted by Wikitext2</summary> |
|
<table> |
|
<thead> |
|
<tr> |
|
<th>GPTQ Variation:</th> |
|
<th>Wikitext2</th> |
|
<th>Ptb-New</th> |
|
<th>C4-New</th> |
|
</tr> |
|
</thead> |
|
<tbody> |
|
<tr> |
|
<td>Metharme-7b-16bit</td> |
|
<td>5.7208476066589355</td> |
|
<td>41.61103439331055</td> |
|
<td>7.512405872344971</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-7b-4bit-32g-damp0.1</td> |
|
<td>6.107605934143066</td> |
|
<td>47.91380310058594</td> |
|
<td>8.068695068359375</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-7b-4bit-32g-sym-trits-damp0.1</td> |
|
<td>6.109438896179199</td> |
|
<td>47.35737228393555</td> |
|
<td>8.075060844421387</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-7b-4bit-ts-32g-trits-damp0.1</td> |
|
<td>6.118431568145752</td> |
|
<td>45.67333221435547</td> |
|
<td>8.077078819274902</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-7b-4bit-32g-trits-damp0.1</td> |
|
<td>6.128157138824463</td> |
|
<td>48.04175567626953</td> |
|
<td>8.0708646774292</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-ao-trits-damp0.1</td> |
|
<td>6.163661956787109</td> |
|
<td>46.50249099731445</td> |
|
<td>7.877425193786621</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-ao-ts-trits-damp0.1</td> |
|
<td>6.172733783721924</td> |
|
<td>45.19890213012695</td> |
|
<td>7.872506141662598</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-7b-4bit-ts-128g-damp0.1</td> |
|
<td>6.17774772644043</td> |
|
<td>46.47630310058594</td> |
|
<td>8.20170783996582</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-ao-ts-damp0.1</td> |
|
<td>6.184001445770264</td> |
|
<td>46.17180633544922</td> |
|
<td>7.880400657653809</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-7b-4bit-32g-sym-trits</td> |
|
<td>6.206963539123535</td> |
|
<td>46.88833999633789</td> |
|
<td>8.13610553741455</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-ao-ts-sym-damp0.1</td> |
|
<td>6.210323333740234</td> |
|
<td>47.66789245605469</td> |
|
<td>7.952476978302002</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-7b-4bit-ts-32g-damp0.1</td> |
|
<td>6.219024658203125</td> |
|
<td>45.834869384765625</td> |
|
<td>8.071272850036621</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-7b-4bit-ao-damp0.1</td> |
|
<td>6.220707893371582</td> |
|
<td>47.82929611206055</td> |
|
<td>7.884565353393555</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-7b-4bit-ts-128g-sym-trits-damp0.1</td> |
|
<td>6.225503921508789</td> |
|
<td>53.12746047973633</td> |
|
<td>8.240595817565918</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-ao-ts-sym-trits-damp0.1</td> |
|
<td>6.232576370239258</td> |
|
<td>48.081459045410156</td> |
|
<td>7.95023250579834</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-7b-4bit-128g-damp0.1</td> |
|
<td>6.242006301879883</td> |
|
<td>45.30938720703125</td> |
|
<td>8.14249324798584</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-7b-4bit-128g-trits-damp0.1</td> |
|
<td>6.262309551239014</td> |
|
<td>47.80055618286133</td> |
|
<td>8.192194938659668</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-7b-4bit-ts-32g-sym-trits-damp0.1</td> |
|
<td>6.266031265258789</td> |
|
<td>47.346702575683594</td> |
|
<td>8.068148612976074</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-7b-4bit-ts-32g-sym-damp0.1</td> |
|
<td>6.274552822113037</td> |
|
<td>47.35737228393555</td> |
|
<td>8.06270980834961</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-7b-4bit-ao</td> |
|
<td>6.281311511993408</td> |
|
<td>46.79158401489258</td> |
|
<td>7.906069755554199</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-ao-trits</td> |
|
<td>6.283935546875</td> |
|
<td>46.57590103149414</td> |
|
<td>7.907411575317383</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-7b-4bit-ts-128g-trits-damp0.1</td> |
|
<td>6.289113521575928</td> |
|
<td>46.06787109375</td> |
|
<td>8.122251510620117</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-ao-ts-trits</td> |
|
<td>6.310682773590088</td> |
|
<td>46.4483757019043</td> |
|
<td>7.898126602172852</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-ao-sym-trits-damp0.1</td> |
|
<td>6.329384803771973</td> |
|
<td>48.06882858276367</td> |
|
<td>7.959168910980225</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-ao-ts</td> |
|
<td>6.329496383666992</td> |
|
<td>46.88129806518555</td> |
|
<td>7.910323143005371</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-7b-4bit-128g-sym-damp0.1</td> |
|
<td>6.379064083099365</td> |
|
<td>52.17233657836914</td> |
|
<td>8.316649436950684</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-7b-4bit-ts-32g-sym</td> |
|
<td>6.387663841247559</td> |
|
<td>48.07244110107422</td> |
|
<td>8.173730850219727</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-ao-sym-trits</td> |
|
<td>6.390106678009033</td> |
|
<td>48.15375900268555</td> |
|
<td>7.9804582595825195</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-7b-4bit-ts-32g</td> |
|
<td>6.424218654632568</td> |
|
<td>48.48588943481445</td> |
|
<td>8.089512825012207</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-ao-ts-sym</td> |
|
<td>6.460526943206787</td> |
|
<td>47.190460205078125</td> |
|
<td>7.9732160568237305</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-ao-ts-sym-trits</td> |
|
<td>6.471063137054443</td> |
|
<td>49.650611877441406</td> |
|
<td>7.969552040100098</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-7b-4bit-ao-sym</td> |
|
<td>6.477842807769775</td> |
|
<td>48.53507614135742</td> |
|
<td>7.993765354156494</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-ao-sym</td> |
|
<td>6.477842807769775</td> |
|
<td>48.53507614135742</td> |
|
<td>7.993765354156494</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-7b-4bit-damp0.1</td> |
|
<td>6.495014190673828</td> |
|
<td>49.39763641357422</td> |
|
<td>8.68167781829834</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-7b-4bit-32g-sym</td> |
|
<td>6.518134117126465</td> |
|
<td>49.66925811767578</td> |
|
<td>8.13516616821289</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-7b-4bit-32g-sym-damp0.1</td> |
|
<td>6.566899299621582</td> |
|
<td>48.0670166015625</td> |
|
<td>8.095841407775879</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-7b-4bit-ts-128g-sym-damp0.1</td> |
|
<td>6.584965705871582</td> |
|
<td>55.20026397705078</td> |
|
<td>8.268644332885742</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-7b-4bit-128g-sym-trits-damp0.1</td> |
|
<td>6.585468769073486</td> |
|
<td>51.55869674682617</td> |
|
<td>8.2803316116333</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-7b-4bit-ts-trits-damp0.1</td> |
|
<td>6.631694793701172</td> |
|
<td>51.7371711730957</td> |
|
<td>8.656966209411621</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-7b-4bit-ts-32g-trits</td> |
|
<td>6.632943153381348</td> |
|
<td>47.973228454589844</td> |
|
<td>8.013848304748535</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-7b-4bit-ts-sym-damp0.1</td> |
|
<td>6.7517595291137695</td> |
|
<td>54.06147384643555</td> |
|
<td>8.821818351745605</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-7b-4bit-ts-128g-sym-trits</td> |
|
<td>6.756448745727539</td> |
|
<td>51.510311126708984</td> |
|
<td>8.292160987854004</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-7b-4bit-ts-damp0.1</td> |
|
<td>6.7783050537109375</td> |
|
<td>51.81301498413086</td> |
|
<td>8.621373176574707</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-7b-4bit-ts-128g-sym</td> |
|
<td>6.819341659545898</td> |
|
<td>55.28740310668945</td> |
|
<td>8.377721786499023</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-7b-4bit-ts-32g-sym-trits</td> |
|
<td>6.82415771484375</td> |
|
<td>48.82029724121094</td> |
|
<td>8.090987205505371</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-7b-4bit-ts-sym-trits-damp0.1</td> |
|
<td>6.886506080627441</td> |
|
<td>64.72743225097656</td> |
|
<td>8.880627632141113</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-sym-damp0.1</td> |
|
<td>6.896804332733154</td> |
|
<td>57.4250602722168</td> |
|
<td>8.703770637512207</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-7b-4bit-32g</td> |
|
<td>6.902080535888672</td> |
|
<td>50.237754821777344</td> |
|
<td>8.081602096557617</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-7b-4bit-128g-sym</td> |
|
<td>6.9575653076171875</td> |
|
<td>49.356834411621094</td> |
|
<td>8.35644817352295</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-7b-4bit-alone</td> |
|
<td>6.997134685516357</td> |
|
<td>58.87525177001953</td> |
|
<td>8.824191093444824</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-7b-4bit-32g-trits</td> |
|
<td>7.017086029052734</td> |
|
<td>45.04129409790039</td> |
|
<td>8.074845314025879</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-7b-4bit-128g-sym-trits</td> |
|
<td>7.056288242340088</td> |
|
<td>48.983768463134766</td> |
|
<td>8.339276313781738</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-sym-trits-damp0.1</td> |
|
<td>7.075908184051514</td> |
|
<td>59.13897705078125</td> |
|
<td>8.919178009033203</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-7b-4bit-ts-trits</td> |
|
<td>7.2306809425354</td> |
|
<td>66.78710174560547</td> |
|
<td>8.879831314086914</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-7b-4bit-ts-sym</td> |
|
<td>7.270263671875</td> |
|
<td>54.35262680053711</td> |
|
<td>8.787986755371094</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-7b-4bit-ts-128g-trits</td> |
|
<td>7.350858688354492</td> |
|
<td>44.25314712524414</td> |
|
<td>8.274221420288086</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-sym-trits</td> |
|
<td>7.494253635406494</td> |
|
<td>63.320709228515625</td> |
|
<td>8.969240188598633</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-sym</td> |
|
<td>7.585672855377197</td> |
|
<td>61.01168441772461</td> |
|
<td>9.032520294189453</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-7b-4bit-ts-sym-trits</td> |
|
<td>7.599876403808594</td> |
|
<td>55.75454330444336</td> |
|
<td>8.932201385498047</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-7b-4bit-ts</td> |
|
<td>7.735969543457031</td> |
|
<td>62.92238235473633</td> |
|
<td>8.913650512695312</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-7b-4bit-trits</td> |
|
<td>7.832409858703613</td> |
|
<td>55.383026123046875</td> |
|
<td>8.806737899780273</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-trits</td> |
|
<td>7.832409858703613</td> |
|
<td>55.383026123046875</td> |
|
<td>8.806737899780273</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-7b-4bit-128g-trits</td> |
|
<td>8.68796443939209</td> |
|
<td>49.73833465576172</td> |
|
<td>8.2406587600708</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-7b-4bit-ts-128g</td> |
|
<td>9.475017547607422</td> |
|
<td>52.358829498291016</td> |
|
<td>8.340700149536133</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-7b-4bit-128g</td> |
|
<td>10.206376075744629</td> |
|
<td>49.00401306152344</td> |
|
<td>8.198845863342285</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-7b-4bit-ao-128g</td> |
|
<td>251321.265625</td> |
|
<td>250117.859375</td> |
|
<td>232929.234375</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-7b-4bit-ao-32g</td> |
|
<td>275425.5</td> |
|
<td>267733.25</td> |
|
<td>254506.71875</td> |
|
</tr> |
|
</tbody> |
|
</table> |
|
</details> |
|
<details> |
|
<summary>Benchmarks Sorted by PTB-new Score</summary> |
|
<table> |
|
<thead> |
|
<tr> |
|
<th>GPTQ Variation:</th> |
|
<th>Wikitext2</th> |
|
<th>Ptb-New</th> |
|
<th>C4-New</th> |
|
</tr> |
|
</thead> |
|
<tbody> |
|
<tr> |
|
<td>Metharme-7b-16bit</td> |
|
<td>5.7208476066589355</td> |
|
<td>41.61103439331055</td> |
|
<td>7.512405872344971</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-7b-4bit-ts-128g-trits</td> |
|
<td>7.350858688354492</td> |
|
<td>44.25314712524414</td> |
|
<td>8.274221420288086</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-7b-4bit-32g-trits</td> |
|
<td>7.017086029052734</td> |
|
<td>45.04129409790039</td> |
|
<td>8.074845314025879</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-ao-ts-trits-damp0.1</td> |
|
<td>6.172733783721924</td> |
|
<td>45.19890213012695</td> |
|
<td>7.872506141662598</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-7b-4bit-128g-damp0.1</td> |
|
<td>6.242006301879883</td> |
|
<td>45.30938720703125</td> |
|
<td>8.14249324798584</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-7b-4bit-ts-32g-trits-damp0.1</td> |
|
<td>6.118431568145752</td> |
|
<td>45.67333221435547</td> |
|
<td>8.077078819274902</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-7b-4bit-ts-32g-damp0.1</td> |
|
<td>6.219024658203125</td> |
|
<td>45.834869384765625</td> |
|
<td>8.071272850036621</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-7b-4bit-ts-128g-trits-damp0.1</td> |
|
<td>6.289113521575928</td> |
|
<td>46.06787109375</td> |
|
<td>8.122251510620117</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-ao-ts-damp0.1</td> |
|
<td>6.184001445770264</td> |
|
<td>46.17180633544922</td> |
|
<td>7.880400657653809</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-ao-ts-trits</td> |
|
<td>6.310682773590088</td> |
|
<td>46.4483757019043</td> |
|
<td>7.898126602172852</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-7b-4bit-ts-128g-damp0.1</td> |
|
<td>6.17774772644043</td> |
|
<td>46.47630310058594</td> |
|
<td>8.20170783996582</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-ao-trits-damp0.1</td> |
|
<td>6.163661956787109</td> |
|
<td>46.50249099731445</td> |
|
<td>7.877425193786621</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-ao-trits</td> |
|
<td>6.283935546875</td> |
|
<td>46.57590103149414</td> |
|
<td>7.907411575317383</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-7b-4bit-ao</td> |
|
<td>6.281311511993408</td> |
|
<td>46.79158401489258</td> |
|
<td>7.906069755554199</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-ao-ts</td> |
|
<td>6.329496383666992</td> |
|
<td>46.88129806518555</td> |
|
<td>7.910323143005371</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-7b-4bit-32g-sym-trits</td> |
|
<td>6.206963539123535</td> |
|
<td>46.88833999633789</td> |
|
<td>8.13610553741455</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-ao-ts-sym</td> |
|
<td>6.460526943206787</td> |
|
<td>47.190460205078125</td> |
|
<td>7.9732160568237305</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-7b-4bit-ts-32g-sym-trits-damp0.1</td> |
|
<td>6.266031265258789</td> |
|
<td>47.346702575683594</td> |
|
<td>8.068148612976074</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-7b-4bit-ts-32g-sym-damp0.1</td> |
|
<td>6.274552822113037</td> |
|
<td>47.35737228393555</td> |
|
<td>8.06270980834961</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-7b-4bit-32g-sym-trits-damp0.1</td> |
|
<td>6.109438896179199</td> |
|
<td>47.35737228393555</td> |
|
<td>8.075060844421387</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-ao-ts-sym-damp0.1</td> |
|
<td>6.210323333740234</td> |
|
<td>47.66789245605469</td> |
|
<td>7.952476978302002</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-7b-4bit-128g-trits-damp0.1</td> |
|
<td>6.262309551239014</td> |
|
<td>47.80055618286133</td> |
|
<td>8.192194938659668</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-7b-4bit-ao-damp0.1</td> |
|
<td>6.220707893371582</td> |
|
<td>47.82929611206055</td> |
|
<td>7.884565353393555</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-7b-4bit-32g-damp0.1</td> |
|
<td>6.107605934143066</td> |
|
<td>47.91380310058594</td> |
|
<td>8.068695068359375</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-7b-4bit-ts-32g-trits</td> |
|
<td>6.632943153381348</td> |
|
<td>47.973228454589844</td> |
|
<td>8.013848304748535</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-7b-4bit-32g-trits-damp0.1</td> |
|
<td>6.128157138824463</td> |
|
<td>48.04175567626953</td> |
|
<td>8.0708646774292</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-7b-4bit-32g-sym-damp0.1</td> |
|
<td>6.566899299621582</td> |
|
<td>48.0670166015625</td> |
|
<td>8.095841407775879</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-ao-sym-trits-damp0.1</td> |
|
<td>6.329384803771973</td> |
|
<td>48.06882858276367</td> |
|
<td>7.959168910980225</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-7b-4bit-ts-32g-sym</td> |
|
<td>6.387663841247559</td> |
|
<td>48.07244110107422</td> |
|
<td>8.173730850219727</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-ao-ts-sym-trits-damp0.1</td> |
|
<td>6.232576370239258</td> |
|
<td>48.081459045410156</td> |
|
<td>7.95023250579834</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-ao-sym-trits</td> |
|
<td>6.390106678009033</td> |
|
<td>48.15375900268555</td> |
|
<td>7.9804582595825195</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-7b-4bit-ts-32g</td> |
|
<td>6.424218654632568</td> |
|
<td>48.48588943481445</td> |
|
<td>8.089512825012207</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-7b-4bit-ao-sym</td> |
|
<td>6.477842807769775</td> |
|
<td>48.53507614135742</td> |
|
<td>7.993765354156494</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-ao-sym</td> |
|
<td>6.477842807769775</td> |
|
<td>48.53507614135742</td> |
|
<td>7.993765354156494</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-7b-4bit-ts-32g-sym-trits</td> |
|
<td>6.82415771484375</td> |
|
<td>48.82029724121094</td> |
|
<td>8.090987205505371</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-7b-4bit-128g-sym-trits</td> |
|
<td>7.056288242340088</td> |
|
<td>48.983768463134766</td> |
|
<td>8.339276313781738</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-7b-4bit-128g</td> |
|
<td>10.206376075744629</td> |
|
<td>49.00401306152344</td> |
|
<td>8.198845863342285</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-7b-4bit-128g-sym</td> |
|
<td>6.9575653076171875</td> |
|
<td>49.356834411621094</td> |
|
<td>8.35644817352295</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-7b-4bit-damp0.1</td> |
|
<td>6.495014190673828</td> |
|
<td>49.39763641357422</td> |
|
<td>8.68167781829834</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-ao-ts-sym-trits</td> |
|
<td>6.471063137054443</td> |
|
<td>49.650611877441406</td> |
|
<td>7.969552040100098</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-7b-4bit-32g-sym</td> |
|
<td>6.518134117126465</td> |
|
<td>49.66925811767578</td> |
|
<td>8.13516616821289</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-7b-4bit-128g-trits</td> |
|
<td>8.68796443939209</td> |
|
<td>49.73833465576172</td> |
|
<td>8.2406587600708</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-7b-4bit-32g</td> |
|
<td>6.902080535888672</td> |
|
<td>50.237754821777344</td> |
|
<td>8.081602096557617</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-7b-4bit-ts-128g-sym-trits</td> |
|
<td>6.756448745727539</td> |
|
<td>51.510311126708984</td> |
|
<td>8.292160987854004</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-7b-4bit-128g-sym-trits-damp0.1</td> |
|
<td>6.585468769073486</td> |
|
<td>51.55869674682617</td> |
|
<td>8.2803316116333</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-7b-4bit-ts-trits-damp0.1</td> |
|
<td>6.631694793701172</td> |
|
<td>51.7371711730957</td> |
|
<td>8.656966209411621</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-7b-4bit-ts-damp0.1</td> |
|
<td>6.7783050537109375</td> |
|
<td>51.81301498413086</td> |
|
<td>8.621373176574707</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-7b-4bit-128g-sym-damp0.1</td> |
|
<td>6.379064083099365</td> |
|
<td>52.17233657836914</td> |
|
<td>8.316649436950684</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-7b-4bit-ts-128g</td> |
|
<td>9.475017547607422</td> |
|
<td>52.358829498291016</td> |
|
<td>8.340700149536133</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-7b-4bit-ts-128g-sym-trits-damp0.1</td> |
|
<td>6.225503921508789</td> |
|
<td>53.12746047973633</td> |
|
<td>8.240595817565918</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-7b-4bit-ts-sym-damp0.1</td> |
|
<td>6.7517595291137695</td> |
|
<td>54.06147384643555</td> |
|
<td>8.821818351745605</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-7b-4bit-ts-sym</td> |
|
<td>7.270263671875</td> |
|
<td>54.35262680053711</td> |
|
<td>8.787986755371094</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-7b-4bit-ts-128g-sym-damp0.1</td> |
|
<td>6.584965705871582</td> |
|
<td>55.20026397705078</td> |
|
<td>8.268644332885742</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-7b-4bit-ts-128g-sym</td> |
|
<td>6.819341659545898</td> |
|
<td>55.28740310668945</td> |
|
<td>8.377721786499023</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-7b-4bit-trits</td> |
|
<td>7.832409858703613</td> |
|
<td>55.383026123046875</td> |
|
<td>8.806737899780273</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-trits</td> |
|
<td>7.832409858703613</td> |
|
<td>55.383026123046875</td> |
|
<td>8.806737899780273</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-7b-4bit-ts-sym-trits</td> |
|
<td>7.599876403808594</td> |
|
<td>55.75454330444336</td> |
|
<td>8.932201385498047</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-sym-damp0.1</td> |
|
<td>6.896804332733154</td> |
|
<td>57.4250602722168</td> |
|
<td>8.703770637512207</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-7b-4bit-alone</td> |
|
<td>6.997134685516357</td> |
|
<td>58.87525177001953</td> |
|
<td>8.824191093444824</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-sym-trits-damp0.1</td> |
|
<td>7.075908184051514</td> |
|
<td>59.13897705078125</td> |
|
<td>8.919178009033203</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-sym</td> |
|
<td>7.585672855377197</td> |
|
<td>61.01168441772461</td> |
|
<td>9.032520294189453</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-7b-4bit-ts</td> |
|
<td>7.735969543457031</td> |
|
<td>62.92238235473633</td> |
|
<td>8.913650512695312</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-sym-trits</td> |
|
<td>7.494253635406494</td> |
|
<td>63.320709228515625</td> |
|
<td>8.969240188598633</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-7b-4bit-ts-sym-trits-damp0.1</td> |
|
<td>6.886506080627441</td> |
|
<td>64.72743225097656</td> |
|
<td>8.880627632141113</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-7b-4bit-ts-trits</td> |
|
<td>7.2306809425354</td> |
|
<td>66.78710174560547</td> |
|
<td>8.879831314086914</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-7b-4bit-ao-128g</td> |
|
<td>251321.265625</td> |
|
<td>250117.859375</td> |
|
<td>232929.234375</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-7b-4bit-ao-32g</td> |
|
<td>275425.5</td> |
|
<td>267733.25</td> |
|
<td>254506.71875</td> |
|
</tr> |
|
</tbody> |
|
</table> |
|
</details> |
|
<details> |
|
<summary>Benchmarks Sorted in Alphabetical Order</summary> |
|
<table> |
|
<thead> |
|
<tr> |
|
<th>GPTQ Variation:</th> |
|
<th>Wikitext2</th> |
|
<th>Ptb-New</th> |
|
<th>C4-New</th> |
|
</tr> |
|
</thead> |
|
<tbody> |
|
<tr> |
|
<td>Metharme-7b-16bit</td> |
|
<td>5.7208476066589355</td> |
|
<td>41.61103439331055</td> |
|
<td>7.512405872344971</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-7b-4bit-128g-damp0.1</td> |
|
<td>6.242006301879883</td> |
|
<td>45.30938720703125</td> |
|
<td>8.14249324798584</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-7b-4bit-128g-sym-damp0.1</td> |
|
<td>6.379064083099365</td> |
|
<td>52.17233657836914</td> |
|
<td>8.316649436950684</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-7b-4bit-128g-sym-trits-damp0.1</td> |
|
<td>6.585468769073486</td> |
|
<td>51.55869674682617</td> |
|
<td>8.2803316116333</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-7b-4bit-128g-trits-damp0.1</td> |
|
<td>6.262309551239014</td> |
|
<td>47.80055618286133</td> |
|
<td>8.192194938659668</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-7b-4bit-128g</td> |
|
<td>10.206376075744629</td> |
|
<td>49.00401306152344</td> |
|
<td>8.198845863342285</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-7b-4bit-32g-damp0.1</td> |
|
<td>6.107605934143066</td> |
|
<td>47.91380310058594</td> |
|
<td>8.068695068359375</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-7b-4bit-32g-sym-damp0.1</td> |
|
<td>6.566899299621582</td> |
|
<td>48.0670166015625</td> |
|
<td>8.095841407775879</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-7b-4bit-32g-sym-trits-damp0.1</td> |
|
<td>6.109438896179199</td> |
|
<td>47.35737228393555</td> |
|
<td>8.075060844421387</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-7b-4bit-32g-trits-damp0.1</td> |
|
<td>6.128157138824463</td> |
|
<td>48.04175567626953</td> |
|
<td>8.0708646774292</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-7b-4bit-32g</td> |
|
<td>6.902080535888672</td> |
|
<td>50.237754821777344</td> |
|
<td>8.081602096557617</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-7b-4bit-alone</td> |
|
<td>6.997134685516357</td> |
|
<td>58.87525177001953</td> |
|
<td>8.824191093444824</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-7b-4bit-ao-128g</td> |
|
<td>251321.265625</td> |
|
<td>250117.859375</td> |
|
<td>232929.234375</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-7b-4bit-ao-32g</td> |
|
<td>275425.5</td> |
|
<td>267733.25</td> |
|
<td>254506.71875</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-7b-4bit-ao-damp0.1</td> |
|
<td>6.220707893371582</td> |
|
<td>47.82929611206055</td> |
|
<td>7.884565353393555</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-7b-4bit-ao-sym</td> |
|
<td>6.477842807769775</td> |
|
<td>48.53507614135742</td> |
|
<td>7.993765354156494</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-7b-4bit-ao</td> |
|
<td>6.281311511993408</td> |
|
<td>46.79158401489258</td> |
|
<td>7.906069755554199</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-7b-4bit-damp0.1</td> |
|
<td>6.495014190673828</td> |
|
<td>49.39763641357422</td> |
|
<td>8.68167781829834</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-7b-4bit-trits</td> |
|
<td>7.832409858703613</td> |
|
<td>55.383026123046875</td> |
|
<td>8.806737899780273</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-7b-4bit-ts-128g-damp0.1</td> |
|
<td>6.17774772644043</td> |
|
<td>46.47630310058594</td> |
|
<td>8.20170783996582</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-7b-4bit-ts-128g-sym-damp0.1</td> |
|
<td>6.584965705871582</td> |
|
<td>55.20026397705078</td> |
|
<td>8.268644332885742</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-7b-4bit-ts-128g-sym-trits-damp0.1</td> |
|
<td>6.225503921508789</td> |
|
<td>53.12746047973633</td> |
|
<td>8.240595817565918</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-7b-4bit-ts-128g-trits-damp0.1</td> |
|
<td>6.289113521575928</td> |
|
<td>46.06787109375</td> |
|
<td>8.122251510620117</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-7b-4bit-ts-128g</td> |
|
<td>9.475017547607422</td> |
|
<td>52.358829498291016</td> |
|
<td>8.340700149536133</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-7b-4bit-ts-32g-damp0.1</td> |
|
<td>6.219024658203125</td> |
|
<td>45.834869384765625</td> |
|
<td>8.071272850036621</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-7b-4bit-ts-32g-sym-damp0.1</td> |
|
<td>6.274552822113037</td> |
|
<td>47.35737228393555</td> |
|
<td>8.06270980834961</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-7b-4bit-ts-32g-sym-trits-damp0.1</td> |
|
<td>6.266031265258789</td> |
|
<td>47.346702575683594</td> |
|
<td>8.068148612976074</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-7b-4bit-ts-32g-trits-damp0.1</td> |
|
<td>6.118431568145752</td> |
|
<td>45.67333221435547</td> |
|
<td>8.077078819274902</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-7b-4bit-ts-32g</td> |
|
<td>6.424218654632568</td> |
|
<td>48.48588943481445</td> |
|
<td>8.089512825012207</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-7b-4bit-ts-damp0.1</td> |
|
<td>6.7783050537109375</td> |
|
<td>51.81301498413086</td> |
|
<td>8.621373176574707</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-7b-4bit-ts-sym-damp0.1</td> |
|
<td>6.7517595291137695</td> |
|
<td>54.06147384643555</td> |
|
<td>8.821818351745605</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-7b-4bit-ts-sym-trits-damp0.1</td> |
|
<td>6.886506080627441</td> |
|
<td>64.72743225097656</td> |
|
<td>8.880627632141113</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-7b-4bit-ts-trits-damp0.1</td> |
|
<td>6.631694793701172</td> |
|
<td>51.7371711730957</td> |
|
<td>8.656966209411621</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-7b-4bit-ts</td> |
|
<td>7.735969543457031</td> |
|
<td>62.92238235473633</td> |
|
<td>8.913650512695312</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-7b-4bit-128g-sym-trits</td> |
|
<td>7.056288242340088</td> |
|
<td>48.983768463134766</td> |
|
<td>8.339276313781738</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-7b-4bit-128g-sym</td> |
|
<td>6.9575653076171875</td> |
|
<td>49.356834411621094</td> |
|
<td>8.35644817352295</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-7b-4bit-128g-trits</td> |
|
<td>8.68796443939209</td> |
|
<td>49.73833465576172</td> |
|
<td>8.2406587600708</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-7b-4bit-32g-sym-trits</td> |
|
<td>6.206963539123535</td> |
|
<td>46.88833999633789</td> |
|
<td>8.13610553741455</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-7b-4bit-32g-sym</td> |
|
<td>6.518134117126465</td> |
|
<td>49.66925811767578</td> |
|
<td>8.13516616821289</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-7b-4bit-32g-trits</td> |
|
<td>7.017086029052734</td> |
|
<td>45.04129409790039</td> |
|
<td>8.074845314025879</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-7b-4bit-ts-128g-sym-trits</td> |
|
<td>6.756448745727539</td> |
|
<td>51.510311126708984</td> |
|
<td>8.292160987854004</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-7b-4bit-ts-128g-sym</td> |
|
<td>6.819341659545898</td> |
|
<td>55.28740310668945</td> |
|
<td>8.377721786499023</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-7b-4bit-ts-128g-trits</td> |
|
<td>7.350858688354492</td> |
|
<td>44.25314712524414</td> |
|
<td>8.274221420288086</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-7b-4bit-ts-32g-sym-trits</td> |
|
<td>6.82415771484375</td> |
|
<td>48.82029724121094</td> |
|
<td>8.090987205505371</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-7b-4bit-ts-32g-sym</td> |
|
<td>6.387663841247559</td> |
|
<td>48.07244110107422</td> |
|
<td>8.173730850219727</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-7b-4bit-ts-32g-trits</td> |
|
<td>6.632943153381348</td> |
|
<td>47.973228454589844</td> |
|
<td>8.013848304748535</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-7b-4bit-ts-sym-trits</td> |
|
<td>7.599876403808594</td> |
|
<td>55.75454330444336</td> |
|
<td>8.932201385498047</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-7b-4bit-ts-sym</td> |
|
<td>7.270263671875</td> |
|
<td>54.35262680053711</td> |
|
<td>8.787986755371094</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-7b-4bit-ts-trits</td> |
|
<td>7.2306809425354</td> |
|
<td>66.78710174560547</td> |
|
<td>8.879831314086914</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-ao-sym-trits-damp0.1</td> |
|
<td>6.329384803771973</td> |
|
<td>48.06882858276367</td> |
|
<td>7.959168910980225</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-ao-sym-trits</td> |
|
<td>6.390106678009033</td> |
|
<td>48.15375900268555</td> |
|
<td>7.9804582595825195</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-ao-sym</td> |
|
<td>6.477842807769775</td> |
|
<td>48.53507614135742</td> |
|
<td>7.993765354156494</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-ao-trits-damp0.1</td> |
|
<td>6.163661956787109</td> |
|
<td>46.50249099731445</td> |
|
<td>7.877425193786621</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-ao-trits</td> |
|
<td>6.283935546875</td> |
|
<td>46.57590103149414</td> |
|
<td>7.907411575317383</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-ao-ts-damp0.1</td> |
|
<td>6.184001445770264</td> |
|
<td>46.17180633544922</td> |
|
<td>7.880400657653809</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-ao-ts-sym-damp0.1</td> |
|
<td>6.210323333740234</td> |
|
<td>47.66789245605469</td> |
|
<td>7.952476978302002</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-ao-ts-sym-trits-damp0.1</td> |
|
<td>6.232576370239258</td> |
|
<td>48.081459045410156</td> |
|
<td>7.95023250579834</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-ao-ts-sym-trits</td> |
|
<td>6.471063137054443</td> |
|
<td>49.650611877441406</td> |
|
<td>7.969552040100098</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-ao-ts-sym</td> |
|
<td>6.460526943206787</td> |
|
<td>47.190460205078125</td> |
|
<td>7.9732160568237305</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-ao-ts-trits-damp0.1</td> |
|
<td>6.172733783721924</td> |
|
<td>45.19890213012695</td> |
|
<td>7.872506141662598</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-ao-ts-trits</td> |
|
<td>6.310682773590088</td> |
|
<td>46.4483757019043</td> |
|
<td>7.898126602172852</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-ao-ts</td> |
|
<td>6.329496383666992</td> |
|
<td>46.88129806518555</td> |
|
<td>7.910323143005371</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-sym-damp0.1</td> |
|
<td>6.896804332733154</td> |
|
<td>57.4250602722168</td> |
|
<td>8.703770637512207</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-sym-trits-damp0.1</td> |
|
<td>7.075908184051514</td> |
|
<td>59.13897705078125</td> |
|
<td>8.919178009033203</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-sym-trits</td> |
|
<td>7.494253635406494</td> |
|
<td>63.320709228515625</td> |
|
<td>8.969240188598633</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-sym</td> |
|
<td>7.585672855377197</td> |
|
<td>61.01168441772461</td> |
|
<td>9.032520294189453</td> |
|
</tr> |
|
<tr> |
|
<td>Metharme-trits</td> |
|
<td>7.832409858703613</td> |
|
<td>55.383026123046875</td> |
|
<td>8.806737899780273</td> |
|
</tr> |
|
</tbody> |
|
</table> |
|
</details> |
|
</body> |
|
</html> |
