--- license: apache-2.0 library_name: peft tags: - generated_from_trainer base_model: unsloth/tinyllama-bnb-4bit metrics: - accuracy model-index: - name: tinyllama-qwantz-coherent results: [] --- # tinyllama-qwantz-coherent This model is a fine-tuned version of [unsloth/tinyllama-bnb-4bit](https://huggingface.co/unsloth/tinyllama-bnb-4bit) on an unknown dataset. It achieves the following results on the evaluation set: - Loss: 0.5015 - Accuracy: 0.8650 ## Model description More information needed ## Intended uses & limitations More information needed ## Training and evaluation data More information needed ## Training procedure ### Training hyperparameters The following hyperparameters were used during training: - learning_rate: 0.0001 - train_batch_size: 8 - eval_batch_size: 8 - seed: 42 - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08 - lr_scheduler_type: linear - num_epochs: 3 ### Training results | Training Loss | Epoch | Step | Validation Loss | Accuracy | |:-------------:|:-----:|:----:|:---------------:|:--------:| | 0.4508 | 1.0 | 1428 | 0.3631 | 0.8509 | | 0.4223 | 2.0 | 2856 | 0.4421 | 0.8617 | | 0.147 | 3.0 | 4284 | 0.5015 | 0.8650 | ``` Can save 90% of coherent strings by discarding 79% of dp strings (cutoff is -60.36113500595093) Can save 95% of coherent strings by discarding 70% of dp strings (cutoff is -76.16859674453735) Can save 98% of coherent strings by discarding 58% of dp strings (cutoff is -91.0932719707489) Can save 99% of coherent strings by discarding 40% of dp strings (cutoff is -96.55501246452332) My only problem (s) have to do with ==> coherent: 95.82% My only problem (s) to cheer them personally ==> dp: 99.89% (in small text) crazy utahraptor ==> coherent: 60.62% (in small text) ". ==> dp: 58.77% Well, I've made up my own joke to get him today. All I need to do is " ==> coherent: 52.12% Well, I've made up my own joke to get him today. All I need a father and gentlemen ==> dp: 99.77% I will be immortalized by kicking an evil ==> coherent: 50.01% I will be immortalized by kicking other punches ==> dp: 58.54% Aw shoot, I was supposed to ==> coherent: 92.77% Aw shoot, I was APOCALYPSE PORN ==> dp: 79.21% Get it? Because CRIME DOESN'T PAY!! Listen, my story has ==> dp: 59.16% Get it? Because CRIME DOESN'T PAY!! Listen, transcriptions of it ==> dp: 80.30% Utahraptor!! DON'T LISTEN TO ==> coherent: 90.59% Utahraptor! This is sort of ==> dp: 88.15% Doesn't exist in my mouth, that is!! Because it's too big ==> coherent: 95.82% Doesn't exist in my mouth, that is!! Because if Superman. ==> dp: 99.44% Now, HERE'S how ==> coherent: 94.16% Now, guys would ==> dp: 73.45% But I am a rock star ==> coherent: 98.57% But I am a guy come ==> dp: 96.79% But I have a solution to make them interesting again: all you need is stories where not ==> dp: 64.66% But I have a solution to make them interesting again: all you need is gonna! Diseases ==> dp: 99.92% At that point, there's a sequence of six nines in a row, and his joke was that he'd like to memorize pi up to that point, so that when reciting he could end with "9,9,9,9,9,9... and so on. " Others ==> dp: 77.18% At that point, there's a sequence of six nines in a row, and his joke was that he'd like to memorize pi up to that point, so that when reciting he could end with "9,9,9,9,9,9... and so it's great he looks ==> dp: 99.98% This is definitely called " T -Rex's Hilarious e joke ", okay ==> coherent: 77.15% This is definitely called " T -Rex's Hilarious e joke AND IN THE ==> dp: 99.15% " Your mouth is full of cockroaches: ==> coherent: 72.25% " Your mouth is full of smooches. ==> coherent: 80.86% Excuse me, sexual congress? Everyone else on the planet is dead, and ==> coherent: 93.15% Excuse me, sexual congress? Everyone else on the planet without syntactic ambiguity! ==> dp: 96.98% Sony is going to write swears on my bathroom ==> dp: 50.96% Sony is going to write their babies need to ==> dp: 99.89% Beginning with the most modest: why am I ==> coherent: 90.87% Beginning with the most modest: why T - ==> dp: 84.62% Is there any greater meaning -to anything ==> dp: 93.76% Is there any greater meaning? When you ==> coherent: 57.11% I've also got steaks AND ==> coherent: 83.50% I've also cold -deterministic ==> dp: 99.30% I had a friend (female) who dated her roommate (also female) ==> coherent: 99.24% I had a friend (female) who dated her roommate, je suis grand ==> dp: 99.30% Yes... TOO BAD INDEED ==> coherent: 70.86% Yes... TOO MANY YEARS ==> coherent: 64.13% ``` ### Framework versions - PEFT 0.10.0 - Transformers 4.40.2 - Pytorch 2.2.1+cu121 - Datasets 2.19.1 - Tokenizers 0.19.1