Upload folder using huggingface_hub

.gitattributes

@@ -1,35 +1,9 @@
-*.7z filter=lfs diff=lfs merge=lfs -text
-*.arrow filter=lfs diff=lfs merge=lfs -text
-*.bin filter=lfs diff=lfs merge=lfs -text
-*.bz2 filter=lfs diff=lfs merge=lfs -text
-*.ckpt filter=lfs diff=lfs merge=lfs -text
-*.ftz filter=lfs diff=lfs merge=lfs -text
-*.gz filter=lfs diff=lfs merge=lfs -text
-*.h5 filter=lfs diff=lfs merge=lfs -text
-*.joblib filter=lfs diff=lfs merge=lfs -text
-*.lfs.* filter=lfs diff=lfs merge=lfs -text
-*.mlmodel filter=lfs diff=lfs merge=lfs -text
-*.model filter=lfs diff=lfs merge=lfs -text
-*.msgpack filter=lfs diff=lfs merge=lfs -text
-*.npy filter=lfs diff=lfs merge=lfs -text
-*.npz filter=lfs diff=lfs merge=lfs -text
-*.onnx filter=lfs diff=lfs merge=lfs -text
-*.ot filter=lfs diff=lfs merge=lfs -text
-*.parquet filter=lfs diff=lfs merge=lfs -text
-*.pb filter=lfs diff=lfs merge=lfs -text
-*.pickle filter=lfs diff=lfs merge=lfs -text
-*.pkl filter=lfs diff=lfs merge=lfs -text
-*.pt filter=lfs diff=lfs merge=lfs -text
-*.pth filter=lfs diff=lfs merge=lfs -text
-*.rar filter=lfs diff=lfs merge=lfs -text
-*.safetensors filter=lfs diff=lfs merge=lfs -text
-saved_model/**/* filter=lfs diff=lfs merge=lfs -text
-*.tar.* filter=lfs diff=lfs merge=lfs -text
-*.tar filter=lfs diff=lfs merge=lfs -text
-*.tflite filter=lfs diff=lfs merge=lfs -text
-*.tgz filter=lfs diff=lfs merge=lfs -text
-*.wasm filter=lfs diff=lfs merge=lfs -text
-*.xz filter=lfs diff=lfs merge=lfs -text
-*.zip filter=lfs diff=lfs merge=lfs -text
-*.zst filter=lfs diff=lfs merge=lfs -text
-*tfevents* filter=lfs diff=lfs merge=lfs -text
+limarp-miqu-1-70b-Q2_K.gguf filter=lfs diff=lfs merge=lfs -text
+limarp-miqu-1-70b-Q3_K_L.gguf filter=lfs diff=lfs merge=lfs -text
+limarp-miqu-1-70b-Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text
+limarp-miqu-1-70b-Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text
+limarp-miqu-1-70b-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text
+limarp-miqu-1-70b-Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text
+limarp-miqu-1-70b-Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text
+limarp-miqu-1-70b-Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text
+limarp-miqu-1-70b-Q6_K.gguf filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,188 @@
+---
+library_name: peft
+tags:
+- generated_from_trainer
+- llama
+- llama 2
+model-index:
+- name: volume/limarp-70b-qlora
+  results: []
+datasets:
+- lemonilia/LimaRP
+language:
+- en
+---
+
+[<img src="https://raw.githubusercontent.com/OpenAccess-AI-Collective/axolotl/main/image/axolotl-badge-web.png" alt="Built with Axolotl" width="200" height="32"/>](https://github.com/OpenAccess-AI-Collective/axolotl)
+<details><summary>See axolotl config</summary>
+
+axolotl version: `0.4.0`
+```yaml
+base_model: models/miqu-1-70b-sf
+model_type: LlamaForCausalLM
+tokenizer_type: LlamaTokenizer
+is_llama_derived_model: true
+
+load_in_8bit: false
+load_in_4bit: true
+strict: false
+
+datasets:
+  - path: train-all-max-alpaca-llama.jsonl
+    type: completion
+dataset_prepared_path:
+val_set_size: 0.0
+output_dir: ./volume/limarp-70b-qlora
+
+adapter: qlora
+lora_model_dir:
+
+sequence_len: 16384
+sample_packing: true
+pad_to_sequence_len: true
+
+lora_r: 32
+lora_alpha: 16
+lora_dropout: 0.05
+lora_target_modules:
+lora_target_linear: true
+lora_fan_in_fan_out:
+
+wandb_project: 70b-lora
+wandb_entity:
+wandb_watch:
+wandb_name:
+wandb_log_model:
+
+gradient_accumulation_steps: 4
+micro_batch_size: 1
+num_epochs: 2
+optimizer: adamw_bnb_8bit
+lr_scheduler: cosine
+learning_rate: 0.0001
+
+train_on_inputs: true
+group_by_length: false
+bf16: true
+fp16: false
+tf32: true
+
+gradient_checkpointing: true
+gradient_checkpointing_kwargs:
+  use_reentrant: true
+early_stopping_patience:
+resume_from_checkpoint:
+local_rank:
+logging_steps: 1
+xformers_attention:
+flash_attention: true
+
+warmup_steps: 10
+eval_steps:
+eval_table_size:
+save_steps:
+debug:
+deepspeed:
+weight_decay: 0.0
+fsdp:
+fsdp_config:
+special_tokens:
+  bos_token: "<s>"
+  eos_token: "</s>"
+  unk_token: "<unk>"
+
+```
+
+</details><br>
+
+# limarp-miqu-1-70b-qlora
+
+Experimental limarp qlora trained at 16384 ctx length (greater than size of the longest limarp sample when tokenized via llama's tokenizer) on the fixed dequantized miqu-1-70b model by 152334H.
+
+I wasn't particularly happy with the results I got when I tried applying the lora at varying weights to the miqu-1-70b model. It's possible that this is related to the fact that the model was dequantized from Q5_K_M GGUF, or perhaps due to it already being an instruct-tuned model.
+
+However, I decided to go ahead and release this in case someone else finds a use for it. Provided as-is and YMMV.
+
+## Model description
+
+The intended prompt format is the Alpaca instruction format of LimaRP v3:
+```
+### Instruction:
+Character's Persona: {bot character description}
+
+User's Persona: {user character description}
+
+Scenario: {what happens in the story}
+
+Play the role of Character. Taking the above information into consideration, you must engage in a roleplaying chat with User below this line. Do not write dialogues and narration for User.
+
+### Input:
+User: {utterance}
+
+### Response:
+Character: {utterance}
+
+### Input:
+User: {utterance}
+
+### Response:
+Character: {utterance}
+
+(etc.)
+```
+Inspired by the previously named "Roleplay" preset in SillyTavern, with this version of LimaRP it is possible to append a length modifier to the response instruction sequence, like this:
+
+```
+### Input
+User: {utterance}
+
+### Response: (length = medium)
+Character: {utterance}
+```
+
+This has an immediately noticeable effect on bot responses. The lengths using during training are:
+`micro`, `tiny`, `short`, `medium`, `long`, `massive`, `huge`, `enormous`, `humongous`, `unlimited`.
+**The recommended starting length is medium**. Keep in mind that the AI can ramble or impersonate
+the user with very long messages.
+
+The length control effect is reproducible, but the messages will not necessarily follow
+lengths very precisely, rather follow certain ranges on average, as seen in this table
+with data from tests made with one reply at the beginning of the conversation:
+
+![lengths](https://i.imgur.com/2WXGgaV.png)
+
+Response length control appears to work well also deep into the conversation. **By omitting
+the modifier, the model will choose the most appropriate response length** (although it might
+not necessarily be what the user desires).
+
+## Intended uses & limitations
+
+The model will show biases similar to those observed in niche roleplaying forums on the Internet, besides those exhibited by the base model.
+
+## Training and evaluation data
+
+For more details about LimaRP, see the dataset page.
+
+## Training procedure
+
+### Training hyperparameters
+
+The following hyperparameters were used during training:
+- learning_rate: 0.0001
+- train_batch_size: 1
+- eval_batch_size: 1
+- seed: 42
+- gradient_accumulation_steps: 4
+- total_train_batch_size: 4
+- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
+- lr_scheduler_type: cosine
+- lr_scheduler_warmup_steps: 10
+- num_epochs: 2
+
+### Framework versions
+
+- PEFT 0.7.2.dev0
+- Transformers 4.37.0
+- Pytorch 2.1.2+cu118
+- Datasets 2.16.1
+- Tokenizers 0.15.0

limarp-miqu-1-70b-Q2_K.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:bceb7a840de80cdacf0208be6edbdeadae45a598dba059ffd78383d49e9a42bf
+size 25462437440

limarp-miqu-1-70b-Q3_K_L.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:258c4aa696bd9e97ca9d4bf42ee83c0ed2f808cb17e691cb6bbd843502b974d1
+size 36147836480

limarp-miqu-1-70b-Q3_K_M.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f071405a0cbf3188d2bf13e81b6ce8f763d2559fa4e80addb1a7748c5f3e4c02
+size 33274738240

limarp-miqu-1-70b-Q3_K_S.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:be427b4911c4f3f5f1f43cb22e79cbed51b355e4f9dd744183dbb272b4162d7e
+size 29919295040

limarp-miqu-1-70b-Q4_K_M.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:eda042d9f4325b7dd91e78c8d34eb0c4e4e98882380a11bafa0324d94935b169
+size 41422911040

limarp-miqu-1-70b-Q4_K_S.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b22fd533b6f1fa327aeda461b748badbbfb91f4ecf26037d101f35c718450a85
+size 39249737280

limarp-miqu-1-70b-Q5_K_M.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9bcf8f0805206400b29a5299f385d55252ecde4df6c757c052fba0a6ba86bfd1
+size 48753768000

limarp-miqu-1-70b-Q5_K_S.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0ad90f8cfcbc1a1b4869124a81ad9d50c86e3fb64e84ca5abce3001b040706ea
+size 47461398080

limarp-miqu-1-70b-Q6_K.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1567a0a62df7201dab22d5fcc8e8cd87d2452006ed1181bc4a4f86c0e3d1c040
+size 215840320