Upload 11 files

models/TinyLlama_TinyLlama-1.1B-Chat-v1.0/README.md

@@ -0,0 +1,61 @@
+---
+license: apache-2.0
+datasets:
+- cerebras/SlimPajama-627B
+- bigcode/starcoderdata
+- HuggingFaceH4/ultrachat_200k
+- HuggingFaceH4/ultrafeedback_binarized
+language:
+- en
+widget:
+- text: "<|system|>\nYou are a chatbot who can help code!</s>\n<|user|>\nWrite me a function to calculate the first 10 digits of the fibonacci sequence in Python and print it out to the CLI.</s>\n<|assistant|>\n"
+---
+<div align="center">
+
+# TinyLlama-1.1B
+</div>
+
+https://github.com/jzhang38/TinyLlama
+
+The TinyLlama project aims to **pretrain** a **1.1B Llama model on 3 trillion tokens**. With some proper optimization, we can achieve this within a span of "just" 90 days using 16 A100-40G GPUs 🚀🚀. The training has started on 2023-09-01. 
+
+
+We adopted exactly the same architecture and tokenizer as Llama 2. This means TinyLlama can be plugged and played in many open-source projects built upon Llama. Besides, TinyLlama is compact with only 1.1B parameters. This compactness allows it to cater to a multitude of applications demanding a restricted computation and memory footprint.
+
+#### This Model
+This is the chat model finetuned on top of [TinyLlama/TinyLlama-1.1B-intermediate-step-1431k-3T](https://huggingface.co/TinyLlama/TinyLlama-1.1B-intermediate-step-1431k-3T). **We follow [HF's Zephyr](https://huggingface.co/HuggingFaceH4/zephyr-7b-alpha/edit/main/README.md)'s training recipe.** The model was " initially fine-tuned on a variant of the [`UltraChat`](https://huggingface.co/datasets/stingning/ultrachat) dataset, which contains a diverse range of synthetic dialogues generated by ChatGPT. 
+We then further aligned the model with [🤗 TRL's](https://github.com/huggingface/trl) `DPOTrainer` on the [openbmb/UltraFeedback](https://huggingface.co/datasets/openbmb/UltraFeedback) dataset, which contain 64k prompts and model completions that are ranked by GPT-4." 
+
+
+#### How to use
+You will need the transformers>=4.34
+Do check the [TinyLlama](https://github.com/jzhang38/TinyLlama) github page for more information.
+
+```python
+# Install transformers from source - only needed for versions <= v4.34
+# pip install git+https://github.com/huggingface/transformers.git
+# pip install accelerate
+
+import torch
+from transformers import pipeline
+
+pipe = pipeline("text-generation", model="TinyLlama/TinyLlama-1.1B-Chat-v1.0", torch_dtype=torch.bfloat16, device_map="auto")
+
+# We use the tokenizer's chat template to format each message - see https://huggingface.co/docs/transformers/main/en/chat_templating
+messages = [
+    {
+        "role": "system",
+        "content": "You are a friendly chatbot who always responds in the style of a pirate",
+    },
+    {"role": "user", "content": "How many helicopters can a human eat in one sitting?"},
+]
+prompt = pipe.tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
+outputs = pipe(prompt, max_new_tokens=256, do_sample=True, temperature=0.7, top_k=50, top_p=0.95)
+print(outputs[0]["generated_text"])
+# <|system|>
+# You are a friendly chatbot who always responds in the style of a pirate.</s>
+# <|user|>
+# How many helicopters can a human eat in one sitting?</s>
+# <|assistant|>
+# ...
+```

models/TinyLlama_TinyLlama-1.1B-Chat-v1.0/config.json

@@ -0,0 +1,26 @@
+{
+  "architectures": [
+    "LlamaForCausalLM"
+  ],
+  "attention_bias": false,
+  "bos_token_id": 1,
+  "eos_token_id": 2,
+  "hidden_act": "silu",
+  "hidden_size": 2048,
+  "initializer_range": 0.02,
+  "intermediate_size": 5632,
+  "max_position_embeddings": 2048,
+  "model_type": "llama",
+  "num_attention_heads": 32,
+  "num_hidden_layers": 22,
+  "num_key_value_heads": 4,
+  "pretraining_tp": 1,
+  "rms_norm_eps": 1e-05,
+  "rope_scaling": null,
+  "rope_theta": 10000.0,
+  "tie_word_embeddings": false,
+  "torch_dtype": "bfloat16",
+  "transformers_version": "4.35.0",
+  "use_cache": true,
+  "vocab_size": 32000
+}

models/TinyLlama_TinyLlama-1.1B-Chat-v1.0/eval_results.json

@@ -0,0 +1,16 @@
+{
+    "epoch": 3.0,
+    "eval_logits/chosen": -2.707406759262085,
+    "eval_logits/rejected": -2.656524419784546,
+    "eval_logps/chosen": -370.1297607421875,
+    "eval_logps/rejected": -296.0738525390625,
+    "eval_loss": 0.513750433921814,
+    "eval_rewards/accuracies": 0.738095223903656,
+    "eval_rewards/chosen": -0.02744222804903984,
+    "eval_rewards/margins": 1.0087225437164307,
+    "eval_rewards/rejected": -1.03616464138031,
+    "eval_runtime": 93.5908,
+    "eval_samples": 2000,
+    "eval_samples_per_second": 21.37,
+    "eval_steps_per_second": 0.673
+}

models/TinyLlama_TinyLlama-1.1B-Chat-v1.0/generation_config.json

@@ -0,0 +1,7 @@
+{
+  "bos_token_id": 1,
+  "eos_token_id": 2,
+  "max_length": 2048,
+  "pad_token_id": 0,
+  "transformers_version": "4.35.0"
+}

models/TinyLlama_TinyLlama-1.1B-Chat-v1.0/huggingface-metadata.txt

@@ -0,0 +1,6 @@
+url: https://huggingface.co/TinyLlama/TinyLlama-1.1B-Chat-v1.0
+branch: main
+download date: 2024-01-04 21:10:57
+sha256sum:
+    6e6001da2106d4757498752a021df6c2bdc332c650aae4bae6b0c004dcf14933 model.safetensors
+    9e556afd44213b6bd1be2b850ebbbd98f5481437a8021afaf58ee7fb1818d347 tokenizer.model

models/TinyLlama_TinyLlama-1.1B-Chat-v1.0/special_tokens_map.json

@@ -0,0 +1,30 @@
+{
+  "bos_token": {
+    "content": "<s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "</s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "</s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "unk_token": {
+    "content": "<unk>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

models/TinyLlama_TinyLlama-1.1B-Chat-v1.0/tokenizer_config.json

@@ -0,0 +1,40 @@
+{
+  "added_tokens_decoder": {
+    "0": {
+      "content": "<unk>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "1": {
+      "content": "<s>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "2": {
+      "content": "</s>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "bos_token": "<s>",
+  "chat_template": "{% for message in messages %}\n{% if message['role'] == 'user' %}\n{{ '<|user|>\n' + message['content'] + eos_token }}\n{% elif message['role'] == 'system' %}\n{{ '<|system|>\n' + message['content'] + eos_token }}\n{% elif message['role'] == 'assistant' %}\n{{ '<|assistant|>\n'  + message['content'] + eos_token }}\n{% endif %}\n{% if loop.last and add_generation_prompt %}\n{{ '<|assistant|>' }}\n{% endif %}\n{% endfor %}",
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "</s>",
+  "legacy": false,
+  "model_max_length": 2048,
+  "pad_token": "</s>",
+  "padding_side": "right",
+  "sp_model_kwargs": {},
+  "tokenizer_class": "LlamaTokenizer",
+  "unk_token": "<unk>",
+  "use_default_system_prompt": false
+}

models/config-user.yaml

@@ -0,0 +1,20 @@
+TinyLlama_TinyLlama-1.1B-Chat-v1.0$:
+  loader: Transformers
+  cpu_memory: 2048
+  auto_devices: false
+  disk: false
+  cpu: true
+  bf16: true
+  load_in_8bit: false
+  trust_remote_code: false
+  no_use_fast: false
+  use_flash_attention_2: false
+  load_in_4bit: false
+  compute_dtype: bfloat16
+  quant_type: fp4
+  use_double_quant: false
+  disable_exllama: false
+  disable_exllamav2: false
+  compress_pos_emb: 1
+  alpha_value: 1
+  rope_freq_base: 0

models/config.yaml

@@ -0,0 +1,192 @@
+.*(llama|alpac|vicuna|guanaco|koala|llava|wizardlm|metharme|pygmalion-7b|pygmalion-2|mythalion|wizard-mega|openbuddy|vigogne|h2ogpt-research|manticore):
+  model_type: 'llama'
+.*(opt-|opt_|opt1|opt3|optfor|galactica|galpaca|pygmalion-350m):
+  model_type: 'opt'
+.*(gpt-j|gptj|gpt4all-j|malion-6b|pygway|pygmalion-6b|dolly-v1):
+  model_type: 'gptj'
+.*(gpt-neox|koalpaca-polyglot|polyglot.*koalpaca|polyglot-ko|polyglot_ko|pythia|stablelm|incite|dolly-v2|polycoder|h2ogpt-oig|h2ogpt-oasst1|h2ogpt-gm):
+  model_type: 'gptneox'
+.*bloom:
+  model_type: 'bloom'
+.*gpt2:
+  model_type: 'gpt2'
+.*falcon:
+  model_type: 'falcon'
+.*mpt:
+  model_type: 'mpt'
+.*(starcoder|starchat):
+  model_type: 'starcoder'
+.*dolly-v2:
+  model_type: 'dollyv2'
+.*replit:
+  model_type: 'replit'
+.*(oasst|openassistant-|stablelm-7b-sft-v7-epoch-3):
+  instruction_template: 'Open Assistant'
+  skip_special_tokens: false
+(?!.*galactica)(?!.*reward).*openassistant:
+  instruction_template: 'Open Assistant'
+  skip_special_tokens: false
+.*galactica:
+  skip_special_tokens: false
+.*dolly-v[0-9]-[0-9]*b:
+  instruction_template: 'Alpaca'
+  skip_special_tokens: false
+.*alpaca-native-4bit:
+  instruction_template: 'Alpaca'
+  custom_stopping_strings: '"### End"'
+.*llava:
+  instruction_template: 'LLaVA'
+  custom_stopping_strings: '"\n###"'
+.*llava.*1.5:
+  instruction_template: 'Vicuna-v1.1'
+.*wizard.*mega:
+  instruction_template: 'Wizard-Mega'
+  custom_stopping_strings: '"</s>"'
+.*starchat-beta:
+  instruction_template: 'Starchat-Beta'
+  custom_stopping_strings: '"<|end|>"'
+(?!.*v0)(?!.*1.1)(?!.*1_1)(?!.*stable)(?!.*chinese).*vicuna:
+  instruction_template: 'Vicuna-v0'
+.*vicuna.*v0:
+  instruction_template: 'Vicuna-v0'
+.*vicuna.*(1.1|1_1|1.3|1_3):
+  instruction_template: 'Vicuna-v1.1'
+.*vicuna.*(1.5|1_5):
+  instruction_template: 'Vicuna-v1.1'
+.*stable.*vicuna:
+  instruction_template: 'StableVicuna'
+(?!.*chat).*chinese-vicuna:
+  instruction_template: 'Alpaca'
+.*chinese-vicuna.*chat:
+  instruction_template: 'Chinese-Vicuna-Chat'
+.*alpaca:
+  instruction_template: 'Alpaca'
+.*koala:
+  instruction_template: 'Koala'
+.*chatglm:
+  instruction_template: 'ChatGLM'
+.*(metharme|pygmalion|mythalion):
+  instruction_template: 'Metharme'
+.*raven:
+  instruction_template: 'RWKV-Raven'
+.*moss-moon.*sft:
+  instruction_template: 'MOSS'
+.*stablelm-tuned:
+  instruction_template: 'StableLM'
+.*galactica.*finetuned:
+  instruction_template: 'Galactica Finetuned'
+.*galactica.*-v2:
+  instruction_template: 'Galactica v2'
+(?!.*finetuned)(?!.*-v2).*galactica:
+  instruction_template: 'Galactica'
+.*guanaco:
+  instruction_template: 'Guanaco non-chat'
+.*baize:
+  instruction_template: 'Baize'
+.*mpt-.*instruct:
+  instruction_template: 'Alpaca'
+.*mpt-.*chat:
+  instruction_template: 'ChatML'
+(?!.*-flan-)(?!.*-t5-).*lamini-:
+  instruction_template: 'Alpaca'
+.*incite.*chat:
+  instruction_template: 'INCITE-Chat'
+.*incite.*instruct:
+  instruction_template: 'INCITE-Instruct'
+.*ziya-:
+  instruction_template: 'Ziya'
+.*koalpaca:
+  instruction_template: 'KoAlpaca'
+.*openbuddy:
+  instruction_template: 'OpenBuddy'
+(?!.*chat).*vigogne:
+  instruction_template: 'Vigogne-Instruct'
+.*vigogne.*chat:
+  instruction_template: 'Vigogne-Chat'
+.*(llama-deus|supercot|llama-natural-instructions|open-llama-0.3t-7b-instruct-dolly-hhrlhf|open-llama-0.3t-7b-open-instruct):
+  instruction_template: 'Alpaca'
+.*bactrian:
+  instruction_template: 'Bactrian'
+.*(h2ogpt-oig-|h2ogpt-oasst1-|h2ogpt-research-oasst1-):
+  instruction_template: 'INCITE-Chat'
+.*h2ogpt-gm-:
+  instruction_template: 'H2O-prompt_answer'
+.*manticore:
+  instruction_template: 'Manticore Chat'
+.*bluemoonrp-(30|13)b:
+  instruction_template: 'Bluemoon'
+.*Nous-Hermes-13b:
+  instruction_template: 'Alpaca'
+.*airoboros:
+  instruction_template: 'Vicuna-v1.1'
+.*airoboros.*1.2:
+  instruction_template: 'Airoboros-v1.2'
+.*alpa(cino|sta):
+  instruction_template: 'Alpaca'
+.*hippogriff:
+  instruction_template: 'Hippogriff'
+.*lazarus:
+  instruction_template: 'Alpaca'
+.*guanaco-.*(7|13|33|65)b:
+  instruction_template: 'Vicuna-v0'
+.*hypermantis:
+  instruction_template: 'Alpaca'
+.*open-llama-.*-open-instruct:
+  instruction_template: 'Alpaca'
+.*starcoder-gpteacher-code-instruct:
+  instruction_template: 'Alpaca'
+.*tulu:
+  instruction_template: 'Tulu'
+.*chronos:
+  instruction_template: 'Alpaca'
+.*samantha:
+  instruction_template: 'Samantha'
+.*wizardcoder:
+  instruction_template: 'Alpaca'
+.*minotaur:
+  instruction_template: 'Manticore Chat'
+.*orca_mini:
+  instruction_template: 'Orca Mini'
+.*(platypus|gplatty|superplatty):
+  instruction_template: 'Alpaca'
+.*(openorca-platypus2):
+  instruction_template: 'OpenOrca-Platypus2'
+  custom_stopping_strings: '"### Instruction:", "### Response:"'
+.*longchat:
+  instruction_template: 'Vicuna-v1.1'
+.*vicuna-33b:
+  instruction_template: 'Vicuna-v1.1'
+.*redmond-hermes-coder:
+  instruction_template: 'Alpaca'
+.*wizardcoder-15b:
+  instruction_template: 'Alpaca'
+.*wizardlm:
+  instruction_template: 'Vicuna-v1.1'
+.*godzilla:
+  instruction_template: 'Alpaca'
+.*llama(-?)(2|v2).*chat:
+  instruction_template: 'Llama-v2'
+.*newhope:
+  instruction_template: 'NewHope'
+.*stablebeluga2:
+  instruction_template: 'StableBeluga2'
+.*openchat:
+  instruction_template: 'OpenChat'
+.*codellama.*instruct:
+  instruction_template: 'Llama-v2'
+.*(mistral|mixtral).*instruct:
+  instruction_template: 'Mistral'
+.*mistral.*openorca:
+  instruction_template: 'ChatML'
+.*(WizardCoder-Python-34B-V1.0|Phind-CodeLlama-34B-v2|CodeBooga-34B-v0.1):
+  instruction_template: 'Alpaca'
+.*orca-2-(13|7)b:
+  instruction_template: 'ChatML'
+.*openhermes.*mistral:
+  instruction_template: 'ChatML'
+.*Yi-34B-Chat:
+  instruction_template: 'ChatML'
+(dolphin).*:
+  instruction_template: 'ChatML'
+.*synthia:
+  instruction_template: 'Synthia'