Quantization made by Richard Erkhov.
mistral_tv-neural-marconroni - GGUF
- Model creator: https://huggingface.co/aqweteddy/
- Original model: https://huggingface.co/aqweteddy/mistral_tv-neural-marconroni/
Original model description:
language: - en license: mit model-index: - name: mistral_tv-neural-marconroni results: - task: type: text-generation name: Text Generation dataset: name: AI2 Reasoning Challenge (25-Shot) type: ai2_arc config: ARC-Challenge split: test args: num_few_shot: 25 metrics: - type: acc_norm value: 69.2 name: normalized accuracy source: url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=aqweteddy/mistral_tv-neural-marconroni name: Open LLM Leaderboard - task: type: text-generation name: Text Generation dataset: name: HellaSwag (10-Shot) type: hellaswag split: validation args: num_few_shot: 10 metrics: - type: acc_norm value: 86.26 name: normalized accuracy source: url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=aqweteddy/mistral_tv-neural-marconroni name: Open LLM Leaderboard - task: type: text-generation name: Text Generation dataset: name: MMLU (5-Shot) type: cais/mmlu config: all split: test args: num_few_shot: 5 metrics: - type: acc value: 65.07 name: accuracy source: url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=aqweteddy/mistral_tv-neural-marconroni name: Open LLM Leaderboard - task: type: text-generation name: Text Generation dataset: name: TruthfulQA (0-shot) type: truthful_qa config: multiple_choice split: validation args: num_few_shot: 0 metrics: - type: mc2 value: 60.03 source: url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=aqweteddy/mistral_tv-neural-marconroni name: Open LLM Leaderboard - task: type: text-generation name: Text Generation dataset: name: Winogrande (5-shot) type: winogrande config: winogrande_xl split: validation args: num_few_shot: 5 metrics: - type: acc value: 80.9 name: accuracy source: url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=aqweteddy/mistral_tv-neural-marconroni name: Open LLM Leaderboard - task: type: text-generation name: Text Generation dataset: name: GSM8k (5-shot) type: gsm8k config: main split: test args: num_few_shot: 5 metrics: - type: acc value: 66.19 name: accuracy source: url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=aqweteddy/mistral_tv-neural-marconroni name: Open LLM Leaderboard
Chat Vector
CHAT VECTOR: A SIMPLE APPROACH TO EQUIP LLMS WITH NEW LANGUAGE CHAT CAPABILITIES https://arxiv.org/pdf/2310.04799.pdf
With the advancements in conversational AI, such as ChatGPT, this paper focuses on exploring developing Large Language Models (LLMs) for non-English languages, especially emphasizing alignment with human preferences. We introduce a computationally efficient method, leveraging “chat vector,” to synergize pre-existing knowledge and behaviors in LLMs, restructuring the conventional training paradigm from continual pretrain SFT RLHF to continual pretrain + chat. Our empirical studies, primarily focused on Traditional Chinese, employ LLaMA2 as the base model and acquire the chat vector by subtracting the pre-trained weights, LLaMA2, from the weights of LLaMA2-chat. Evaluating from three distinct facets, which are toxicity, ability of instruction following and multi-turn dialogue demonstrates the chat vector's superior efficacy in “chatting”. To confirm the adaptability of our approach, we extend our experiments to include models pre-trained in both Korean and Simplified Chinese, illustrating the versatility of our methodology. Overall, we present a significant solution in aligning LLMs with human preferences efficiently across various languages, accomplished by the chat vector.
Merged LM
- mistral 7b
- chat vector
- neural-chat
- marconroni
Open LLM Leaderboard Evaluation Results
Detailed results can be found here
| Metric | Value |
|---|---|
| Avg. | 71.27 |
| AI2 Reasoning Challenge (25-Shot) | 69.20 |
| HellaSwag (10-Shot) | 86.26 |
| MMLU (5-Shot) | 65.07 |
| TruthfulQA (0-shot) | 60.03 |
| Winogrande (5-shot) | 80.90 |
| GSM8k (5-shot) | 66.19 |
- Downloads last month
- 22