--- license: cc-by-nc-4.0 tags: - merge model-index: - name: Lonepino-11B results: - task: type: text-generation name: Text Generation dataset: name: AI2 Reasoning Challenge (25-Shot) type: ai2_arc config: ARC-Challenge split: test args: num_few_shot: 25 metrics: - type: acc_norm value: 68.26 name: normalized accuracy source: url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=beberik/Lonepino-11B name: Open LLM Leaderboard - task: type: text-generation name: Text Generation dataset: name: HellaSwag (10-Shot) type: hellaswag split: validation args: num_few_shot: 10 metrics: - type: acc_norm value: 84.57 name: normalized accuracy source: url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=beberik/Lonepino-11B name: Open LLM Leaderboard - task: type: text-generation name: Text Generation dataset: name: MMLU (5-Shot) type: cais/mmlu config: all split: test args: num_few_shot: 5 metrics: - type: acc value: 63.76 name: accuracy source: url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=beberik/Lonepino-11B name: Open LLM Leaderboard - task: type: text-generation name: Text Generation dataset: name: TruthfulQA (0-shot) type: truthful_qa config: multiple_choice split: validation args: num_few_shot: 0 metrics: - type: mc2 value: 63.45 source: url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=beberik/Lonepino-11B name: Open LLM Leaderboard - task: type: text-generation name: Text Generation dataset: name: Winogrande (5-shot) type: winogrande config: winogrande_xl split: validation args: num_few_shot: 5 metrics: - type: acc value: 78.93 name: accuracy source: url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=beberik/Lonepino-11B name: Open LLM Leaderboard - task: type: text-generation name: Text Generation dataset: name: GSM8k (5-shot) type: gsm8k config: main split: test args: num_few_shot: 5 metrics: - type: acc value: 61.64 name: accuracy source: url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=beberik/Lonepino-11B name: Open LLM Leaderboard --- ## Description This repo contains bf16 files of Lonepino-11B. Just a normal model. ## Model used - [Intel/neural-chat-7b-v3-3-Slerp](https://huggingface.co/Intel/neural-chat-7b-v3-3-Slerp) - [NeverSleep/Noromaid-7b-v0.2](https://huggingface.co/NeverSleep/Noromaid-7b-v0.2) - [chargoddard/loyal-piano-m7-cdpo](https://huggingface.co/chargoddard/loyal-piano-m7-cdpo) - [maywell/PiVoT-0.1-Starling-LM-RP](https://huggingface.co/maywell/PiVoT-0.1-Starling-LM-RP) ## The secret sauce neural-maid-11B: ``` slices: - sources: - model: Intel/neural-chat-7b-v3-3-Slerp layer_range: [0, 24] - sources: - model: NeverSleep/Noromaid-7b-v0.2 layer_range: [8, 32] merge_method: passthrough dtype: bfloat16 ``` loyal-PiVoT-11B: ``` slices: - sources: - model: chargoddard/loyal-piano-m7-cdpo layer_range: [0, 24] - sources: - model: maywell/PiVoT-0.1-Starling-LM-RP layer_range: [8, 32] merge_method: passthrough dtype: bfloat16 ``` Lonepino-11B: ``` slices: - sources: - model: "./neural-maid-11B" layer_range: [0, 48] - model: "./loyal-PiVoT-11B" layer_range: [0, 48] merge_method: slerp base_model: "./neural-maid-11B" parameters: t: - value: 0.4 dtype: bfloat16 ``` ## Prompt template Alpaca. Or chatml. Or any you like. =w= I use [mergekit](https://github.com/cg123/mergekit) for all the manipulation told here. Thanks to the [Undi95](https://huggingface.co/Undi95) for the original [11B mistral merge](https://huggingface.co/Undi95/Mistral-11B-OmniMix) recipe. # [Open LLM Leaderboard Evaluation Results](https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard) Detailed results can be found [here](https://huggingface.co/datasets/open-llm-leaderboard/details_beberik__Lonepino-11B) | Metric |Value| |---------------------------------|----:| |Avg. |70.10| |AI2 Reasoning Challenge (25-Shot)|68.26| |HellaSwag (10-Shot) |84.57| |MMLU (5-Shot) |63.76| |TruthfulQA (0-shot) |63.45| |Winogrande (5-shot) |78.93| |GSM8k (5-shot) |61.64|