--- library_name: transformers tags: - merge - llama-3.1 - roleplay - function calling base_model: - unsloth/Meta-Llama-3.1-8B-Instruct - ergotts/llama_3.1_8b_prop_logic_ft datasets: - ergotts/propositional-logic base_model_relation: merge model-index: - name: KRONOS-8B-V1-P2 results: - task: type: text-generation name: Text Generation dataset: name: IFEval (0-Shot) type: wis-k/instruction-following-eval split: train args: num_few_shot: 0 metrics: - type: inst_level_strict_acc and prompt_level_strict_acc value: 67.24 name: averaged accuracy source: url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard#/?search=T145%2FKRONOS-8B-V1-P2 name: Open LLM Leaderboard - task: type: text-generation name: Text Generation dataset: name: BBH (3-Shot) type: SaylorTwift/bbh split: test args: num_few_shot: 3 metrics: - type: acc_norm value: 25.86 name: normalized accuracy source: url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard#/?search=T145%2FKRONOS-8B-V1-P2 name: Open LLM Leaderboard - task: type: text-generation name: Text Generation dataset: name: MATH Lvl 5 (4-Shot) type: lighteval/MATH-Hard split: test args: num_few_shot: 4 metrics: - type: exact_match value: 12.92 name: exact match source: url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard#/?search=T145%2FKRONOS-8B-V1-P2 name: Open LLM Leaderboard - task: type: text-generation name: Text Generation dataset: name: GPQA (0-shot) type: Idavidrein/gpqa split: train args: num_few_shot: 0 metrics: - type: acc_norm value: 5.59 name: acc_norm source: url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard#/?search=T145%2FKRONOS-8B-V1-P2 name: Open LLM Leaderboard - task: type: text-generation name: Text Generation dataset: name: MuSR (0-shot) type: TAUR-Lab/MuSR args: num_few_shot: 0 metrics: - type: acc_norm value: 5.06 name: acc_norm source: url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard#/?search=T145%2FKRONOS-8B-V1-P2 name: Open LLM Leaderboard - task: type: text-generation name: Text Generation dataset: name: MMLU-PRO (5-shot) type: TIGER-Lab/MMLU-Pro config: main split: test args: num_few_shot: 5 metrics: - type: acc value: 27.26 name: accuracy source: url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard#/?search=T145%2FKRONOS-8B-V1-P2 name: Open LLM Leaderboard --- # KRONOS V1 P2 This is a merge of Meta Llama 3.1 Instruct and the "Propositional Logic" LORA, created using [llm-tools](https://github.com/oobabooga/llm-tools). The primary purpose of this model is to be merged into other models in the same family using the TIES merge method. Creating quants for this is entirely unnecessary. ## Merge Details ### Configuration The following Bash command was used to produce this model: ```bash python /llm-tools/merge-lora.py -m unsloth/Meta-Llama-3.1-8B-Instruct -l ergotts/llama_3.1_8b_prop_logic_ft ``` # [Open LLM Leaderboard Evaluation Results](https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard) Detailed results can be found [here](https://huggingface.co/datasets/open-llm-leaderboard/T145__KRONOS-8B-V1-P2-details)! Summarized results can be found [here](https://huggingface.co/datasets/open-llm-leaderboard/contents/viewer/default/train?q=T145%2FKRONOS-8B-V1-P2&sort[column]=Average%20%E2%AC%86%EF%B8%8F&sort[direction]=desc)! | Metric |Value (%)| |-------------------|--------:| |**Average** | 23.99| |IFEval (0-Shot) | 67.24| |BBH (3-Shot) | 25.86| |MATH Lvl 5 (4-Shot)| 12.92| |GPQA (0-shot) | 5.59| |MuSR (0-shot) | 5.06| |MMLU-PRO (5-shot) | 27.26|