--- license: apache-2.0 model-index: - name: Soniox-7B-v1.0 results: - task: type: text-generation name: Text Generation dataset: name: AI2 Reasoning Challenge (25-Shot) type: ai2_arc config: ARC-Challenge split: test args: num_few_shot: 25 metrics: - type: acc_norm value: 63.91 name: normalized accuracy source: url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=soniox/Soniox-7B-v1.0 name: Open LLM Leaderboard - task: type: text-generation name: Text Generation dataset: name: HellaSwag (10-Shot) type: hellaswag split: validation args: num_few_shot: 10 metrics: - type: acc_norm value: 82.55 name: normalized accuracy source: url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=soniox/Soniox-7B-v1.0 name: Open LLM Leaderboard - task: type: text-generation name: Text Generation dataset: name: MMLU (5-Shot) type: cais/mmlu config: all split: test args: num_few_shot: 5 metrics: - type: acc value: 64.38 name: accuracy source: url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=soniox/Soniox-7B-v1.0 name: Open LLM Leaderboard - task: type: text-generation name: Text Generation dataset: name: TruthfulQA (0-shot) type: truthful_qa config: multiple_choice split: validation args: num_few_shot: 0 metrics: - type: mc2 value: 53.84 source: url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=soniox/Soniox-7B-v1.0 name: Open LLM Leaderboard - task: type: text-generation name: Text Generation dataset: name: Winogrande (5-shot) type: winogrande config: winogrande_xl split: validation args: num_few_shot: 5 metrics: - type: acc value: 78.06 name: accuracy source: url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=soniox/Soniox-7B-v1.0 name: Open LLM Leaderboard - task: type: text-generation name: Text Generation dataset: name: GSM8k (5-shot) type: gsm8k config: main split: test args: num_few_shot: 5 metrics: - type: acc value: 56.25 name: accuracy source: url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=soniox/Soniox-7B-v1.0 name: Open LLM Leaderboard --- # Model Card for Soniox-7B-v1.0 Soniox 7B is a powerful large language model. Supports English and code with 8K context. Matches GPT-4 performance on some benchmarks. Built on top of Mistral 7B, enhanced with additional pre-training and fine-tuning for strong problem-solving capabilities. Apache 2.0 License. For more details, please read our [blog post](https://soniox.com/news/soniox-7B). ## Usage in Transformers The model is available in transformers and can be used as follows: ```python import torch from transformers import AutoModelForCausalLM, AutoTokenizer model_path = "soniox/Soniox-7B-v1.0" model = AutoModelForCausalLM.from_pretrained(model_path, torch_dtype=torch.float16) tokenizer = AutoTokenizer.from_pretrained(model_path) device = "cuda" model.to(device) messages = [ {"role": "user", "content": "12 plus 21?"}, {"role": "assistant", "content": "33."}, {"role": "user", "content": "Five minus one?"}, ] tok_prompt = tokenizer.apply_chat_template(messages, return_tensors="pt") model_inputs = tok_prompt.to(device) generated_ids = model.generate(model_inputs, max_new_tokens=1000, do_sample=True) decoded = tokenizer.batch_decode(generated_ids) print(decoded[0]) ``` ## Inference deployment Refer to our [documentation](https://docs.soniox.com) for inference with vLLM and other deployment options. # [Open LLM Leaderboard Evaluation Results](https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard) Detailed results can be found [here](https://huggingface.co/datasets/open-llm-leaderboard/details_soniox__Soniox-7B-v1.0) | Metric |Value| |---------------------------------|----:| |Avg. |66.50| |AI2 Reasoning Challenge (25-Shot)|63.91| |HellaSwag (10-Shot) |82.55| |MMLU (5-Shot) |64.38| |TruthfulQA (0-shot) |53.84| |Winogrande (5-shot) |78.06| |GSM8k (5-shot) |56.25|