--- language: - en tags: - falcon3 --- # Falcon3-7B-Instruct **Falcon3** family of Open Foundation Models is a set of pretrained and instruct LLMs ranging from 1B to 10B. This repository contains the **Falcon3-7B-Instruct**. It achieves state of art results (at release's time) on reasoning, language understanding, instruction following, code and mathematics tasks. Falcon3-7B-Instruct supports 4 languages (english, french, spanish, portuguese) and a context length up to 32K. ## Model Details - Architecture - transformer based causal decoder only architecture - 28 decoder blocks - grouped query attention (GQA) for faster inference: 12 query heads and 4 KV heads - wider head dimension: 256 - high RoPE value to support long context understanding: 1000042 - 32k context length - 131k vocab size - Pretrained on 14 Gigatokens of datasets comprising of web, code, STEM, high quality and mutlilingual data using 2048 H100 GPU chips - Postrained on 1.2 million samples of STEM, conversations, code, safety and function call data - Supports EN, FR, ES, PT - Developed by [Technology Innovation Institute](https://www.tii.ae) - License: TII Falcon-LLM License 2.0 - Model Release Date: December 2024 ## Getting started
Click to expand ```python from transformers import AutoTokenizer, AutoModelForCausalLM from transformers import AutoModelForCausalLM, AutoTokenizer model_name = "tiiuae/Falcon3-7B-Instruct" model = AutoModelForCausalLM.from_pretrained( model_name, torch_dtype="auto", device_map="auto" ) tokenizer = AutoTokenizer.from_pretrained(model_name) prompt = "How many hours in one day?" messages = [ {"role": "system", "content": "You are a helpful friendly assistant Falcon3 from TII, try to follow instructions as much as possible."}, {"role": "user", "content": prompt} ] text = tokenizer.apply_chat_template( messages, tokenize=False, add_generation_prompt=True ) model_inputs = tokenizer([text], return_tensors="pt").to(model.device) generated_ids = model.generate( **model_inputs, max_new_tokens=1024 ) generated_ids = [ output_ids[len(input_ids):] for input_ids, output_ids in zip(model_inputs.input_ids, generated_ids) ] response = tokenizer.batch_decode(generated_ids, skip_special_tokens=True)[0] print(response) ```

# Benchmarks We report in the following table our internal pipeline benchmarks:
Category Benchmark Llama-3.1-8B-Instruct Qwen2-7B-Instruct Qwen2.5-7B-Instruct gemma-2-9b-it Falcon3-7B-Instruct
General MMLU (5-shot) - - - - -
MMLU-PRO (5-shot) - - - - -
IFEval - - - - -
Math GSM8K (5-shot) - - - - -
MATH(4-shot) - - - - -
Reasoning Arc Challenge (25-shot) - - - - -
GPQA (0-shot) - - - - -
MUSR (0-shot) - - - - -
BBH (3-shot) - - - - -
CommonSense Understanding PIQA (0-shot) - - - - -
SciQ (0-shot) - - - - -
Winogrande (0-shot) - - - - -
OpenbookQA (0-shot) - - - - -
# Citation If Falcon3 family were helpful to your work, feel free to give us a cite. ``` @misc{Falcon3, title = {Falcon 3 family of Open Foundation Models}, author = {TII Team}, month = {December}, year = {2024} } ```