--- language: - en - pl pipeline_tag: text-generation inference: false tags: - voicelab - pytorch - llama-2 - trurl - trurl-2 --- logo voicelab nlp # Trurl 2 -- Polish Llama 2 The new OPEN TRURL is a finetuned Llama 2, trained on over 1.7b tokens (970k conversational **Polish** and **English** samples) with a large context of 4096 tokens. TRURL was trained on a large number of Polish data. TRURL 2 is a collection of fine-tuned generative text models with 7 billion and 13 billion parameters. This is the repository for the 7b fine-tuned model, optimized for dialogue use cases. # Overview **TRURL developers** Voicelab.AI **Variations** Trurl 2 comes in 7B and 13B versions. **Input** Models input text only. **Output** Models generate text only. **Model Architecture** Trurl is an auto-regressive language model that uses an optimized transformer architecture. ||Training Data|Params|Content Length|Num. Samples|Num. Tokens|start LR| |---|---|---|---|---|---|---| |Trurl 2|*A new mix of private and publicly available online data*|7B|4k|970k|1.7b|2.0 x 10-5| |Trurl 2|*A new mix of private and publicly available online data*|13B|4k|970k|1.7b|2.0 x 10-5| ## Training data The training data includes Q&A pairs from various sources including Alpaca comparison data with GPT, Falcon comparison data, Dolly 15k, Oasst1, Phu saferlfhf, ShareGPT version 2023.05.08v0 filtered and cleaned, Voicelab private datasets for JSON data extraction, modification, and analysis, CURLICAT dataset containing journal entries, dataset from Polish wiki with Q&A pairs grouped into conversations, MMLU data in textual format, Voicelab private dataset with sales conversations, arguments and objections, paraphrases, contact reason detection, and corrected dialogues. ## Intended Use Trurl 2 is intended for commercial and research use in Polish and English. Tuned models are intended for assistant-like chat, but also adapted for a variety of natural language generation tasks. To get the expected features and performance for the chat versions, a specific Llama 2 formatting needs to be followed, including the `INST` and `<>` tags, `BOS` and `EOS` tokens, and the whitespaces and breaklines in between (we recommend calling `strip()` on inputs to avoid double-spaces). See our reference code in github for details: [`chat_completion`](https://github.com/facebookresearch/llama/blob/main/llama/generation.py#L212). # Evaluation Results |Model | Size| hellaswag | arc_challenge | MMLU| |---|---|---|---|---| | Llama-2-chat | 7B | 78.55% | 52.9% | 48.32% | | Llama-2-chat | 13B | 81.94% | 59.04% | 54.64% | | Trurl 2.0 (with MMLU) | 13B | 80.09% | 59.30% | 78.35% | | Trurl 2.0 (no MMLU) | 13B | TO-DO | TO-DO | TO-DO| | Trurl 2.0 (no MMLU) | 7b | TO-DO | TO-DO | TO-DO| trurl graphic # Ethical Considerations and Limitations Trurl 2, same as a Llama 2, is a new technology that carries risks with use. Testing conducted to date has been in Polish and English, and has not covered, nor could it cover all scenarios. For these reasons, as with all LLMs, Trurl 2’s potential outputs cannot be predicted in advance, and the model may in some instances produce inaccurate, biased or other objectionable responses to user prompts. Therefore, before deploying any applications of Trurl 2, developers should perform safety testing and tuning tailored to their specific applications of the model. Please see the Meta's Responsible Use Guide available at [https://ai.meta.com/llama/responsible-use-guide/](https://ai.meta.com/llama/responsible-use-guide) # Authors The model was trained by NLP Research Team at Voicelab.ai. You can contact us [here](https://voicelab.ai/contact/). * [TRURL 13b](https://huggingface.co/Voicelab/trurl-2-13b/) * [TRURL 7b](https://huggingface.co/Voicelab/trurl-2-7b/) * [TRURL DEMO](https://trurl.ai)