metadata

language:
  - en
license: apache-2.0
tags:
  - text-generation-inference
  - transformers
  - unsloth
  - llama
  - trl
base_model: unsloth/llama-3-8b-Instruct-bnb-4bit

Model: BAI_LLM_FinArg

Developed by: varadsrivastava
License: apache-2.0
Base Model : unsloth/llama-3-8b-Instruct-bnb-4bit

For Proper Inference, please use:

!pip install "unsloth[colab-new] @ git+https://GitHub.com/unslothai/unsloth.git

Loading the fine-tuned model and the tokenizer for inference

from unsloth import FastLanguageModel

model, tokenizer = FastLanguageModel.from_pretrained( model_name = "varadsrivastava/BAI_LLM_FinArg", max_seq_length = 20, dtype = torch.bfloat16, load_in_4bit = True )

Using FastLanguageModel for fast inference

FastLanguageModel.for_inference(model)

Prompt template:

"""<|begin_of_text|><|start_header_id|>system<|end_header_id|>

{instruction}<|eot_id|><|start_header_id|>user<|end_header_id|>

Sentence: {row['text']}<|eot_id|><|start_header_id|>assistant<|end_header_id|>

Class: {row['label']}<|eot_id|>"""

NOTE: This model was trained 2x faster using Unsloth and Huggingface's TRL library.