Uploaded model

  • Developed by: zmilczarek
  • License: apache-2.0
  • Finetuned from model : unsloth/llama-3-8b-bnb-4bit

This llama model was trained using Unsloth and Huggingface's TRL library.

Training details

Dataset

The model was trained on the following dataset: link, for the task of extracting the Publication Date of the document.

The model is fine-tuned on prompts with the following structure:

{'role': 'user', 'content': f'Beggining and end of the document :\n{context}\nWhat is the publication date of the document? Output as a structured JSON object with a format DD/MM/YYYY.'}, {'role': 'assistant', 'content':f'{gold_date}'}

Where the context is the first 3000 and last 3000 characters of the document, and the gold_date is a date from the original dataset.

Training

Trained using QLoRa adapters with 3 epochs.

Evaluation

The model scores 70% on accuracy measured as exact date matches and 80% on accuracy measured as partial date matches (combined month and year matches).

Usage

To load the model for inference, use the following code snippet:

from unsloth import FastLanguageModel, get_chat_template
model, tokenizer = FastLanguageModel.from_pretrained(
        model_name = "zmilczarek/llama3_8b-finetuned-nlp_industry-adapters",
        max_seq_length = 2048,
        dtype = None,
        load_in_4bit = True,
    )
FastLanguageModel.for_inference(model)
tokenizer = get_chat_template(
        tokenizer,
        chat_template = "llama-3.1",
    )

Or if you don't want to use unsloth, it is also possible to use Huggingface PEFT and Transformers libraries:

from peft import AutoPeftModelForCausalLM
from transformers import AutoTokenizer
model = AutoPeftModelForCausalLM.from_pretrained(
    "zmilczarek/llama3_8b-finetuned-nlp_industry-adapters",
    load_in_4bit = True,
)
tokenizer = AutoTokenizer.from_pretrained("zmilczarek/llama3_8b-finetuned-nlp_industry-adapters")

To perform inference on a document, you can use the following prompt:

def predict_date(message:list[dict]):
    inputs = tokenizer.apply_chat_template(
        message,
        tokenize = True,
        add_generation_prompt = True,
        return_tensors = "pt",
    ).to("cuda")

    outputs = model.generate(input_ids = inputs, max_new_tokens = 13, use_cache = True,
                         temperature = 1.5, min_p = 0.1)
    outputs_ans_only = outputs[:,len(inputs[0]):]
    answer_only = tokenizer.batch_decode(outputs_ans_only, skip_special_tokens=True)
    return answer_only

context = "[YOUR DOCUMENT]"
prompt = "What is the publication date of the document? Output as a structured JSON object with a format DD/MM/YYYY."

msg = [{'role': 'user', 'content': f'Beggining and end of the document :\n{context}\n{prompt}'}]

predicted_date = predict_date(msg)
Downloads last month

-

Downloads are not tracked for this model. How to track
Inference Providers NEW
This model is not currently available via any of the supported Inference Providers.
The model cannot be deployed to the HF Inference API: The model has no pipeline_tag.

Model tree for zmilczarek/llama3_8b-finetuned-nlp_industry-adapters

Finetuned
(2636)
this model