llama-3.2-1B-it-Procurtech-Assistant

This model is a fine-tuned version of meta-llama/Llama-3.2-1B-Instruct on Procurtech Assistant dataset.

Model description

A customer support model to help customers with their orders, incase they encounter any difficulty.

Intended uses & limitations

The training dataset can be modified, see original at customer support dataset .. I edited the system message with a bit of prompt engineering, included additional details about the eCommerce company.
You can decide what you want and further fine tune the model...

Training and evaluation data

Training data.
Used the complete dataset for training, no evaluation data, I evaluated with random prompts...

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0002
• train_batch_size: 1
• eval_batch_size: 8
• seed: 42
• gradient_accumulation_steps: 2
• total_train_batch_size: 2
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: cosine
• lr_scheduler_warmup_steps: 682
• num_epochs: 1

Training results

Training Loss from wandb

Framework versions

• PEFT 0.13.2
• Transformers 4.45.2
• Pytorch 2.4.1+cu121
• Datasets 3.0.1
• Tokenizers 0.20.1