metadata
library_name: peft
pipeline_tag: text-generation
tags:
- pytorch
- llama-2
datasets:
- timdettmers/openassistant-guanaco
This model was fine-tuned following the instructions in https://huggingface.co/blog/llama2#fine-tuning-with-peft. I used a g5.xlarge instance on AWS (1xA10G GPU), with the Deep Learning AMI for PyTorch. Training time was about 10 hours. The full log is included.
Training procedure
The following bitsandbytes quantization config was used during training:
- quant_method: bitsandbytes
- load_in_8bit: False
- load_in_4bit: True
- llm_int8_threshold: 6.0
- llm_int8_skip_modules: None
- llm_int8_enable_fp32_cpu_offload: False
- llm_int8_has_fp16_weight: False
- bnb_4bit_quant_type: fp4
- bnb_4bit_use_double_quant: False
- bnb_4bit_compute_dtype: float32
Framework versions
- PEFT 0.5.0