LouisML's picture
Update README.md
f48aa8e
metadata
license: mit

This repo contains a low-rank adapter for LLaMA-7b finetuned on Ntropy proprietary dataset (consumer financial transactions).

This version of the weights was trained with the following hyperparameters:

  • Base Model: decapoda-research/llama-7b-hf
  • Epochs: 10 (load from best epoch)
  • Batch size: 32
  • Cutoff length: 1024
  • Learning rate: 3e-4
  • Lora r: 16
  • Lora target modules: q_proj, k_proj, v_proj, o_proj

Instructions for running it can be found at https://github.com/ntropy-network/enrichment_models/blob/main/notebooks/llama.ipynb