epfml/landmark-attention-llama7b-wdiff

LLaMA-7B + Landmark Attention

This repo hosts the weight diff between LLaMA 7B trained with landmark attention for 15000 steps on RedPajama and the original model. Please visit the Github repository for further instructions on how to recover the full weights and how to use them.

Github repository: https://github.com/epfml/landmark-attention