LoRA-Learns-Less-and-Forgets-Less

community

https://arxiv.org/abs/2405.09673

AI & ML interests

None defined yet.

Organization Card

Community About org cards

These are the model weights associated with the TMLR 2024 publication LoRA Learns Less and Forgets Less (Biderman et al. 2024). This work was done in collaboration with Databricks Mosaic AI Research.

Collections 4

models 17

LoRA-TMLR-2024/magicoder-lora-rank-16-alpha-32

LoRA-TMLR-2024/starcoder-full-finetuning-lr-1e-05-20B-token

LoRA-TMLR-2024/starcoder-lora-rank-256-20B-tokens

Updated Sep 27 • 2

LoRA-TMLR-2024/openwebmath-full-finetuning-lr-1e-05-20B-tokens

LoRA-TMLR-2024/openwebmath-lora-rank-256-20B-tokens

Updated Sep 27 • 4

LoRA-TMLR-2024/openwebmath-lora-rank-16-20B-tokens

Updated Sep 27 • 4

LoRA-TMLR-2024/starcoder-lora-rank-16-20B-tokens

LoRA-TMLR-2024/magicoder-lora-rank-64-alpha-128

Updated Sep 27 • 5

LoRA-TMLR-2024/magicoder-lora-rank-256-alpha-512

LoRA-TMLR-2024/magicoder-full-finetuning-lr-5e-05

datasets

None public yet