File size: 1,958 Bytes
cfbfaf1
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
## What Works

| Loader         | Loading 1 LoRA | Loading 2 or more LoRAs | Training LoRAs | Multimodal extension | Perplexity evaluation |
|----------------|----------------|-------------------------|----------------|----------------------|-----------------------|
| Transformers   |       βœ…       |           βœ…***            |       βœ…*       |          βœ…          |           βœ…          |
| ExLlamav2_HF   |       βœ…       |           βœ…            |       ❌       |          ❌          |           βœ…          |
| ExLlamav2      |       βœ…       |           βœ…            |       ❌       |          ❌          |           use ExLlamav2_HF    |
| AutoGPTQ       |       βœ…       |           ❌            |       ❌       |          βœ…          |           βœ…          |
| GPTQ-for-LLaMa |       βœ…**       |           βœ…***            |       βœ…       |          βœ…          |           βœ…          |
| llama.cpp      |       ❌       |           ❌            |       ❌       |          ❌          |           use llamacpp_HF    |
| llamacpp_HF    |       ❌       |           ❌            |       ❌       |          ❌          |           βœ…          |
| ctransformers  |       ❌       |           ❌            |       ❌       |          ❌          |           ❌          |
| AutoAWQ        |       ?        |           ❌            |       ?       |          ?          |           βœ…          |

❌ = not implemented

βœ… = implemented

\* Training LoRAs with GPTQ models also works with the Transformers loader. Make sure to check "auto-devices" and "disable_exllama" before loading the model.

\*\* Requires the monkey-patch. The instructions can be found [here](https://github.com/oobabooga/text-generation-webui/wiki/08-%E2%80%90-Additional-Tips#using-loras-with-gptq-for-llama).

\*\*\* Multi-LoRA in PEFT is tricky and the current implementation does not work reliably in all cases.