Update README.md
Browse files
README.md
CHANGED
@@ -106,7 +106,7 @@ So instead of having just a single vector that we add (and in essence adding a `
|
|
106 |
|
107 |
![image/png](https://cdn-uploads.huggingface.co/production/uploads/65995c45539c808e84c38bf1/vZ2Gys3huKAWIVe0wz2-q.png)
|
108 |
|
109 |
-
but
|
110 |
|
111 |
---
|
112 |
|
|
|
106 |
|
107 |
![image/png](https://cdn-uploads.huggingface.co/production/uploads/65995c45539c808e84c38bf1/vZ2Gys3huKAWIVe0wz2-q.png)
|
108 |
|
109 |
+
but whereas `lora_A` looks at the ***input*** to the transformation for "additive-LoRAs"; these new (non-orthogonal) "multiplicative-LoRAs" instead use `lora_A` to look at the ***output*** of the (`down_proj`) transformation...
|
110 |
|
111 |
---
|
112 |
|