Update README.md
Browse files
README.md
CHANGED
|
@@ -91,7 +91,7 @@ For example, these can't perform the orthogonal projection needed for ["ablitera
|
|
| 91 |
|
| 92 |
`h' = h - v @ v^T @ h`
|
| 93 |
|
| 94 |
-
whereas the general (non-orthogonal) "multiplicative-LoRA" method can do this by choosing to set `u = -v` like so:
|
| 95 |
|
| 96 |
`h' = h + u @ v^T @ h`
|
| 97 |
|
|
|
|
| 91 |
|
| 92 |
`h' = h - v @ v^T @ h`
|
| 93 |
|
| 94 |
+
whereas the general (non-orthogonal) "multiplicative-LoRA" method can (in theory) do this by choosing to set `u = -v` like so:
|
| 95 |
|
| 96 |
`h' = h + u @ v^T @ h`
|
| 97 |
|