Update README.md

README.md

@@ -19,6 +19,9 @@ This project explores two options to reduce the original LoRA checkpoint into an
 * Random projections
 * SVD
 
+> [!TIP]
+> We have also explored the opposite direction of the above i.e., take a low-rank LoRA and increase its rank with orthoginal completion. Check out [this section](#lora-rank-upsampling) for more details (code, results, etc.).
+
 ## Random projections
 
 Basic idea:
@@ -140,4 +143,70 @@ Code: [`svd_low_rank_lora.py`](https://huggingface.co/sayakpaul/lower-rank-flux-
 
 * Randomized SVD: [How2Draw-V2_000002800_rand_svd.safetensors](./How2Draw-V2_000002800_rand_svd.safetensors)
 * Full SVD: [How2Draw-V2_000002800_svd.safetensors](./How2Draw-V2_000002800_svd.safetensors)
-* Random projections: [How2Draw-V2_000002800_reduced.safetensors](./How2Draw-V2_000002800_reduced.safetensors)
+* Random projections: [How2Draw-V2_000002800_reduced.safetensors](./How2Draw-V2_000002800_reduced.safetensors)
+
+## LoRA rank upsampling
+
+We also explored the opposite direction of what we presented above. We do this by using "orthogonal extension" across
+the rank dimensions. Since we are increasing the ranks, we thought "rank upsampling" was a cool name! Check out [upsample_lora_rank.py](./upsample_lora_rank.py) script for
+the implementation.
+
+We applied this technique to [`cocktailpeanut/optimus`](https://huggingface.co/cocktailpeanut/optimus) to increase the rank from 4 to 16. You can find the
+checkpoint [here](https://huggingface.co/sayakpaul/flux-lora-resizing/blob/main/optimus_16.safetensors. 
+
+### Results
+
+Right: original Left: upsampled
+
+<table style="border-collapse: collapse;">
+  <tbody>
+    <tr>
+      <td align="center"><img src="https://huggingface.co/sayakpaul/flux-lora-resizing/resolve/main/upsampled_lora/0_collage.png" alt="Image 1"></td>
+      <td align="center">optimus is cleaning the house with broomstick</td>
+    </tr>
+    <tr>
+      <td align="center"><img src="https://huggingface.co/sayakpaul/flux-lora-resizing/resolve/main/upsampled_lora/1_collage.png" alt="Image 2"></td>
+      <td align="center">optimus is a DJ performing at a hip nightclub</td>
+    </tr>
+    <tr>
+      <td align="center"><img src="https://huggingface.co/sayakpaul/flux-lora-resizing/resolve/main/upsampled_lora/2_collage.png" alt="Image 3"></td>
+      <td align="center">optimus is competing in a bboy break dancing competition</td>
+    </tr>
+    <tr>
+      <td align="center"><img src="https://huggingface.co/sayakpaul/flux-lora-resizing/resolve/main/upsampled_lora/3_collage.png" alt="Image 4"></td>
+      <td align="center">optimus is playing tennis in a tennis court</td>
+    </tr>
+  </tbody>
+</table> 
+
+<details>
+  <summary>Code</summary>
+
+```python
+from diffusers import FluxPipeline
+import torch 
+
+pipeline = FluxPipeline.from_pretrained(
+    "black-forest-labs/FLUX.1-dev", torch_dtype=torch.bfloat16
+).to("cuda")
+# Change this.
+pipeline.load_lora_weights("optimus_16.safetensors")
+
+prompts = [
+    "optimus is cleaning the house with broomstick",
+    "optimus is a DJ performing at a hip nightclub",
+    "optimus is competing in a bboy break dancing competition",
+    "optimus is playing tennis in a tennis court"
+]
+images = pipeline(
+    prompts, 
+    num_inference_steps=50,
+    guidance_scale=3.5,
+    max_sequence_length=512,
+    generator=torch.manual_seed(0)
+).images
+for i, image in enumerate(images):
+    image.save(f"{i}_{'upsampled' if upsample else 'non_upsampled'}.png")
+```
+
+</details>