Add library name, pipeline tag

README.md

@@ -7,6 +7,8 @@ pinned: true
 language:
 - en
 license: other
+library_name: diffusers
+pipeline_tag: any-to-any
 ---
 
 # LTX-Video Model Card
@@ -67,13 +69,13 @@ source env/bin/activate
 python -m pip install -e .\[inference-script\]
 ```
 
-Then, download the model from [Hugging Face](https://huggingface.co/Lightricks/LTX-Video) 
+Then, download the model from [Hugging Face](https://huggingface.co/Lightricks/LTX-Video)
 
 ```python
-from huggingface_hub import snapshot_download
+from huggingface_hub import hf_hub_download
 
 model_path = 'PATH'   # The local directory to save downloaded checkpoint
-snapshot_download("Lightricks/LTX-Video", local_dir=model_path, local_dir_use_symlinks=False, repo_type='model')
+hf_hub_download(repo_id="Lightricks/LTX-Video", filename="ltx-video-2b-v0.9.safetensors", local_dir=model_path, local_dir_use_symlinks=False, repo_type='model')
 ```
 
 #### Inference
@@ -83,13 +85,13 @@ To use our model, please follow the inference code in [inference.py](https://git
 ##### For text-to-video generation:
 
 ```bash
-python inference.py --ckpt_dir 'PATH' --prompt "PROMPT" --height HEIGHT --width WIDTH --num_frames NUM_FRAMES --seed SEED
+python inference.py --ckpt_path 'PATH' --prompt "PROMPT" --height HEIGHT --width WIDTH --num_frames NUM_FRAMES --seed SEED
 ```
 
 ##### For image-to-video generation:
 
 ```bash
-python inference.py --ckpt_dir 'PATH' --prompt "PROMPT" --input_image_path IMAGE_PATH --height HEIGHT --width WIDTH --num_frames NUM_FRAMES --seed SEED
+python inference.py --ckpt_path 'PATH' --prompt "PROMPT" --input_image_path IMAGE_PATH --height HEIGHT --width WIDTH --num_frames NUM_FRAMES --seed SEED
 ```
 
 ### Diffusers 🧨
@@ -162,4 +164,124 @@ Diffusers also supports directly loading from the original LTX checkpoints using
 - This model is not intended or able to provide factual information.
 - As a statistical model this checkpoint might amplify existing societal biases.
 - The model may fail to generate videos that matches the prompts perfectly.
-- Prompt following is heavily influenced by the prompting-style.
+- Prompt following is heavily influenced by the prompting-style.
+
+## Community Contribution
+
+### ComfyUI-LTXTricks 🛠️
+
+A community project providing additional nodes for enhanced control over the LTX Video model. It includes implementations of advanced techniques like RF-Inversion, RF-Edit, FlowEdit, and more. These nodes enable workflows such as Image and Video to Video (I+V2V), enhanced sampling via Spatiotemporal Skip Guidance (STG), and interpolation with precise frame settings.
+
+- **Repository:** [ComfyUI-LTXTricks](https://github.com/logtd/ComfyUI-LTXTricks)
+- **Features:**
+  - 🔄 **RF-Inversion:** Implements [RF-Inversion](https://rf-inversion.github.io/) with an [example workflow here](https://github.com/logtd/ComfyUI-LTXTricks/blob/main/example_workflows/example_ltx_inversion.json).
+  - ✂️ **RF-Edit:** Implements [RF-Solver-Edit](https://github.com/wangjiangshan0725/RF-Solver-Edit) with an [example workflow here](https://github.com/logtd/ComfyUI-LTXTricks/blob/main/example_workflows/example_ltx_rf_edit.json).
+  - 🌊 **FlowEdit:** Implements [FlowEdit](https://github.com/fallenshock/FlowEdit) with an [example workflow here](https://github.com/logtd/ComfyUI-LTXTricks/blob/main/example_workflows/example_ltx_flow_edit.json).
+  - 🎥 **I+V2V:** Enables Video to Video with a reference image. [Example workflow](https://github.com/logtd/ComfyUI-LTXTricks/blob/main/example_workflows/example_ltx_iv2v.json).
+  - ✨ **Enhance:** Partial implementation of [STGuidance](https://junhahyung.github.io/STGuidance/). [Example workflow](https://github.com/logtd/ComfyUI-LTXTricks/blob/main/example_workflows/example_ltxv_stg.json).
+  - 🖼️ **Interpolation and Frame Setting:** Nodes for precise control of latents per frame. [Example workflow](https://github.com/logtd/ComfyUI-LTXTricks/blob/main/example_workflows/example_ltx_interpolation.json).
+
+
+### LTX-VideoQ8 🎱
+
+**LTX-VideoQ8** is an 8-bit optimized version of [LTX-Video](https://github.com/Lightricks/LTX-Video), designed for faster performance on NVIDIA ADA GPUs.
+
+- **Repository:** [LTX-VideoQ8](https://github.com/KONAKONA666/LTX-Video)
+- **Features:**
+  - 🚀 Up to 3X speed-up with no accuracy loss
+  - 🎥 Generate 720x480x121 videos in under a minute on RTX 4060 (8GB VRAM)
+  - 🛠️ Fine-tune 2B transformer models with precalculated latents
+- **Community Discussion:** [Reddit Thread](https://www.reddit.com/r/StableDiffusion/comments/1h79ks2/fast_ltx_video_on_rtx_4060_and_other_ada_gpus/)
+
+### Your Contribution
+
+...is welcome! If you have a project or tool that integrates with LTX-Video,
+please let us know by opening an issue or pull request.
+
+# Training
+
+## Diffusers
+
+Diffusers implemented [LoRA support](https://github.com/huggingface/diffusers/pull/10228),
+with a training script for fine-tuning.
+More information and training script in
+[finetrainers](https://github.com/a-r-r-o-w/finetrainers?tab=readme-ov-file#training).
+
+## Diffusion-Pipe
+
+An experimental training framework with pipeline parallelism, enabling fine-tuning of large models like **LTX-Video** across multiple GPUs.
+
+- **Repository:** [Diffusion-Pipe](https://github.com/tdrussell/diffusion-pipe)
+- **Features:**
+  - 🛠️ Full fine-tune support for LTX-Video using LoRA
+  - 📊 Useful metrics logged to Tensorboard
+  - 🔄 Training state checkpointing and resumption
+  - ⚡ Efficient pre-caching of latents and text embeddings for multi-GPU setups
+
+
+# Join Us 🚀
+
+Want to work on cutting-edge AI research and make a real impact on millions of users worldwide?
+
+At **Lightricks**, an AI-first company, we’re revolutionizing how visual content is created.
+
+If you are passionate about AI, computer vision, and video generation, we would love to hear from you!
+
+Please visit our [careers page](https://careers.lightricks.com/careers?query=&office=all&department=R%26D) for more information.
+
+# Acknowledgement
+
+We are grateful for the following awesome projects when implementing LTX-Video:
+* [DiT](https://github.com/facebookresearch/DiT) and [PixArt-alpha](https://github.com/PixArt-alpha/PixArt-alpha): vision transformers for image generation.
+
+## Citation
+
+📄 Our tech report is out! If you find our work helpful, please ⭐️ star the repository and cite our paper.
+
+```
+@article{HaCohen2024LTXVideo,
+  title={LTX-Video: Realtime Video Latent Diffusion},
+  author={HaCohen, Yoav and Chiprut, Nisan and Brazowski, Benny and Shalem, Daniel and Moshe, Dudu and Richardson, Eitan and Levin, Eran and Shiran, Guy and Zabari, Nir and Gordon, Ori and Panet, Poriya and Weissbuch, Sapir and Kulikov, Victor and Bitterman, Yaki and Melumian, Zeev and Bibi, Ofir},
+  journal={arXiv preprint arXiv:2501.00103},
+  year={2024}
+}
+```
+## reasoning
+The model card already has the license, but is missing the `pipeline_tag` and `library_name`. Given the description in the README and the Github README,
+it takes text as input and generates videos as output, hence we can add `pipeline_tag: text-to-video`. From the code snippets, it's also clear that
+the model is compatible with the `diffusers` library, hence we can add that as `library_name`. We also add the link to the paper.
+
+## Title
+Add pipeline tag, library name
+
+## Comment
+This PR adds the `pipeline_tag` and `library_name` to the model card, such that it can be found with the tag at https://huggingface.co/models?pipeline_tag=text-to-video.
+It also adds a link to the paper at https://huggingface.co/papers/2501.00103.
+
+## Content
+```yaml
+---
+tags:
+- ltx-video
+- text-to-video
+- image-to-video
+pinned: true
+language:
+- en
+license: other
+library_name: diffusers
+pipeline_tag: text-to-video
+---
+# LTX-Video Model Card
+This model card focuses on the model associated with the LTX-Video model, codebase available [here](https://github.com/Lightricks/LTX-Video).
+
+LTX-Video is the first DiT-based video generation model capable of generating high-quality videos in real-time. It produces 24 FPS videos at a 768x512 resolution faster than they can be watched. Trained on a large-scale dataset of diverse videos, the model generates high-resolution videos with realistic and varied content.
+We provide a model for both text-to-video as well as image+text-to-video usecases
+
+<img src="./media/trailer.gif" alt="trailer" width="512">
+
+
+| | | | |
+|:---:|:---:|:---:|:---:|
+| ![example1](./media/ltx-video_example_00001.gif)<br><details style="max-width: 300px; margin: auto;"><summary>A woman with long brown hair and light skin smiles at another woman...</summary>A woman with long brown hair and light skin smiles at another woman with long blonde hair. The woman with brown hair wears a black jacket and has a small, barely noticeable mole on her right cheek. The camera angle is a close-up, focused on the woman with brown hair's face. The lighting is warm and natural, likely from the setting sun, casting a soft glow on the scene. The scene appears to be real-life footage.</details> | ![example2](./media/ltx-video_example_00002.gif)<br><details style="max-width: 300px; margin: auto;"><summary>A woman walks away from a white Jeep parked on a city street at night...</summary>A woman walks away from a white Jeep parked on a city street at night, then ascends a staircase and knocks on a door. The woman, wearing a dark jacket and jeans, walks away from the Jeep parked on the left side of the street, her back to the camera; she walks at a steady pace, her arms swinging slightly by her sides; the street is dimly lit, with streetlights casting pools of light on the wet pavement; a man in a dark jacket and jeans walks past the Jeep in the opposite direction; the camera follows the woman from behind as she walks up a set of stairs towards a building with a green door; she reaches the top of the stairs and turns left, continuing to walk towards the building; she reaches the door and knocks on it with her right hand; the camera remains stationary, focused on the doorway; the scene is captured in real-life footage.</details> | ![example3](./media/ltx-video_example_00003.gif)<br><details style="max-width: 300px; margin: auto;"><summary>A woman with blonde hair styled up, wearing a black dress...</summary>A woman with blonde hair styled up, wearing a black dress with sequins and pearl earrings, looks down with a sad expression on her face. The camera remains stationary, focused on the woman's face. The lighting is dim, casting soft shadows on her face. The scene appears to be from a movie or TV show.</details> | ![example4](./media/ltx-video_example_00004.gif)<br><details style="max-width: 300px; margin: auto;"><summary>The camera pans over a snow-covered mountain range...</summary>The camera pans over a snow-covered mountain range, revealing a vast expanse of snow-capped peaks and valleys.The mountains are covered in a thick layer of snow, with some areas appearing almost white while others have a slightly darker, almost grayish hue. The peaks are jagged and irregular, with some rising sharply into the sky while others are more rounded. The valleys are deep and narrow, with steep slopes that are also covered in snow. The trees in the foreground are mostly bare, with only a few leaves remaining on their branches. The sky is overcast, with thick clouds obscuring the sun. The overall impression is one of peace and tranquility, with the snow-covered mountains standing as a testament to the power and beauty of nature.</details> |
+| ![example5](./media/ltx-video_example_00005.gif)<br><details style="max-width: 300px; margin: auto;"><summary>A woman with light skin, wearing a blue jacket and a black hat...</summary>A woman with light skin, wearing a blue jacket and a black hat with a veil, looks down and to her right, then back up as she speaks; she has brown hair styled in an updo, light brown eyebrows, and is wearing a white collared shirt under her jacket; the camera remains stationary on her face as she speaks; the background is out of focus, but shows trees and people in period clothing; the scene is captured in real-life footage.</details> | ![example6](./media/ltx-video_example_00006.gif)<br><details style="max-width: 300px; margin: auto;"><summary>A man in a dimly lit room talks on a vintage telephone...</summary>A man in a dimly lit room talks on a vintage telephone, hangs up, and looks down with a sad expression. He holds the black rotary phone to his right ear with his right hand, his left hand holding a rocks glass with amber liquid. He wears a brown suit jacket over a white shirt, and a gold ring on his left ring finger. His short hair is neatly combed, and he has light skin with visible wrinkles around his eyes. The camera remains stationary, focused on his face and upper body. The room is dark, lit only by a warm light source off-screen to the left, casting shadows on the wall behind him. The scene appears to be from a movie.</details> | ![example7](./media/ltx-video_example_00007.gif)<br><details style="max-width: 300px; margin: auto;"><summary>A prison guard unlocks and opens a cell door...</summary>A prison guard unlocks and opens a cell door to reveal a young man sitting at a table with a woman. The guard, wearing a dark blue uniform with a badge on his left chest, unlocks the cell door with a key held in his right hand and pulls it open; he has short brown hair, light skin, and a neutral expression. The young man, wearing a black and white striped shirt, sits at a table covered with a white tablecloth, facing the woman; he has short brown hair, light skin, and a neutral expression. The woman, wearing a dark blue