diff --git a/README.md b/README.md index 2268af7..7732f7e 100644 --- a/README.md +++ b/README.md @@ -2,14 +2,14 @@ # LTX-Video -This is the official repository for LTX-Video. +[![Website](https://img.shields.io/badge/Website-LTXV-181717?logo=google-chrome)](https://www.lightricks.com/ltxv) +[![Model](https://img.shields.io/badge/HuggingFace-Model-orange?logo=huggingface)](https://huggingface.co/Lightricks/LTX-Video) +[![Demo](https://img.shields.io/badge/Demo-Try%20Now-brightgreen?logo=vercel)](https://app.ltx.studio/motion-workspace?videoModel=ltxv-13b) +[![Paper](https://img.shields.io/badge/Paper-arXiv-B31B1B?logo=arxiv)](https://arxiv.org/abs/2501.00103) +[![Trainer](https://img.shields.io/badge/LTXV-Trainer-9146FF?logo=github)](https://github.com/Lightricks/LTX-Video-Trainer) +[![Discord](https://img.shields.io/badge/Join-Discord-5865F2?logo=discord)](https://discord.gg/Mn8BRgUKKy) -[Website](https://www.lightricks.com/ltxv) | -[Model](https://huggingface.co/Lightricks/LTX-Video) | -[Demo](https://app.ltx.studio/motion-workspace?videoModel=ltxv-13b) | -[Paper](https://arxiv.org/abs/2501.00103) | -[Trainer](https://github.com/Lightricks/LTX-Video-Trainer) | -[Discord](https://discord.gg/Mn8BRgUKKy) +This is the official repository for LTX-Video. @@ -28,6 +28,7 @@ This is the official repository for LTX-Video. - [Model User Guide](#model-user-guide) - [Community Contribution](#community-contribution) - [Training](#training) +- [Control Models](#control-models) - [Join Us!](#join-us-) - [Acknowledgement](#acknowledgement) @@ -47,8 +48,24 @@ The model supports image-to-video, keyframe-based animation, video extension (bo | ![example4](./docs/_static/ltx-video_i2v_example_00004.gif) | ![example5](./docs/_static/ltx-video_i2v_example_00005.gif) | ![example6](./docs/_static/ltx-video_i2v_example_00006.gif) | | ![example7](./docs/_static/ltx-video_i2v_example_00007.gif) | ![example8](./docs/_static/ltx-video_i2v_example_00008.gif) | ![example9](./docs/_static/ltx-video_i2v_example_00009.gif) | +### Controlled video examples +| | | | +|:---:|:---:|:---:| +| ![control0](./docs/_static/ltx-video_ic_2v_example_00000.gif) | ![control1](./docs/_static/ltx-video_ic_2v_example_00001.gif) | ![control2](./docs/_static/ltx-video_ic_2v_example_00002.gif) | + +| | | +|:---:|:---:| +| ![control3](./docs/_static/ltx-video_ic_2v_example_00003.gif) | ![control4](./docs/_static/ltx-video_ic_2v_example_00004.gif) | + # News +## July, 8th, 2025: New Control Models Released! +- Released three new control models for LTX-Video on HuggingFace: + * **Depth Control**: [LTX-Video-ICLoRA-depth-13b-0.9.7](https://huggingface.co/Lightricks/LTX-Video-ICLoRA-depth-13b-0.9.7) + * **Pose Control**: [LTX-Video-ICLoRA-pose-13b-0.9.7](https://huggingface.co/Lightricks/LTX-Video-ICLoRA-pose-13b-0.9.7) + * **Canny Control**: [LTX-Video-ICLoRA-canny-13b-0.9.7](https://huggingface.co/Lightricks/LTX-Video-ICLoRA-canny-13b-0.9.7) + + ## May, 14th, 2025: New distilled model 13B v0.9.7: - Release a new 13B distilled model [ltxv-13b-0.9.7-distilled](https://huggingface.co/Lightricks/LTX-Video/blob/main/ltxv-13b-0.9.7-distilled.safetensors) * Amazing for iterative work - generates HD videos in 10 seconds, with low-res preview after just 3 seconds (on H100)! @@ -162,7 +179,7 @@ python -m pip install -e \[inference\] ### Inference -πŸ“ **Note:** For best results, we recommend using our [ComfyUI](#comfyui-integration) workflow. We’re working on updating the inference.py script to match the high quality and output fidelity of ComfyUI. +πŸ“ **Note:** For best results, we recommend using our [ComfyUI](#comfyui-integration) workflow. We're working on updating the inference.py script to match the high quality and output fidelity of ComfyUI. To use our model, please follow the inference code in [inference.py](./inference.py): @@ -290,11 +307,21 @@ please let us know by opening an issue or pull request. # ⚑️ Training We provide an open-source repository for fine-tuning the LTX-Video model: [LTX-Video-Trainer](https://github.com/Lightricks/LTX-Video-Trainer). -This repository supports both the 2B and 13B model variants, enabling full fine-tuning as well as LoRA (Low-Rank Adaptation) fine-tuning for more efficient training. +This repository supports both the 2B and 13B model variants, enabling full fine-tuning as well as LoRA (Low-Rank Adaptation) fine-tuning for more efficient training. This includes: + +- **Control LoRAs**: Train custom control models like depth, pose, and canny control +- **Effect LoRAs**: Create specialized effects and transformations for video generation Explore the repository to customize the model for your specific use cases! More information and training instructions can be found in the [README](https://github.com/Lightricks/LTX-Video-Trainer/blob/main/README.md). +# 🎬 Control Models + +[ComfyUI-LTXVideo](https://github.com/Lightricks/ComfyUI-LTXVideo) repository now contains workflows and models for 3 specialized models that enable precise control over LTX-Video generation: + +Pose Control, Depth Control and Canny Control + +**Example ComfyUI Workflow (for all control types):** [ic-lora.json](https://github.com/Lightricks/ComfyUI-LTXVideo/blob/main/example_workflows/ic_lora/ic-lora.json) # πŸš€ Join Us diff --git a/docs/_static/ltx-video_ic_2v_example_00000.gif b/docs/_static/ltx-video_ic_2v_example_00000.gif new file mode 100644 index 0000000..8cb40ef --- /dev/null +++ b/docs/_static/ltx-video_ic_2v_example_00000.gif @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:a70c182ecf7477a8e291f89559b9ddd6bd1cb87aa41fc286c0d5fa853fbf64d7 +size 37147005 diff --git a/docs/_static/ltx-video_ic_2v_example_00001.gif b/docs/_static/ltx-video_ic_2v_example_00001.gif new file mode 100644 index 0000000..427e5a3 --- /dev/null +++ b/docs/_static/ltx-video_ic_2v_example_00001.gif @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:cff7d1c49f4e51ff29e670eda111d1947702b5ffd51a9ac89fd2b539a8a2ebcf +size 77564878 diff --git a/docs/_static/ltx-video_ic_2v_example_00002.gif b/docs/_static/ltx-video_ic_2v_example_00002.gif new file mode 100644 index 0000000..e8f7632 --- /dev/null +++ b/docs/_static/ltx-video_ic_2v_example_00002.gif @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:220ecd1cffe3badecd573e7ad3c954b872bea7ecbafecedaafb11b40f791bde1 +size 32596519 diff --git a/docs/_static/ltx-video_ic_2v_example_00003.gif b/docs/_static/ltx-video_ic_2v_example_00003.gif new file mode 100644 index 0000000..e823634 --- /dev/null +++ b/docs/_static/ltx-video_ic_2v_example_00003.gif @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:57d4e848a3dc76e2b1a8b2562ae5c213399d5aeb98587cc8d9a21faefc470ead +size 32318190 diff --git a/docs/_static/ltx-video_ic_2v_example_00004.gif b/docs/_static/ltx-video_ic_2v_example_00004.gif new file mode 100644 index 0000000..ff08aed --- /dev/null +++ b/docs/_static/ltx-video_ic_2v_example_00004.gif @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:ac7a88e756ad15e04f645f69b3d8b4b0f2dd61a7ee39924f18f63ddfc8a926bf +size 10254972