Skip to content

Comments

[GenAI] Fix CI and stability for LTX-Video LoRA support (Supersedes #3309)#3362

Open
goyaladitya05 wants to merge 4 commits intoopenvinotoolkit:masterfrom
goyaladitya05:fix-lora-video-support
Open

[GenAI] Fix CI and stability for LTX-Video LoRA support (Supersedes #3309)#3362
goyaladitya05 wants to merge 4 commits intoopenvinotoolkit:masterfrom
goyaladitya05:fix-lora-video-support

Conversation

@goyaladitya05
Copy link
Contributor

@goyaladitya05 goyaladitya05 commented Feb 20, 2026

This PR builds upon the work started by @Eshaan-byte in #3309. I've integrated his logic and added the fixes for the set_adapters bindings and AdapterController initialization to resolve the CI issues.

Changes:

  • Added binding for LTXVideoTransformer3DModel.set_adapters() to resolve AttributeError in the TestLoRAVideoGeneration test suite.
  • Added a safety check in set_adapters() to prevent segmentation faults when the AdapterController is not initialized (e.g., when no adapters are provided at construction).

Closes #3306
Supersedes #3309

Verification Results:

without LoRA
without_lora

with LoRA (Adapter: ltxv_095_cakeify_lora.safetensors)
with_lora

These samples demonstrate that the LoRA adapter is successfully loaded and applied during the denoising process in the Text2VideoPipeline.

Environment:
Model: LTX-Video
Alpha: 1.5
Prompt:"A cute golden retriever puppy running in a green grassy field on a sunny day, high quality, photorealistic"

Checklist:

  • This PR follows GenAI Contributing guidelines.
  • Tests have been updated or added to cover the new code.
    • Verified tests/python_tests/test_video_generation.py passes locally.
  • This PR fully addresses the ticket.
  • I have made corresponding changes to the documentation.

Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR fixes CI failures from #3309 by adding the missing Python binding for LTXVideoTransformer3DModel.set_adapters() and implementing a safety check to prevent segmentation faults when the AdapterController is not initialized. The changes enable LoRA adapter support for LTX-Video generation, following the same architectural pattern used in image generation models.

Changes:

  • Added Python binding for set_adapters() method on the LTXVideoTransformer3DModel
  • Implemented safety check in C++ to prevent crashes when calling set_adapters() on models compiled without adapter support
  • Added comprehensive tests for LoRA functionality in the video generation pipeline

Reviewed changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
src/cpp/src/video_generation/models/ltx_video_transformer_3d_model.cpp Removed blocking assertion for adapters, added adapter initialization in compile(), implemented set_adapters() with safety check
src/cpp/include/openvino/genai/video_generation/ltx_video_transformer_3d_model.hpp Added lora_adapter.hpp include, declared set_adapters() method, added AdapterController member
src/python/py_video_generation_models.cpp Added Python binding for set_adapters() method with proper documentation
tests/python_tests/test_video_generation.py Added TestLoRAVideoGeneration class with tests for adapter functionality in constructor, generate(), and set_adapters()
samples/cpp/video_generation/lora_text2video.cpp New sample demonstrating LoRA usage with video generation, following the pattern from lora_text2image.cpp
samples/cpp/video_generation/CMakeLists.txt Added build configuration for lora_text2video sample, consistent with image generation samples
Comments suppressed due to low confidence (1)

tests/python_tests/test_video_generation.py:270

  • The test test_generate_without_cfg_after_reshape_with_cfg (lines 256-270) doesn't use any LoRA adapters functionality, so it seems misplaced in the TestLoRAVideoGeneration class. This test appears to be testing CFG (Classifier-Free Guidance) behavior, not LoRA functionality. Consider moving it to a more appropriate test class (e.g., TestText2VideoPipelineAdvanced) or creating a separate TestCFGBehavior class for CFG-related tests.
    def test_generate_without_cfg_after_reshape_with_cfg(self, video_generation_model):
        """Test: reshape with CFG then generate without CFG should raise error."""
        pipe = ov_genai.Text2VideoPipeline(video_generation_model)
        pipe.reshape(1, 9, 32, 32, 3.0)
        pipe.compile("CPU")

        with pytest.raises(RuntimeError, match="guidance_scale <= 1 requested, but the compiled model expects CFG"):
            pipe.generate(
                "test prompt",
                guidance_scale=1.0,
                height=32,
                width=32,
                num_frames=9,
                num_inference_steps=2,
            )

Eshaan-byte and others added 3 commits February 20, 2026 23:48
Implements LoRA (Low-Rank Adaptation) adapter support for LTX-Video
transformer model, following the same pattern as image generation.

Changes:
- Remove blocking assertion in ltx_video_transformer_3d_model.cpp
- Add adapter initialization in compile() method
- Implement set_adapters() method for runtime adapter control
- Add AdapterController member to manage adapter lifecycle
- Create lora_text2video.cpp sample demonstrating usage
- Add comprehensive tests in test_video_generation.py

The implementation allows users to:
- Load and apply one or more LoRA adapters
- Mix multiple adapters with different alpha values
- Enable/disable adapters per generation call
- Use community LoRA adapters from HuggingFace

Fixes openvinotoolkit#3306
- Added Python binding for LTXVideoTransformer3DModel.set_adapters()
  to expose the method to Python, fixing test_transformer_has_set_adapters_method
- Added safety check in set_adapters() to prevent crashes when
  AdapterController is not initialized
- Both fixes follow the same patterns used in image generation models

Fixes the CI failures in TestLoRAVideoGeneration test suite.
@goyaladitya05
Copy link
Contributor Author

Thanks for the feedback.
I’ve addressed the review comments (moved the CFG test, refined the ASSERT message) and rebased on master.
Could someone please run the CI once?

Copy link
Contributor

@likholat likholat left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you please also attach two short videos (converted to GIF) showing samples generated with and without LoRA adapter? Please add them to the PR description

Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 6 out of 6 changed files in this pull request and generated 1 comment.

@github-actions github-actions bot added the category: tokenizers Tokenizer class or submodule update label Feb 21, 2026
@goyaladitya05 goyaladitya05 force-pushed the fix-lora-video-support branch 2 times, most recently from 4270b3e to ed27566 Compare February 21, 2026 05:01
@github-actions github-actions bot removed the category: tokenizers Tokenizer class or submodule update label Feb 21, 2026
@goyaladitya05
Copy link
Contributor Author

Thanks for the feedback. I've updated the PR to address the review comments.

  • src/cpp/src/video_generation/ltx_pipeline.hpp:
    Added the missing m_transformer->set_adapters() invocation.

  • src/cpp/include/openvino/genai/video_generation/generation_config.hpp & .../generation_config_utils.cpp:
    Added the adapters field to VideoGenerationConfig to mirror the image generation pipelines.

  • src/python/py_video_generation_models.cpp: Exposed set_adapters to the Python bindings.

Samples & Documentation

  • samples/python/video_generation/lora_text2video.py: Created the missing Python sample.
  • samples/cpp/video_generation/lora_text2video.cpp: Cleaned the C++ logic.
  • samples/cpp/video_generation/CMakeLists.txt: Connected the new C++ sample.
  • samples/.../video_generation/README.md (both C++ & Python): Added run commands and documentation for the new LoRA samples, keeping the original example prompts used earlier.

Unit Tests

  • tests/python_tests/test_video_generation.py:
    Restored test_generate_without_cfg_default_compile which was accidentally lost in a rebase, and moved the two CFG to the correct location.

@goyaladitya05
Copy link
Contributor Author

goyaladitya05 commented Feb 22, 2026

Could you please also attach two short videos (converted to GIF) showing samples generated with and without LoRA adapter? Please add them to the PR description

@likholat , I've added the samples to the PR description. Please review whenever you have the time.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

LoRA for Video Generation

3 participants