[vk-video] Add color metadata to decoded frames#1831
Merged
Conversation
noituri
force-pushed
the
@noituri/vv-decoder-vui
branch
from
87b8e70 to
587be97
Compare
There was a problem hiding this comment.
Pull request overview
This PR extends vk-video’s decode output to carry H.264 color information (color space + range) alongside frame data, and refactors the frame types to distinguish encoding inputs from decoding outputs.
Changes:
- Split
Frame<T>intoInputFrame<T>(encoding) andOutputFrame<T>(decoding), addingFrameMetadatato decoded frames. - Derive decoded frame color metadata from the H.264 SPS (VUI) and propagate it through the decoder/sorter pipeline.
- Update examples and downstream usage (
smelter-core) to the new frame types; adjust crate feature defaults.
Reviewed changes
Copilot reviewed 17 out of 17 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| vk-video/src/wgpu_helpers.rs | Doc comment tweak for NV12/RGBA conversion parameters. |
| vk-video/src/vulkan_video/wgpu_api.rs | Update WGPU decode/encode API signatures to use InputFrame/OutputFrame. |
| vk-video/src/vulkan_video.rs | Introduce InputFrame, OutputFrame, and decoded FrameMetadata in the public API. |
| vk-video/src/vulkan_transcoder.rs | Update transcoder to accept OutputFrame where it previously used Frame. |
| vk-video/src/vulkan_encoder.rs | Update encoder APIs to accept InputFrame instead of the old Frame. |
| vk-video/src/vulkan_decoder/frame_sorter.rs | Emit OutputFrame and attach FrameMetadata when sorting/flushing frames. |
| vk-video/src/vulkan_decoder.rs | Add color metadata to decode results by reading it from the SPS. |
| vk-video/src/device.rs | Add From<&SeqParameterSet> conversions to map SPS VUI to ColorSpace/ColorRange. |
| vk-video/examples/player/player/decoder.rs | Update example player to use OutputFrame. |
| vk-video/examples/encode_wgpu.rs | Update WGPU encode example to use InputFrame. |
| vk-video/examples/encode.rs | Update bytes encode example to use InputFrame. |
| vk-video/examples/decode_wgpu.rs | Update WGPU decode example to use OutputFrame. |
| vk-video/examples/decode.rs | Update bytes decode example to use OutputFrame. |
| vk-video/Cargo.toml | Enable transcoder feature by default (in addition to wgpu). |
| vk-video/CHANGELOG.md | Document the Frame<T> split and decoded color metadata addition. |
| smelter-core/src/pipeline/encoder/vulkan_h264.rs | Update integration to construct vk_video::InputFrame. |
| smelter-core/src/pipeline/decoder/vulkan_h264.rs | Update integration to consume vk_video::OutputFrame. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
You can also share your feedback on Copilot code review. Take the survey.
noituri
force-pushed
the
@noituri/vv-decoder-vui
branch
from
587be97 to
3a20a51
Compare
noituri
commented
Mar 19, 2026
Comment on lines
+157
to
+168
| fn from(sps: &h264_reader::nal::sps::SeqParameterSet) -> Self { | ||
| sps.vui_parameters | ||
| .as_ref() | ||
| .and_then(|v| v.video_signal_type.as_ref()) | ||
| .map(|vst| { | ||
| if vst.video_full_range_flag { | ||
| ColorRange::Full | ||
| } else { | ||
| ColorRange::Limited | ||
| } | ||
| }) | ||
| .unwrap_or(ColorRange::Limited) |
Member
Author
There was a problem hiding this comment.
I'm defaulting here to the limited range because that's what's assumed when video_full_range_flag is not present but maybe returning None would be better here, or maybe adding ColorRange::Unspecified?
Collaborator
There was a problem hiding this comment.
no, this is what the spec says in h264. its correct
noituri
force-pushed
the
@noituri/vv-decoder-vui
branch
from
3a20a51 to
f4a74f9
Compare
noituri
force-pushed
the
@noituri/vv-decoder-vui
branch
from
f4a74f9 to
ef51690
Compare
jerzywilczek
approved these changes
Mar 19, 2026
Collaborator
jerzywilczek
left a comment
jerzywilczek
left a comment
There was a problem hiding this comment.
Looks good, but remember how I said we should warn if a user tries to use our converter on the wrong color space? this makes it trivial for NV12 -> RGBA
Comment on lines
+157
to
+168
| fn from(sps: &h264_reader::nal::sps::SeqParameterSet) -> Self { | ||
| sps.vui_parameters | ||
| .as_ref() | ||
| .and_then(|v| v.video_signal_type.as_ref()) | ||
| .map(|vst| { | ||
| if vst.video_full_range_flag { | ||
| ColorRange::Full | ||
| } else { | ||
| ColorRange::Limited | ||
| } | ||
| }) | ||
| .unwrap_or(ColorRange::Limited) |
Collaborator
There was a problem hiding this comment.
no, this is what the spec says in h264. its correct
jerzywilczek
approved these changes
Mar 20, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.