Skip to content

Serialize mixer output frame ordering via AsyncStream.#1890

Merged
shogo4405 merged 1 commit intoHaishinKit:mainfrom
dawilster:fix/serialize-mixer-output-frame-ordering
Mar 22, 2026
Merged

Serialize mixer output frame ordering via AsyncStream.#1890
shogo4405 merged 1 commit intoHaishinKit:mainfrom
dawilster:fix/serialize-mixer-output-frame-ordering

Conversation

@dawilster
Copy link
Contributor

Fixes #1889

Replace unstructured Task {} in MediaMixerOutput and StreamOutput conformances with AsyncStream channels that preserve FIFO ordering.

Problem

The nonisolated mixer delegate methods in RTMPStream, SRTStream, and StreamRecorder each create a new Task {} per callback to hop into the actor. Swift provides no ordering guarantee for unstructured tasks entering an actor's serial executor, so adjacent frames can arrive out of order.

This causes:

  • RTMPTimestamp.invalidSequence errors and silent frame drops in RTMPStream / SRTStream
  • AVAssetWriter failures (StreamRecorder.Error.failedToAppend) when out-of-order timestamps reach the writer

Changes

  • RTMPStream: Mixer callbacks yield to AsyncStream continuations instead of creating Task {}. Consumer tasks started in publish() pull frames in FIFO order.
  • SRTStream: Same approach as RTMPStream.
  • StreamRecorder: Both StreamOutput and MediaMixerOutput callbacks yield to a single AsyncStream. Consumer task started in startRecording().

Continuations are declared nonisolated(unsafe) since AsyncStream.Continuation.yield is thread-safe and mutations only occur from within the actor.

Replace unstructured Task {} in MediaMixerOutput and StreamOutput
conformances with AsyncStream channels that preserve FIFO ordering.

The previous pattern created a new Task for each audio/video callback,
which provides no ordering guarantee when entering an actor's serial
executor. This caused adjacent frames to arrive out of order, resulting
in RTMPTimestamp.invalidSequence errors (silent frame drops) and
AVAssetWriter failures in StreamRecorder.
@dawilster dawilster force-pushed the fix/serialize-mixer-output-frame-ordering branch from ad4e94e to ea4d6b2 Compare March 10, 2026 05:28
CricHeroesiOS added a commit to CricHeroesiOS/HaishinKit.swift that referenced this pull request Mar 19, 2026
Replace unstructured Task {} in MediaMixerOutput and StreamOutput
conformances with AsyncStream channels that preserve FIFO ordering.

The previous pattern created a new Task for each audio/video callback,
which provides no ordering guarantee when entering an actor's serial
executor. This caused adjacent frames to arrive out of order, resulting
in RTMPTimestamp.invalidSequence errors (silent frame drops) and
AVAssetWriter failures in StreamRecorder.
@shogo4405
Copy link
Collaborator

Thank you for your patience.

I haven’t been able to come up with a solid design yet, so I’d like to merge this PR and proceed with testing.
Thank you for the excellent PR—I really appreciate it.

@shogo4405 shogo4405 merged commit d4ce2eb into HaishinKit:main Mar 22, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

Issue in RTMPStream and StreamRecorder causes silent frame drops and AVAssetWriter failures due to reordering

2 participants