Live stream missing last segment (2–15 min)

[noho501]

Hi,

First, I’m sorry if this topic has been discussed before — I’ve read all related issues and discussions, but I still can’t solve my problem.

My issue is similar to #1297
During live streaming, the final part of the stream (from 2 up to 15 minutes) is often missing when the stream stops.

In that discussion, it was suggested to implement streamWillDropFrame to control latency and dropped frames manually, but it seems this method no longer exists in the latest version.

I researched and found that the new version has automatic dropped frame handling here:
https://github.com/HaishinKit/HaishinKit.swift/blob/main/HaishinKit/Sources/Stream/MediaLink.swift#L85

However, if it’s handled automatically, why does my stream still lose the last segment?
Do I need to enable or configure any parameters to activate this dropped frame logic?

Any help or clarification would be greatly appreciated.

Thank you very much 🙏

[shogo4405]

On the HaishinKit side, Basically don’t do anything since handling is intended to be done on the user side.
Therefore, if you use it as-is, and the cause is the same, you’ll get the same result.

Case #1297

Premise: While streaming with HaishinKit, the client side has insufficient bandwidth. As a result, data that couldn’t be fully sent remains on the mobile side.
If the stream ends in the middle of that state, the last part of the video will be cut off.

In the latest version, this kind of handling can be done using BitRateStrategy:

HaishinKit.swift/HaishinKit/Sources/Stream/StreamBitRateStrategy.swift

Lines 3 to 73 in 33818de

	/// A type with a network bitrate strategy representation.
	public protocol StreamBitRateStrategy: Sendable {
	/// The mamimum video bitRate.
	var mamimumVideoBitRate: Int { get }
	/// The mamimum audio bitRate.
	var mamimumAudioBitRate: Int { get }
	/// Adjust a bitRate.
	func adjustBitrate(_ event: NetworkMonitorEvent, stream: some StreamConvertible) async
	}
	/// An actor provides an algorithm that focuses on video bitrate control.
	public final actor StreamVideoAdaptiveBitRateStrategy: StreamBitRateStrategy {
	/// The status counts threshold for restoring the status
	public static let statusCountsThreshold: Int = 15
	public let mamimumVideoBitRate: Int
	public let mamimumAudioBitRate: Int = 0
	private var sufficientBWCounts: Int = 0
	private var zeroBytesOutPerSecondCounts: Int = 0
	/// Creates a new instance.
	public init(mamimumVideoBitrate: Int) {
	self.mamimumVideoBitRate = mamimumVideoBitrate
	}
	public func adjustBitrate(_ event: NetworkMonitorEvent, stream: some StreamConvertible) async {
	switch event {
	case .status:
	var videoSettings = await stream.videoSettings
	if videoSettings.bitRate == mamimumVideoBitRate {
	return
	}
	if Self.statusCountsThreshold <= sufficientBWCounts {
	let incremental = mamimumVideoBitRate / 10
	videoSettings.bitRate = min(videoSettings.bitRate + incremental, mamimumVideoBitRate)
	try? await stream.setVideoSettings(videoSettings)
	sufficientBWCounts = 0
	} else {
	sufficientBWCounts += 1
	}
	case .publishInsufficientBWOccured(let report):
	sufficientBWCounts = 0
	var videoSettings = await stream.videoSettings
	let audioSettings = await stream.audioSettings
	if 0 < report.currentBytesOutPerSecond {
	let bitRate = Int(report.currentBytesOutPerSecond * 8) / (zeroBytesOutPerSecondCounts + 1)
	videoSettings.bitRate = max(bitRate - audioSettings.bitRate, mamimumVideoBitRate / 10)
	videoSettings.frameInterval = 0.0
	sufficientBWCounts = 0
	zeroBytesOutPerSecondCounts = 0
	} else {
	switch zeroBytesOutPerSecondCounts {
	case 2:
	videoSettings.frameInterval = VideoCodecSettings.frameInterval10
	case 4:
	videoSettings.frameInterval = VideoCodecSettings.frameInterval05
	default:
	break
	}
	try? await stream.setVideoSettings(videoSettings)
	zeroBytesOutPerSecondCounts += 1
	}
	case .reset:
	var videoSettings = await stream.videoSettings
	zeroBytesOutPerSecondCounts = 0
	videoSettings.bitRate = mamimumVideoBitRate
	try? await stream.setVideoSettings(videoSettings)
	}
	}
	}

await stream.setBitRateStrategy(StreamVideoAdaptiveBitRateStrategy())

About MediaLink

This refers to processing that occurs when playback is being done with HaishinKit.
Is it okay to focus on the case during streaming(publish) this time?

[noho501]

Hi,

Sorry, I misunderstood MediaLink — it’s not related to my case.

I’m currently using a custom BitRateStrategy like this, but the problem still there:

let bitRateStrategy = LiveStreamBitRateStrategy()
await bitRateStrategy.setOnBitrateUpdate { [weak self] kbps in
    await self?.notifyBitrateUpdate(kbps: kbps)
}
Task {
    await stream.setBitRateStrategy(bitRateStrategy)
}

final actor LiveStreamBitRateStrategy: StreamBitRateStrategy {
    
    let mamimumVideoBitRate: Int = 4000 * 1000
    let mamimumAudioBitRate: Int = 0
    
    public static let statusCountsThreshold: Int = 15
    private var sufficientBWCounts: Int = 0
    private var zeroBytesOutPerSecondCounts: Int = 0
    
    private var onBitrateUpdate: ((Int) -> Void)?
        
    func setOnBitrateUpdate(_ callback: @escaping (Int) -> Void) {
        self.onBitrateUpdate = callback
    }
    
    func adjustBitrate(_ event: HaishinKit.NetworkMonitorEvent, stream: some HaishinKit.StreamConvertible) async {
        switch event {
        case .status(let report):
            let kbps = Int(report.currentBytesOutPerSecond * 8) / 1000
            onBitrateUpdate?(kbps)
            
            var videoSettings = await stream.videoSettings
            if videoSettings.bitRate == mamimumVideoBitRate {
                return
            }
            if Self.statusCountsThreshold <= sufficientBWCounts {
                let incremental = mamimumVideoBitRate / 10
                videoSettings.bitRate = min(videoSettings.bitRate + incremental, mamimumVideoBitRate)
                try? await stream.setVideoSettings(videoSettings)
                sufficientBWCounts = 0
            } else {
                sufficientBWCounts += 1
            }
           // print("[adapter] sufficientBWCounts", sufficientBWCounts, videoSettings)
        case .publishInsufficientBWOccured(let report):
            let kbps = Int(report.currentBytesOutPerSecond * 8) / 1000
            onBitrateUpdate?(kbps)
            
            sufficientBWCounts = 0
            var videoSettings = await stream.videoSettings
            if 0 < report.currentBytesOutPerSecond {
                let bitRate = Int(report.currentBytesOutPerSecond * 8) / (zeroBytesOutPerSecondCounts + 1)
                videoSettings.bitRate = max(bitRate - videoSettings.bitRate, mamimumVideoBitRate / 10)
                videoSettings.frameInterval = 0.0
                sufficientBWCounts = 0
                zeroBytesOutPerSecondCounts = 0
            } else {
                switch zeroBytesOutPerSecondCounts {
                case 2:
                    videoSettings.frameInterval = VideoCodecSettings.frameInterval10
                case 4:
                    videoSettings.frameInterval = VideoCodecSettings.frameInterval05
                default:
                    break
                }
                zeroBytesOutPerSecondCounts += 1
            }
            try? await stream.setVideoSettings(videoSettings)
           // print("[adapter] zeroBytesOutPerSecondCounts", zeroBytesOutPerSecondCounts, videoSettings)

        case .reset:
            var videoSettings = await stream.videoSettings
            zeroBytesOutPerSecondCounts = 0
            videoSettings.bitRate = mamimumVideoBitRate
            try? await stream.setVideoSettings(videoSettings)
        }
    }
}

I understand that when the network becomes unstable, unsent frames may remain on the device side and never reach the server.

So, Is there any parameter or API to check how many frames (or bytes) are still pending to be sent before calling stop? This would allow us to send all remaining data before stopping the stream, and help prevent losing the last part of the video.

Thank you very much for your help 🙏

[shogo4405]

The value obtained here, report.currentQueueBytesOut: Int, represents the number of bytes that have not yet been sent.
It’s updated every second. You can store this value and check it at the time of stopping.

• currentBytesOutPerSecond is the actual measured bitrate of the current transmission.
• you can estimate how many seconds’ worth of data is queued up.
  • currentQueueBytesOut / currentBytesOutPerSecond

[noho501]

Thank you so much