How to ensure track alignment on ABR

[irg1008]

Hi! (You must think I am crazy at this point)

Well I am trying and succeding (sometimes) to create multiple qualities for a stream. It works with stream proccesses both with ffmpeg and AV. The problem is that consuming the streaming with HLS (hls.js) sometimes receives all the video tracks sometimes just the original one:

index.m3u8

#EXTM3U
#EXT-X-INDEPENDENT-SEGMENTS
#EXT-X-MEDIA:TYPE=AUDIO,GROUP-ID="aud",LANGUAGE="und-1",NAME="AAC-1",URI="1/index.m3u8?mTrack=2&iMsn=67&tkn=668057077"
#EXT-X-STREAM-INF:AUDIO="aud",CODECS="avc1.64002a,mp4a.40.2",RESOLUTION=1920x1080,FRAME-RATE=60,BANDWIDTH=10618598,AVERAGE-BANDWIDTH=8984967
2/index.m3u8?mTrack=2&iMsn=67&tkn=668057077
## NOTE: Track 3 is available, but ignored because it is not aligned with track 2.
## NOTE: Track 4 is available, but ignored because it is not aligned with track 2.

Sometimes the tracks are able to align and I get all tracks and so I am able to swap between them.

My Hls config in the frontend is this one:

const hlsConfig: Partial<HlsConfig> = {
  debug: false,
  enableWorker: true,
  lowLatencyMode: true,
  backBufferLength: 90,
  maxBufferLength: 300,
}

A stream proccess for 720p with ffmpeg looks like this:

Sometimes it just works, then you load the stream event and it doesn't anymore.

Is there any way to ensure track alignment even if it takes a bit for the player to load the stream??

Thank you and I know I am a pain in the ass!! Have a nice day

[Rokamun]

Hmm, are there multiple encodes on the same stream?
MistServer will by default take the "last" added live track and use that as a source. So you might need to mask every encode from processes. You can do that by setting the Source track mask to not be usable for processes. That way you can never re-encode from an already encoded stream.
Chances are that the times it worked sometimes are when all processes managed to start encoding from the original source track.

Normally you should be able to force keyframe alignment with the -force_key_frames source which you set by the keyframes Match input keyframes which does make it a tad weird that it is not aligning though...

[irg1008]

It seems opting out on proccessing is not available

I opted out from the output track for proccessing but that didn't work, still getting misaligned tracks, maybe I have too many stream proccesses?

[irg1008]

I think I am able to "fix" this for now by encoding the original stream in the same quality. Example OBS streams 1080p, I encode into another ouput with same quality. Those then are always (or at least seems to be) algined. The problem is you have one more proccess for each stream

[irg1008]

The only problem facing know it's the bitrate it's absolutely ignored. For example I set 6000k minimun bitrate and output has 1200k, but I guess that's a problem for another day, and tomorow it's friday, so I guess another week haah

[Rokamun]

Heya,

I see I said it a bit confusing. So let me make it a bit more clear, as I definitely could do better there:

You need the Source track to only be available for processing
Then all the output tracks available for viewers (and optionally processing)

Even easier honestly is to just make the whole bitrate tree a single command through MKV-exec. It would be more reliable considering you can then at least force all those qualities to have the same keyframe interval. The multiple processes clearly isn't doing well for you:

ffmpeg -loglevel quiet -hide_banner -fflags nobuffer -i -  -map v:0 -map v:0 -map v:0 -map a:0  -c:v:0 h264 -b:v:0 5000k -s:v:0 1920x1080 -c:v:1 h264 -b:v:1 2000k -s:v:1  1280x720  -c:v:2 h264 -b:v:2 1000k -s:v:2 854x480 -c:a copy -force_key_frames source -cluster_time_limit 0 -f matroska -

Chances are your CPU will not be happy with this, you can try to alleviate that by setting -preset veryfast right behind -c:a copy or using a hardware encoder like h264_nvenc at every place it says h264 assuming you have an Nvidia card available.

That should generate a video with the following qualities:

• 1080p at ~5000kbps
• 720p at ~2000kbps
• 4380p at ~1000kbps

The reason you see the bitrate "ignored" is because FFmpeg will by see the bitrate as a target, if it's capable of doing the target quality with lesser bitrate it will. You can counter this with setting a constant bitrate.

That would be adding a -minrate X , -maxrate X and -bufsize x to every -b:v:0/1/2

-minrate X this is the target minimum bitrate,
-maxrate X this is the target maximum bitrate,
-bufsize X This is the size of the buffer it can use to cut these up. Usually target 1 to 5 times the target bitrate in order to make sure the target bitrate can be met.

But honestly I would first try it without the minrate/maxrate/bufsize and see if you're happy with that quality.

[irg1008]

Hi, sorry for the late reply. This absolutely fixes both the bitrate and track alignment. The only problem now It's I am not able to inhibit some track based on source width and height.

Before I would use your built-in solution like:
"source_track": ">1920x1080,~1920x1080",

But now with the single ffmpeg command I don't know how to approach this. Using ffmprobe seems to consume the stream and ffmpeg doesn't work

[Thulinma]

You can still do that with the Source selector(s): option (called track_select in the JSON config). Since the MKV process can take any combination of video and/or audio, you'd need to set a selector for both. For example: "track_select": "video=>1920x1080,~1920x1080&audio=none" would select video the same way you did before, and not select any audio tracks. (Though you likely instead want video=maxbps&audio=none in your case, to select the highest quality video track available and no audio.)

[irg1008]

I went with that and it works great. One thing that would make this greate would be to be able to pass video input resolution to a custom script somehow. I tried using ffprobe and it feeds with the input video and ffmpeg does not work anymore. This works for certain fixed resolution but there is no way I know of to mantain both the input video resolution (transcoded for alignment) and new downscaled streamings.

Example: Input is 1536x864. Trancode 1536x864 and generate all downscaled streamings down to 854x480