Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Memory leak when InbandEventStream is set in the AdaptationSet of the live stream manifest. #4561

Open
4 tasks done
grand719 opened this issue Aug 30, 2024 · 7 comments
Open
4 tasks done

Memory leak when InbandEventStream is set in the AdaptationSet of the live stream manifest. #4561

grand719 opened this issue Aug 30, 2024 · 7 comments
Assignees
@dsilhavy
Labels
Bug Pri-Low
Milestone
5.0.0

Comments

@grand719
Copy link

Environment
Steps to reproduce
  1. Prepare Stream with AWS Elemental MediaPackage
  2. Run the prepared stream, which contains the <InbandEventStream schemeIdUri="https://aomedia.org/emsg/ID3" value="0"/> field in the AdaptationSet, in the referenced player."
  3. Take a Heap Snapshot and Analyze JsArrayBuffer
Observed behavior

In the taken snapshot, we can observe that the JsArrayBuffer is consistently increasing, and the memory usage is not dropping. During the debug session, I noticed that unusual memory management behavior is caused by the <InbandEventStream schemeIdUri="https://aomedia.org/emsg/ID3" value="0"/> element, which is added to the video AdaptationSet. Additionally, after deleting the InbandEventStream and InbandEventStream_asArray properties from the adaptation set object in DashManifestModel.js within the getRealAdaptations function, everything works as intended.

Memory usage when Adaptation Set contains InbandEventStream:
image

Manifest
<?xml version="1.0" encoding="utf-8"?>
<MPD xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="urn:mpeg:dash:schema:mpd:2011" xsi:schemaLocation="urn:mpeg:dash:schema:mpd:2011 http://standards.iso.org/ittf/PubliclyAvailableStandards/MPEG-DASH_schema_files/DASH-MPD.xsd" id="201" type="dynamic" publishTime="2024-08-30T10:09:25+00:00" minimumUpdatePeriod="PT2S" availabilityStartTime="2024-08-29T14:57:30+00:00" minBufferTime="PT10S" suggestedPresentationDelay="PT20.000S" timeShiftBufferDepth="PT37.000S" profiles="urn:mpeg:dash:profile:isoff-live:2011">
  <Period start="PT0.000S" id="1">
    <AdaptationSet id="1485523442" mimeType="video/mp4" segmentAlignment="true" startWithSAP="1" subsegmentAlignment="true" subsegmentStartsWithSAP="1" bitstreamSwitching="true">
      <InbandEventStream schemeIdUri="https://aomedia.org/emsg/ID3" value="0"/>
      <Representation id="1" width="1920" height="1080" frameRate="30/1" bandwidth="2000000" codecs="avc1.640028">
        <SegmentTemplate timescale="30000" media="index_video_3_0_$Time$.mp4?m=1724934975" initialization="index_video_3_0_init.mp4?m=1724934975" startNumber="15912">
          <SegmentTimeline>
            <S t="2072232040" d="180000" r="6"/>
          </SegmentTimeline>
        </SegmentTemplate>
      </Representation>
      <Representation id="2" width="1280" height="720" frameRate="30/1" bandwidth="1000000" codecs="avc1.64001F">
        <SegmentTemplate timescale="30000" media="index_video_5_0_$Time$.mp4?m=1724934975" initialization="index_video_5_0_init.mp4?m=1724934975" startNumber="15912">
          <SegmentTimeline>
            <S t="2072232040" d="180000" r="6"/>
          </SegmentTimeline>
        </SegmentTemplate>
      </Representation>
      <Representation id="3" width="853" height="480" frameRate="30/1" bandwidth="499968" codecs="avc1.4D401E">
        <SegmentTemplate timescale="30000" media="index_video_7_0_$Time$.mp4?m=1724934975" initialization="index_video_7_0_init.mp4?m=1724934975" startNumber="15912">
          <SegmentTimeline>
            <S t="2072232040" d="180000" r="6"/>
          </SegmentTimeline>
        </SegmentTemplate>
      </Representation>
      <Representation id="4" width="426" height="240" frameRate="30/1" bandwidth="249984" codecs="avc1.4D400D">
        <SegmentTemplate timescale="30000" media="index_video_9_0_$Time$.mp4?m=1724934975" initialization="index_video_9_0_init.mp4?m=1724934975" startNumber="15912">
          <SegmentTimeline>
            <S t="2072232040" d="180000" r="6"/>
          </SegmentTimeline>
        </SegmentTemplate>
      </Representation>
    </AdaptationSet>
    <AdaptationSet id="2083690061" mimeType="audio/mp4" segmentAlignment="true" startWithSAP="1" lang="und" bitstreamSwitching="true">
      <Label>und</Label>
      <Representation id="5" bandwidth="192357" audioSamplingRate="48000" codecs="mp4a.40.2">
        <AudioChannelConfiguration schemeIdUri="urn:mpeg:dash:23003:3:audio_channel_configuration:2011" value="2"/>
        <SegmentTemplate timescale="48000" media="index_audio_4_0_$Time$.mp4?m=1724934975" initialization="index_audio_4_0_init.mp4?m=1724934975" startNumber="15912">
          <SegmentTimeline>
            <S t="3315571968" d="287744" r="1"/>
            <S t="3316147456" d="288768"/>
            <S t="3316436224" d="287744" r="2"/>
            <S t="3317299456" d="288768"/>
          </SegmentTimeline>
        </SegmentTemplate>
      </Representation>
      <Representation id="6" bandwidth="192357" audioSamplingRate="48000" codecs="mp4a.40.2">
        <AudioChannelConfiguration schemeIdUri="urn:mpeg:dash:23003:3:audio_channel_configuration:2011" value="2"/>
        <SegmentTemplate timescale="48000" media="index_audio_6_0_$Time$.mp4?m=1724934975" initialization="index_audio_6_0_init.mp4?m=1724934975" startNumber="15912">
          <SegmentTimeline>
            <S t="3315571968" d="287744" r="1"/>
            <S t="3316147456" d="288768"/>
            <S t="3316436224" d="287744" r="2"/>
            <S t="3317299456" d="288768"/>
          </SegmentTimeline>
        </SegmentTemplate>
      </Representation>
      <Representation id="7" bandwidth="128651" audioSamplingRate="48000" codecs="mp4a.40.2">
        <AudioChannelConfiguration schemeIdUri="urn:mpeg:dash:23003:3:audio_channel_configuration:2011" value="2"/>
        <SegmentTemplate timescale="48000" media="index_audio_8_0_$Time$.mp4?m=1724934975" initialization="index_audio_8_0_init.mp4?m=1724934975" startNumber="15912">
          <SegmentTimeline>
            <S t="3315571968" d="287744" r="1"/>
            <S t="3316147456" d="288768"/>
            <S t="3316436224" d="287744" r="2"/>
            <S t="3317299456" d="288768"/>
          </SegmentTimeline>
        </SegmentTemplate>
      </Representation>
      <Representation id="8" bandwidth="128651" audioSamplingRate="48000" codecs="mp4a.40.2">
        <AudioChannelConfiguration schemeIdUri="urn:mpeg:dash:23003:3:audio_channel_configuration:2011" value="2"/>
        <SegmentTemplate timescale="48000" media="index_audio_10_0_$Time$.mp4?m=1724934975" initialization="index_audio_10_0_init.mp4?m=1724934975" startNumber="15912">
          <SegmentTimeline>
            <S t="3315571968" d="287744" r="1"/>
            <S t="3316147456" d="288768"/>
            <S t="3316436224" d="287744" r="2"/>
            <S t="3317299456" d="288768"/>
          </SegmentTimeline>
        </SegmentTemplate>
      </Representation>
    </AdaptationSet>
    <SupplementalProperty schemeIdUri="urn:scte:dash:utc-time" value="2024-08-29T14:57:25.400Z"/>
  </Period>
  <UTCTiming schemeIdUri="urn:mpeg:dash:utc:http-iso:2014" value="https://time.akamai.com/?iso&amp;ms"/>
</MPD>
Expected behavior

There should be no memory leaks when playing a live stream, even with InbandEventStream set in the AdaptationSet. Memory should be released correctly, and the size of the JsArrayBuffer should remain stable or decrease as unneeded resources are freed.
@grand719 grand719 added the Bug label Aug 30, 2024
@grand719
Copy link
Author

grand719 commented Sep 2, 2024

After further investigation, I managed to find out that the issue occurs due to the event_duration being set to 40 hours in IsoBox - emsg. How should this situation be approached, and how should the player react in such a case? We also managed to remove the emsg IsoBox using AWS MediaLive, but I believe the player should also have logic in place to prevent behavior like overloading the memory when events are too long.

@dsilhavy dsilhavy added this to the 5.1.0 milestone Sep 23, 2024
@dsilhavy dsilhavy mentioned this issue Oct 10, 2024
Merged
@haudiobe
Copy link

I believe inband events should not have any duration that is longer than the containing segment. We should restrict it in MPEG to this. Also in case the event duration is longer than a segment, the event payload should be deleted on the client.

@ZmGorynych
Copy link

Events can legitimately be longer (e.g., duration of an ad break). They can appear before their presentation time.
I would rather do (maybe tunable) timeouts in the player.

@dsilhavy
Copy link
Collaborator

@grand719 Do you still have a stream running with the ID3 events? I don't see any events in the example you linked above.

@grand719
Copy link
Author

@dsilhavy Unfortunately, we no longer provide this stream with ID3 events and won't be able to set it up again. Regarding the configuration in AWS, we used the default settings in MediaPackage and MediaLive to create the stream.

@dsilhavy dsilhavy modified the milestones: 5.1.0, 5.0.0 Jan 13, 2025
@dsilhavy dsilhavy mentioned this issue Jan 13, 2025
Open
@dsilhavy
Copy link
Collaborator

Thanks for the quick feedback @grand719 . I suggested a basic fix in #4661 . Please let me know if you have any feedback

@grand719
Copy link
Author

Sure, I'll try to check it this week and let you know. Thank you!

@dsilhavy dsilhavy moved this to In Progress in dash.js Version 5.0.0 Jan 15, 2025
@dsilhavy dsilhavy self-assigned this Jan 15, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@dsilhavy dsilhavy

Labels
Bug Pri-Low
Projects
dash.js Version 5.0.0
Status: In Progress
Milestone
5.0.0
Development

No branches or pull requests
4 participants
@ZmGorynych @dsilhavy @haudiobe @grand719