Memory managment in MP4Box.js

The memory consumption of MP4Box.js is highly dependent on the size of the input file and on parameters/behavior when using MP4Box.js.

File types

There are broadly 2 types of ISOBMFF files:

• files with the moov box after the mdat box(es). These files are easy to produce (default in ffmpeg, requires -flat in MP4Box.exe) and widespread. They are not ideal for use across a network in a browser and need a bit more work when using mp4box.js if memory usage matters (i.e. if the file is large).
• files with the moov box before the mdat box(es). They are a bit harder to produce (typically 2-pass). They are produced by default in MP4Box.exe command line but you need a faststart option in ffmpeg. Note that they can be fragmented or not.

MP4Box.js allocation behavior and logging

There are 2 main sources of important memory allocations in MP4Box.js. They are all related to media data:

• Buffers passed to MP4Box.js (e.g. by a file reader or a downloader, outside of MP4Box.js). They can be kept in memory to allow for later processing of samples, but can become very large. So, unless indicated otherwise, the actual usage of the data in the buffer is tracked and when all bytes have been read, the buffer is deallocated at the first opportunity. 'Read' means either box data is parsed to create a box or sample data is parsed to create a sample (for extraction or segmentation). The logs shows lines like the following to indicate how many buffers are kept, the used size, the total size and the ranges of the original file covered by these buffers:

[MultiBufferStream] 1064 stored buffer(s) (347109/1209461457 bytes), continuous ranges: [0-1209461456]

• Buffers created by MP4Box.js when processing samples (either during extraction or segmentation). The logs indicate lines like the following to indicate this memory usage:

[ISOFile] Sample data size in memory: 14091371

MP4Box.js allocation controls

There are 2 ways to tell MP4Box.js how to handle memory usage based on the above 2 sources of memory uses.

The createFile constructor

It takes a parameter which indicates if the media data shall be kept in memory while parsing.

Example: If you take a large file (e.g >1GB) (with moov before or after mdat), and you just pass the buffers to MP4Box.js without doing anything with the data (no segmentation, no extraction of samples), by default MP4Box.js will automatically release the buffers and display the following at the end of the parsing:

[0:00:14.965] [ISOFile] Sample data size in memory: 0
[0:00:14.970] [ISOFile] Flushing remaining samples
[0:00:14.970] [MultiBufferStream] No more buffer in memory

I also verified the memory consumption with Google Chrome's memory tracker and it showed ~13MB allocated in the MP4Box object, mostly consumed by the sample tables.

Now, if you do want to keep the data for some reason, use mp4 = MP4Box.createFile(true), the end of the parsing will show something like:

[0:00:17.467] [ISOFile] Sample data size in memory: 0
[0:00:17.539] [ISOFile] Flushing remaining samples
[0:00:17.547] [MultiBufferStream] 1074 stored buffer(s) (344261/1209458601 bytes), continuous ranges: [0-1209458600]

Chrome's memory tool confirms the memory usage is about 1.2GB. And you can see that I passed 1074 buffers all forming a single range of 1209458601 bytes and that 344261 bytes of them were marked as used, roughly making the size of all the boxes except mdat.

Releasing sample data

When you configure MP4Box.js to extract media samples or to create media segments, MP4Box.js allocates sample data that you need to release when you are done with the data. This is the role of releaseUsedSamples.

If you take the example above, if you indicate that you don't want to keep the data after parsing (assuming the file has moov before mdat), segment the file but don't release the samples, you'll get something like the following:

[0:01:09.222] [ISOFile] Sample data size in memory: 1209114340
[0:01:09.224] [ISOFile] Flushing remaining samples
[0:01:09.226] [MultiBufferStream] 1 stored buffer(s) (2998015/1499049 bytes), continuous ranges: [1207959552-1209458600]

And Google Chrome's memory snapshot will tell you the page consumes about 1.2GB of memory. (yes, there is a problem with MultiBufferStream log line...)

But if you actually call releaseUsedSamples after each onSegment callback, you should get:

[0:01:06.466] [ISOFile] Sample data size in memory: 0
[0:01:06.469] [ISOFile] Flushing remaining samples
[0:01:06.471] [MultiBufferStream] 1 stored buffer(s) (2998015/1499049 bytes), continuous ranges: [1207959552-1209458600]

And your memory inspector should end with about 14.4 MB used.

Note that if your file has the moov box at the end of the file, all media data will be released before MP4Box.js had any chance of creating the segments. The onSegment callback will never be called!! You'll get a small memory usage (~12M in my example) and log ending like this:

[0:01:06.614] [ISOFile] Sample data size in memory: 1209114340
[0:01:06.617] [ISOFile] Flushing remaining samples
[0:01:06.617] [MultiBufferStream] No more buffer in memory

The sample data size in memory is likely due to the fact that the last buffer passed to MP4Box.js had the moov box but possibly also some of the last track samples.

To avoid this situation, either set the createFile constructor to keep the media data (if the file is small enough); or use the byte offsets returned by appendBuffer until you get the onReady (or onMoov) callbacks and then start fetching the media data from the missing byte ranges of the file.

Finally, when your file has multiple tracks but you are only interested in segmenting or extracting some of them, if the moov box is before the mdat box(es), the data from the other tracks will automatically get garbage collected. In the case the mdat box(es) is (are) before the moov box, and you decide to use the approach to keep the media data, you'd have to manually extract the samples from the other track and release them.

Conclusion

You can experiment with the above controls in this simple file reader with optional segmentation: https://gpac.github.io/mp4box.js/test/file-segmenter.html

If you have other use cases or if you experience does not reflect what is described here, then maybe MP4Box.js is missing something or has a bug. In that case, consider opening an issue (with detailed description of the issue and steps/code and files to reproduce) and even better create a pull request to fix the bug.