Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add creation option LEVEL=MAX #11598

Open
Rdataflow opened this issue Jan 7, 2025 · 6 comments
Open

add creation option LEVEL=MAX #11598

Rdataflow opened this issue Jan 7, 2025 · 6 comments

Comments

@Rdataflow
Copy link

Feature description

for different compressions (LZW, ZSTD, ...) there exists different numbers of LEVELs.
As a data provider I'd frequently use MAX i.e. (22 for ZSTD, or whichever is the maximum for other compressions chosen).
It would be easier to explain and remember an expression like MAX 😃

Additional context

example of how it would be useful

gdal_translate in.tif out.tif -of COG -co COMPRESS=ZSTD -co LEVEL=MAX
@jratike80
Copy link
Collaborator

On the other side, MAX tends to be awfully slow with nominal benefit and sometimes even yields bigger files than some lower value. Simple syntax could encourage users to use max without thinking first ;)

@rouault
Copy link
Member

rouault commented Jan 7, 2025

The max value for the level parameter is compression dependent, hence it is better to have a dedicated setting per compression method.

@rouault rouault closed this as completed Jan 7, 2025
@jratike80
Copy link
Collaborator

Actually, @Rdataflow knew that there are different maximums

22 for ZSTD, or whichever is the maximum for other compressions chosen

By the documentation it would be either 9 or 12 for deflate depending on the library. LZW does not have such option.

@Rdataflow
Copy link
Author

@rouault it is indeed intentional part of the userstory:

  • As a open data producer and publisher I can afford slow compression while striving for lower storage and bandwidth consumption and keeping decompression speed at acceptable rate.

In this context I d like a handy and hopefully stable option to max compression level to easier instruct our data providers.

How can one generically explain wheter max is at 9 or 12 because it depends on how your distro got compiled? Not to imagine the next compression algorithm introduces new levels.

While it should not become the default setting I d still vote to have MAX available for such use cases of compress once, decompress often.

@rouault
Copy link
Member

rouault commented Jan 7, 2025

Re-opening (I read too fast initially missing the MAX aspect, and just the LEVEL single option), although I don't plan personally for immediate action on this.

@rouault rouault reopened this Jan 7, 2025
@jratike80
Copy link
Collaborator

It seems that there is not extremely much to remember. For ZSTD use ZSTD_LEVEL=22 and for deflate use ZLEVEL=12.

How can one generically explain wheter max is at 9 or 12 because it depends on how your distro got compiled?

I don't know if this is a bug or a feature, but there is no need to explain that. My GDAL from OSGeo4W comes with zlib and the max is then 9, but zlevel=12 is still accepted and the result is identical to zlevel=9. Using zlevel=22 for deflate yields an error "Warning 5: ZLEVEL=22 value not recognised, ignoring", and the result is using the default zlevel=6.

From other compression methods only JXL seems to have a similar option for selecting how hard to work for a minimal lossless result. I had a try with JXL_EFFORT=9 on a RGB orthophoto and it does make a difference to the default jxl_effort=5 but for a very high price, so slow it is. The JXL size shrinked from 143 446 039 to 134 017 833 bytes. As a reference,
gdal_translate -of gtiff -co tiled=yes -co compress=zstd -co zstd_level=22 -co predictor=2 p4433h.tif zstd_22_pred2.tif gives 227 372 684 bytes. Or 279 559 483 bytes with the default predictor.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants