feat: texture splitting/stacking, JPEG XL/DDS images, GLES3.0 #66
+2,629
−807
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
As GetImageInfo had no callers in the codebase and makes introduction of new image codecs harder, it could and should be removed.
BLP has never been used in Path of Building's modern history, so it's time for it to go.
This change introduces the ability to load JPEG XL files. It is made to handle files with 8 bits per channel, either as greyscale, RGB or RGBA. For distribution some number of DLLs and license files need to be bundled with the runtime. The licenses involved are: Brotli - MIT Highway - Apache-2.0, BSD-3-Clause libjxl - BSD-3-Clause Little-CMS - MIT The required DLLs are: brotlicommon.dll brotlidec.dll brotlienc.dll hwy.dll jxl.dll jxl_cms.dll lcms2.dll
Historically, Image handles represented an image resident in video memory and were sourced from individual files on disk. As the game's passive tree connectors are nested tightly in a single file, this means that drawing such irregularly shaped would need richer geometric primitives than what SimpleGraphic currently supports. This change introduces an Art handle which represents an image in system memory, making it available to image manipulation algorithms. It can be used as an input to new texture loading functions in Image intended to be able to handle the two kinds of connectors present in the connector art: * `LoadArtRectangle` - for slicing out the line connector; * `LoadArtArcBand` - for slicing out the stacked orbital arc connectors. Rectangles are defined by their corners where the upper left corner is inclusive and the lower right corner is exclusive. Arc bands are defined by the coordinate for the centre of the arc and by inner and outer radii. The arc is always cut top-left of the centre.
|
New functions exposed to Lua for reference, also documented in ** artHandle = NewArtHandle("<filename>")
** width, height = artHandle:Size()
**
** imgHandle:LoadArtRectangle(art, x1, y1, x2, y2[, "flag1"[, "flag2"... ]]) flag:{"CLAMP"|"MIPMAP"}
** imgHandle:LoadArtArcBand(art, xC, yC, rMin, rMax[, "flag1"[, "flag2"... ]]) flag:{"CLAMP"|"MIPMAP"} |
As JPEG XL comes in via PkgConfig there's no immediate way to auto- install the runtime libraries needed for it. We resort to enumerating them explicitly on Windows like LuaJIT and ZLIB before.
Some textures are stretched into a very elongated form like the line connectors. This causes mip selection to favour a coarser mip level than desired and will in many cases lose all detail across the lines. By bumping up anisotropic filtering to at most 16x, this effect is greatly reduced. An even better result could be gotten by repeating the texture across the major axis to avoid greatly differing U and V deltas.
The reference JPEG XL library has the ability to parallelise the word of decoding individual images. We use that here as a stopgap measure for costly image loading at startup to try to cut down the time spent. When/if we reintroduce async texture loading, this functionality may need to be disabled to avoid oversubscription of CPU cores. Note: this requires distribution of an additional `jxl_threads.dll` DLL.
Mipmap calculations used to assume linear colour values while most 8-bit source art is in sRGB. Instead of our handcrafted resampling and mipmap halving logic we can outsource the effort to a battle-hardened `stb_image_resize.h` library. This also enables us to have mipmapped textures that are not powers of two as those are widely supported on hardware and drivers these days, avoiding a lot of surprise upsampling that our runtime performed for oddly sized textures.
This change reintroduces the concept of async texture uploading. Async loading of images is now capable for communicating the image size earlier than before letting Lua-side code stall for shorter durations. As we do not have shared GL contexts anymore we have to separate the texture loading into multiple phases. We now push the initial file load and mipmap calculation off onto workers and introduce a new main thread step each frame where all pending GPU textures are created and populated from a queue of prepared texture payloads. This had some fallout in the renderer as it has to consider any frames generated while textures are loading as ineligible for elision as the render may differ before/after they're ready.
By wrapping an unique_ptr around a raw pointer argument in shader loading we accidentally attempted to delete a stack-local image_c object as the caller assumed that the callee would take ownership of the internals but not the object itself. This trap was removed by replacing the whole sequence of arguments with an unique_ptr throughout.
The mipset that was squirreled away in the r_tex_c object was never released after the upload had completed, resulting in it taking up space for the remainder of the texture handle's life.
If we want to load and carry around image swith mipmaps and array layers from disk we need a more flexible data format than a single byte array. As GLI is a suitable library for loading assets from DDS in the first place we might as well make gli::texture2d our internal carrier format for image internals and properties. This commit removes the previous `dat` member of `image_c`, superseding it with a GLI texture2d member. We also take the opportunity to do some drive-by cleanup and fixing non-virtual dynamic destruction of image_c objects.
In order to be able to guarantee larger texture dimensions and be able to use texture arrays, this change sets the ANGLE backend to Direct3D11 and bumps the OpenGL ES version to 3.0. This raises the minimum spec a bit.
In order to use ES3 features like texture arrays in shaders we need to update the shader language level to `300 es`. Tinted rendering texcoords now have the layer as their third component.
Using GLI's texture2d_array class to carry image data allows us to now represent texture arrays and mipmap sets in a single image object while also indicating the data type more precisely.
The new functions in `core_tex_manipulation` can be used to construct a handle object that can perform light image manipulation, currently just stacking of DDS textures of the same extents and formats into a layered DDS texture. Image loading has gained the ability to load DDS files, optionally compressed with Zstandard as `.dds.zst`.
sol2 lets us do way more C++:y bindings to Lua, here used for exposing the texture manipulation class used to stack DDS textures and query image information.
The renderer is capable of binding and drawing individual texture layers from 2D texture arrays. In the exporter and API we call these "stacked" textures and as such the parameter is the "stack layer". This improves on sprite sheets in that an individual image (quad) draw can pick from any layer of its texture while retaining the ability to have different wrapping/clamping modes and keeping any original mipmaps from before stacking. As this complicated the draw call parsing those are now parsed more modularly based on the number of arguments to also handle stack layers and also any possible future "mask" functionality we might want to draw a subset of an image. The immediate render interface now operates on vec2 values as the argument counts were becoming ridiculous.
In some circumstances like loading another texture into an image handle some shader slots could become NULL and break code that assumed that all valid handles were densely packed.
Until we can bump the C++ version to have a native `<span>`, this'll do.
Unless explicitly prohibited via `TF_NOMIPMAP` we should build mipmaps for textures destined for the GPU. This currently excludes single-mip block-compressed textures as we would need to decompress them or use the GPU to do so.
Mipmaps computed with `gli::generate_mipmaps` are a bit too dim and unreadable at mid to far zoom levels. Use sRGB-based resizing with `stb_image_resize` for now and revisit colour spaces later.
DrawImage and DrawImageQuad now supports a final maskIdx that indicates which texture in the stack to multiply the final output with; adding the ability to mask out undesired parts of the image. As it needs to be in the same stack, the mask image needs to have the same size and image format as the colour image.
zao
changed the title
The Direct3D11 backend of ANGLE seems to have some trouble with the stock Wine runtime where textures can fade into black like the big class/ascendancy art in the middle of the tree at particular zoom levels and also for error dialog boxes. This can be mitigated by using the DXVK runtime with a Vulkan driver. A cleaner more in-box workaround is to use the Desktop OpenGL backend of ANGLE if we detect that the Windows version of SimpleGraphic runs on top of Wine.
MacOS doesn't support a cool enough desktop OpenGL version for us to launch when using the Desktop OpenGL backend of ANGLE. We now detect the presence of Wine and query it for the host platform (analogous to `uname`). On Wine macOS we select D3D11. This platform requires use of libraries like D3DMetal/GPTK in Kegworks to emulate D3D11 enough. On Wine Linux we select Desktop OpenGL as D3D11 requires use of DXVK to function well enough. On proper Windows we select D3D11 and for any other platforms we don't indicate any preference for now.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR adds a new type of "Art" handle exposed to Lua that represents a CPU-side image to complement the GPU-side "Image" handle. These Art handles can be used as the source for populating Image handles by cutting them into rectangles or arc bands. The intent is to support the rendering of passive tree connectors by splitting the art out from source images with nested artwork.
The renderer has been updated to OpenGL ES 3.0 on ANGLE's D3D11 backend. This raises the minimum requirements to around D3D feature level 10_0. It now uses anisotropic filtering to provide sharper sprite texturing when squashed or stretched non-proportionally.
On Linux under Wine the backend is overridden to be based on desktop OpenGL, as Direct3D11 requires DXVK and thus a Vulkan driver to function there.
While touching the image infrastructure, this PR also introduces decoding of the JPEG XL image format to allow for a smaller disk footprint for bundled assets as it allows for lossy compression. The PR also cleans up some stale unused functionality.
The image storage has been unified around GLI's
texture2d_arraytype across all file formats and support for DDS 2D textures and 2D texture arrays has been added, these DDS files may have optional Zstandard` compression .All image drawing now uses 2D texture arrays and allows the Lua code to select which "stack layer" of a texture to use for the colour texture. Similarly a new mask can be selected from the same texture array to multiply with the sampled colour. This supports the use case of generating masks images offline to mask off the nested arc connector art.
The Lua API has been augmented with functions to build these "stacked textures" during the export phase. In order to stack textures the images going in to it need to have the same extents and the same DDS pixel format (RGB, RGBA, BC1, BC7). It can export these stacks either as plain uncompressed DDS files
.ddsor as Zstandard-compressed.dds.zstfiles.Implementation-wise this change adopts ThePhD's sol3 for binding functionality to Lua, used for the texture stacking code as a proof of concept.
The runtime has limited mipmap generation for uncompressed pixel formats and performs it in sRGB space.
See individual commit descriptions for deeper detail.