Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

qt6.qtwebengine: fix build on darwin, disable metal shader compilation #359755

Merged
merged 2 commits into from
Jan 8, 2025

Conversation

azuwis
Copy link
Contributor

@azuwis azuwis commented Nov 28, 2024

Currently qt6.qtwebengine failed on Hydra with this log:

[5018/29770] ACTION //third_party/angle/src/libANGLE/renderer/metal:angle_metal_internal_shaders_to_air(//build/toolchain/mac:clang_arm64)
FAILED: gen/angle/mtl_internal_shaders_autogen.air 
/nix/store/rjk36fiki39274zlidarq3y14bm585pi-python3-3.12.7-env/bin/python3 ../../../../../src/3rdparty/chromium/third_party/angle/src/libANGLE/renderer/metal/shaders/metal_wrapper.py /nix/store/h3skf4182jjbmli12nqggaj0x5kqq63l-apple-sdk-15.0/Toolchains/XcodeDefault.xctoolchain/usr/bin/metal -c ../../../../../src/3rdparty/chromium/third_party/angle/src/libANGLE/renderer/metal/shaders/mtl_internal_shaders_autogen.metal -o gen/angle/mtl_internal_shaders_autogen.air --std=macos-metal2.1 -mmacosx-version-min=10.14
Traceback (most recent call last):
  File "/nix/store/8b82j10fpjq8r5gbhs0pwjwrw0vyrzm2-qtwebengine-6.8.0/.build/qtwebengine-everywhere-src-6.8.0/build/src/core/Release/arm64/../../../../../src/3rdparty/chromium/third_party/angle/src/libANGLE/renderer/metal/shaders/metal_wrapper.py", line 15, in <module>
    sys.exit(main(sys.argv[1:]))
             ^^^^^^^^^^^^^^^^^^
  File "/nix/store/8b82j10fpjq8r5gbhs0pwjwrw0vyrzm2-qtwebengine-6.8.0/.build/qtwebengine-everywhere-src-6.8.0/build/src/core/Release/arm64/../../../../../src/3rdparty/chromium/third_party/angle/src/libANGLE/renderer/metal/shaders/metal_wrapper.py", line 11, in main
    return subprocess.run(args, stdout=subprocess.PIPE, text=True).returncode
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/nix/store/zyak8iqzh1ww83qa4sqwwz3qax0lrky7-python3-3.12.7/lib/python3.12/subprocess.py", line 548, in run
    with Popen(*popenargs, **kwargs) as process:
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/nix/store/zyak8iqzh1ww83qa4sqwwz3qax0lrky7-python3-3.12.7/lib/python3.12/subprocess.py", line 1026, in __init__
    self._execute_child(args, executable, preexec_fn, close_fds,
  File "/nix/store/zyak8iqzh1ww83qa4sqwwz3qax0lrky7-python3-3.12.7/lib/python3.12/subprocess.py", line 1955, in _execute_child
    raise child_exception_type(errno_num, err_msg, err_filename)
FileNotFoundError: [Errno 2] No such file or directory: '/nix/store/h3skf4182jjbmli12nqggaj0x5kqq63l-apple-sdk-15.0/Toolchains/XcodeDefault.xctoolchain/usr/bin/metal'

It seems that is a Xcode only feature introduce by google/angle@a9b0174

I was trying to workaround that by disabling metal shader compilation:

diff --git a/pkgs/development/libraries/qt-6/modules/qtwebengine.nix b/pkgs/development/libraries/qt-6/modules/qtwebengine.nix
index c97dd03779..c569843fb5 100644
--- a/pkgs/development/libraries/qt-6/modules/qtwebengine.nix
+++ b/pkgs/development/libraries/qt-6/modules/qtwebengine.nix
@@ -143,6 +143,8 @@
       --replace "AppleClang" "Clang"
     substituteInPlace cmake/Functions.cmake \
       --replace "/usr/bin/xcrun" "${xcbuild}/bin/xcrun"
+    substituteInPlace src/3rdparty/chromium/third_party/angle/src/libANGLE/renderer/metal/metal_backend.gni \
+      --replace-fail 'angle_has_build && !is_ios && target_os == host_os' "false"
   '';
 
   cmakeFlags = [

It seems the workaround did the job (failed at [15832/29764] instead of [5019/29770]), but encountered another error:

Edit: The workaround works, and the following error can be fixed using clang 17.

Error log
FAILED: obj/device/bluetooth/bluetooth/bluetooth_discovery_filter.o 
../../../../../../../../v39c0h6xv6hvki2k1bsyp5n090q8y2bp-clang-wrapper-16.0.6/bin/clang++ -MMD -MF obj/device/bluetooth/bluetooth/bluetooth_discovery_filter.o
.d -DDEVICE_BLUETOOTH_IMPLEMENTATION -DTOOLKIT_QT -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -D_LIBCPP_HARDENING_MODE=_LIBCPP_HARDENING_MODE_NONE -DCR_XC
ODE_VERSION=0010 -DNDEBUG -DNVALGRIND -DDYNAMIC_ANNOTATIONS_ENABLED=0 -DSK_ENABLE_SKSL -DSK_UNTIL_CRBUG_1187654_IS_FIXED -DSK_USER_CONFIG_HEADER=\"../../skia/
config/SkUserConfig.h\" -DSK_WIN_FONTMGR_NO_SIMULATIONS -DSK_DISABLE_LEGACY_SKSURFACE_METHODS -DSK_DISABLE_LEGACY_GRAPHITE_IMAGE_FACTORIES -DSK_DISABLE_LEGACY
_GRAPHITE_IMAGE_METHODS -DSK_DISABLE_LEGACY_SKSURFACE_FACTORIES -DSK_DISABLE_LEGACY_SKSURFACE_FLUSH -DSK_DISABLE_LEGACY_SKSURFACE_AS_IMAGE -DSK_DISABLE_LEGACY
_SKSURFACE_DISPLAYLIST -DSK_DISABLE_LEGACY_IMAGE_SUBSET_METHODS -DSK_DISABLE_LEGACY_IMAGE_COLORSPACE_METHODS -DSK_DISABLE_LEGACY_IMAGE_RELEASE_PROCS -DSK_DISA
BLE_LEGACY_GL_BACKEND_SURFACE -DSK_DISABLE_LEGACY_INIT_DECODERS -DSK_DISABLE_LEGACY_GRDIRECTCONTEXT_FLUSH -DSK_DISABLE_LEGACY_GRDIRECTCONTEXT_BOOLS -DSK_DISAB
LE_LEGACY_GL_GRDIRECTCONTEXT_FACTORIES -DSK_DISABLE_LEGACY_PNG_WRITEBUFFER -DSK_SLUG_DISABLE_LEGACY_DESERIALIZE -DSK_DISABLE_LEGACY_VK_GRDIRECTCONTEXT_FACTORI
ES -DSK_DEFAULT_TYPEFACE_IS_EMPTY -DSK_DISABLE_LEGACY_DEFAULT_TYPEFACE -DSK_DISABLE_LEGACY_VULKAN_BACKENDSEMAPHORE -DSK_DISABLE_LEGACY_CREATE_CHARACTERIZATION
 -DSK_DISABLE_LEGACY_FONTMGR_REFDEFAULT -DSK_DISABLE_LEGACY_FONTMGR_FACTORY -DSK_CODEC_DECODES_JPEG -DSK_ENCODE_JPEG -DSK_ENCODE_PNG -DSK_ENCODE_WEBP -DSK_BUI
LD_FOR_MAC -DSK_GANESH -DSK_GPU_WORKAROUNDS_HEADER=\"gpu/config/gpu_driver_bug_workaround_autogen.h\" -DSK_GL -DSK_GRAPHITE -DWEBP_EXTERN=extern -DU_USING_ICU
_NAMESPACE=0 -DU_ENABLE_DYLOAD=0 -DUSE_CHROMIUM_ICU=1 -DU_ENABLE_TRACING=1 -DU_ENABLE_RESOURCE_TRACING=0 -DU_STATIC_IMPLEMENTATION -DICU_UTIL_DATA_IMPL=ICU_UT
IL_DATA_FILE -DGOOGLE_PROTOBUF_NO_RTTI -DGOOGLE_PROTOBUF_NO_STATIC_INITIALIZER -DGOOGLE_PROTOBUF_INTERNAL_DONATE_STEAL_INLINE=0 -DHAVE_PTHREAD -Igen -I../../.
./../../src/3rdparty/chromium -I../../../../../src/3rdparty/chromium/third_party/perfetto/include -Igen/third_party/perfetto/build_config -Igen/third_party/pe
rfetto -I../../../../../src/3rdparty/chromium/net/third_party/quiche/overrides -I../../../../../src/3rdparty/chromium/net/third_party/quiche/src/quiche/common
/platform/default -I../../../../../src/3rdparty/chromium/net/third_party/quiche/src -I../../../../../src/3rdparty/chromium/third_party/skia -Igen/third_party/
skia -I../../../../../src/3rdparty/chromium/third_party/wuffs/src/release/c -I../../../../../src/3rdparty/chromium/third_party/libwebp/src/src -I../../../../.
./src/3rdparty/chromium/base/allocator/partition_allocator/src -Igen/base/allocator/partition_allocator/src -I../../../../../src/3rdparty/chromium/third_party
/abseil-cpp -I../../../../../src/3rdparty/chromium/third_party/boringssl/src/include -I../../../../../src/3rdparty/chromium/third_party/protobuf/src -Igen/pro
toc_out -I../../../../../src/3rdparty/chromium/third_party/ipcz/include -I../../../../../src/3rdparty/chromium/third_party/ced/src -I../../../../../src/3rdpar
ty/chromium/third_party/icu/source/common -I../../../../../src/3rdparty/chromium/third_party/icu/source/i18n -Igen/net/third_party/quiche/src -I../../../../..
/src/3rdparty/chromium/third_party/re2/src -Wall -Wextra -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=2 -Wimplicit-fallthrough -Wthread-safety -Wunguarded-availability
 -Wno-missing-field-initializers -Wno-unused-parameter -Wno-psabi -Wloop-analysis -Wno-unneeded-internal-declaration -Wno-deprecated-declarations -Wenum-compa
re-conditional -Wno-ignored-pragma-optimize -Wno-deprecated-builtins -Wno-bitfield-constant-conversion -Wno-deprecated-this-capture -Wno-invalid-offsetof -Wno
-vla-extension -Wno-thread-safety-reference-return -Wshadow -fno-delete-null-pointer-checks -fno-ident -fno-strict-aliasing -fstack-protector -Wno-unknown-arg
ument -Wno-unknown-attributes -Wno-unknown-warning-option -Wno-ignored-attributes -Wno-predefined-identifier-outside-function -Wno-self-assign -Wno-unknown-pr
agmas -fcolor-diagnostics -fmerge-all-constants --target=arm64-apple-macos -mno-outline -Wno-builtin-macro-redefined -D__DATE__= -D__TIME__= -D__TIMESTAMP__= 
-O2 -fno-math-errno -fno-omit-frame-pointer -g0 -isysroot ../../../../../../../../h3skf4182jjbmli12nqggaj0x5kqq63l-apple-sdk-15.0/Platforms/MacOSX.platform/De
veloper/SDKs/MacOSX15.0.sdk -mmacos-version-min=11.0 -fvisibility=hidden -Wheader-hygiene -Wstring-conversion -Wtautological-overlap-compare -DPROTOBUF_ALLOW_
DEPRECATED=1 -Wno-c++11-narrowing-const-reference -Wno-parentheses-equality -Wno-tautological-compare -Wno-thread-safety-attributes -Wno-undefined-bool-conver
sion -Wno-tautological-undefined-compare -std=c++20 -Wno-trigraphs -fno-exceptions -fno-rtti -fvisibility-inlines-hidden -c ../../../../../src/3rdparty/chromi
um/device/bluetooth/bluetooth_discovery_filter.cc -o obj/device/bluetooth/bluetooth/bluetooth_discovery_filter.o
In file included from ../../../../../src/3rdparty/chromium/device/bluetooth/bluetooth_discovery_filter.cc:5:
In file included from ../../../../../src/3rdparty/chromium/device/bluetooth/bluetooth_discovery_filter.h:15:
In file included from ../../../../../src/3rdparty/chromium/base/containers/flat_set.h:11:
../../../../../src/3rdparty/chromium/base/containers/flat_tree.h:354:22: error: invalid operands to binary expression ('const container_type' (aka 'const std:
:vector<device::BluetoothUUID>') and 'const container_type')
    return lhs.body_ <=> rhs.body_;
           ~~~~~~~~~ ^   ~~~~~~~~~
./../../../../src/3rdparty/chromium/device/bluetooth/bluetooth_discovery_filter.cc:39:18: note: in instantiation of member function 'base::internal::operator<=>' requested here
    return uuids < other.uuids;
                 ^
/nix/store/0mmz5i705116bfykc7i1r9jxqwyghj2a-libcxx-16.0.6-dev/include/c++/v1/__threading_support:680:17: note: candidate function not viable: no known conversion from 'const container_type' (aka 'const std::vector<device::BluetoothUUID>') to '__thread_id' for 1st argument
strong_ordering operator<=>(__thread_id __x, __thread_id __y) noexcept {

Full log https://gist.githubusercontent.com/azuwis/fff9587f6c9e55798fb0baf711a4a432/raw/5bc8795fce95340b5e0d350b22ff136f82a7afa2/qtwebengine.log

Any idea? @emilazy @K900

Fixes #353924

ZHF: #352882

Things done

  • Built on platform(s)
    • x86_64-linux
    • aarch64-linux
    • x86_64-darwin
    • aarch64-darwin
  • For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)
    • sandbox = relaxed
    • sandbox = true
  • Tested, as applicable:
  • Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
  • Tested basic functionality of all binary files (usually in ./result/bin/)
  • 25.05 Release Notes (or backporting 24.11 and 25.05 Release notes)
    • (Package updates) Added a release notes entry if the change is major or breaking
    • (Module updates) Added a release notes entry if the change is significant
    • (Module addition) Added a release notes entry if adding a new NixOS module
  • Fits CONTRIBUTING.md.

Add a 👍 reaction to pull requests you find important.

@emilazy
Copy link
Member

emilazy commented Nov 28, 2024

There are commits from after the Metal shader build was introduced that introduce a proper flag for it, so we shouldn’t need to patch the build unless we maybe need to backport the upstream commits that added that flexibility. (Apparently Google’s toolchain itself can’t use the shader compiler.)

IIRC @niklaskorz (?) got as far as you previously. The Bluetooth thing seemed like it might be a compiler version issue (maybe try on staging with LLVM 19?) or an SDK version issue, but I don’t have any particular insight beyond that.

@niklaskorz
Copy link
Contributor

IIRC @niklaskorz (?) got as far as you previously. The Bluetooth thing seemed like it might be a compiler version issue (maybe try on staging with LLVM 19?) or an SDK version issue, but I don’t have any particular insight beyond that.

My only encounter with the Metal shader compiler was in the context of zed-editor (not chromium), which I also solved by contributing runtime (instead of build time) shader compilation as an optional flag: zed-industries/zed@6dca609

@emilazy
Copy link
Member

emilazy commented Nov 28, 2024

Sorry, must have mixed you up with whoever was trying to get it building :)

I’m not sure about the Bluetooth thing here but hopefully we can figure out how to plumb down the flag to disable the shader compilation from the Qt WebEngine build system down to ANGLE.

@azuwis
Copy link
Contributor Author

azuwis commented Nov 28, 2024

There are commits from after the Metal shader build was introduced that introduce a proper flag for it

Base on the history of metal_backend.gni https://github.com/google/angle/commits/main/src/libANGLE/renderer/metal/metal_backend.gni, I don't see any gnFlags we can use to disable metal shader compilation.

I believe qt6.qtwebengine in nixpkgs already has the latest metal_backend.gni file.

@emilazy
Copy link
Member

emilazy commented Nov 28, 2024

google/angle@a9b0174 added the flag I’m thinking of. Since it’s apparently used by Chrome, it should be quite reliable.

@azuwis
Copy link
Contributor Author

azuwis commented Nov 28, 2024

I don't think metal_internal_shader_compilation_supported is a proper gn arg, since it's not in declare_args, unlike angle_enable_swiftshader and others:

https://github.com/google/angle/blob/2dc072ec71ccba7464600b8a84d6ef98623eb319/gni/angle.gni#L233-L238

@emilazy
Copy link
Member

emilazy commented Nov 28, 2024

Ah, okay, I didn’t realize. So I guess the only way to override things like angle_has_build is patching the source? (I don’t really understand the Chromium build system.)

Anyway, my best guess for the Bluetooth error is that LLVM 16 is too old. So I’d try on staging.

@azuwis
Copy link
Contributor Author

azuwis commented Nov 28, 2024

I've used this diff on master to test:

-qtModule {
+let
+  stdenv' = if stdenv.cc.isClang then llvmPackages_18.stdenv else stdenv;
+in
+(qtModule.override { stdenv = stdenv'; }) {

But get this error:

FAILED: obj/third_party/zlib/bundled_zlib/zutil.o 
../../../../../../../../rwfgxvwy9css9wh1jb0wl5rbyh31v0k3-clang-wrapper-18.1.8/bin/clang -MMD -MF obj/third_party/zlib/bundled_zlib/zutil.o.d -DTOOLKIT_QT -D_LIBCPP_HARDENING_MODE=_LIBCPP_HARDENING_MODE_NONE -DCR_XCODE_VERSION=0010 -DNDEBUG -DNVALGRIND -DDYNAMIC_ANNOTATIONS_ENABLED=0 -DZLIB_IMPLEMENTATION -DADLER32_SIMD_NEON -DINFLATE_CHUNK_SIMD_NEON -DINFLATE_CHUNK_READ_64LE -DDEFLATE_SLIDE_HASH_NEON -DCRC32_ARMV8_CRC32 -DARMV8_OS_MACOS -Igen -I../../../../../src/3rdparty/chromium -I../../../../../src/3rdparty/chromium/third_party/zlib -fno-delete-null-pointer-checks -fno-ident -fno-strict-aliasing -fstack-protector -Wno-unknown-argument -Wno-unknown-attributes -Wno-unknown-warning-option -Wno-ignored-attributes -Wno-predefined-identifier-outside-function -Wno-self-assign -Wno-unknown-pragmas -fcolor-diagnostics -fmerge-all-constants --target=arm64-apple-macos -mno-outline -Wno-builtin-macro-redefined -D__DATE__= -D__TIME__= -D__TIMESTAMP__= -O2 -fno-math-errno -fno-omit-frame-pointer -g0 -isysroot ../../../../../../../../h3skf4182jjbmli12nqggaj0x5kqq63l-apple-sdk-15.0/Platforms/MacOSX.platform/Developer/SDKs/MacOSX15.0.sdk -mmacos-version-min=11.0 -fvisibility=hidden -Wheader-hygiene -Wstring-conversion -Wtautological-overlap-compare -Wall -Wno-unused-variable -Wno-c++11-narrowing -Wno-unused-but-set-variable -Wno-misleading-indentation -Wunguarded-availability -Wno-missing-field-initializers -Wno-unused-parameter -Wno-psabi -Wloop-analysis -Wno-unneeded-internal-declaration -Wno-deprecated-declarations -Wenum-compare-conditional -Wno-ignored-pragma-optimize -Wno-deprecated-builtins -Wno-bitfield-constant-conversion -Wno-deprecated-this-capture -Wno-invalid-offsetof -Wno-vla-extension -Wno-thread-safety-reference-return -O3 -fno-math-errno -Wno-incompatible-pointer-types -Wunused-variable -std=c11 -c ../../../../../src/3rdparty/chromium/third_party/zlib/zutil.c -o obj/third_party/zlib/bundled_zlib/zutil.o
In file included from ../../../../../src/3rdparty/chromium/third_party/zlib/zutil.c:8:
../../../../../src/3rdparty/chromium/third_party/zlib/zutil.h:185:11: warning: 'OS_CODE' macro redefined [-Wmacro-redefined]
  185 | #  define OS_CODE 19
      |           ^
../../../../../src/3rdparty/chromium/third_party/zlib/zutil.h:156:11: note: previous definition is here
  156 | #  define OS_CODE  7
      |           ^
In file included from ../../../../../src/3rdparty/chromium/third_party/zlib/zutil.c:10:
In file included from ../../../../../src/3rdparty/chromium/third_party/zlib/gzguts.h:20:
In file included from ../../../../../../../../h3skf4182jjbmli12nqggaj0x5kqq63l-apple-sdk-15.0/Platforms/MacOSX.platform/Developer/SDKs/MacOSX15.0.sdk/usr/include/stdio.h:61:
../../../../../../../../h3skf4182jjbmli12nqggaj0x5kqq63l-apple-sdk-15.0/Platforms/MacOSX.platform/Developer/SDKs/MacOSX15.0.sdk/usr/include/_stdio.h:314:7: error: expected identifier or '('
  314 | FILE    *fdopen(int, const char *) __DARWIN_ALIAS_STARTING(__MAC_10_6, __IPHONE_2_0, __DARWIN_ALIAS(fdopen));
      |          ^
../../../../../src/3rdparty/chromium/third_party/zlib/zutil.h:162:33: note: expanded from macro 'fdopen'
  162 | #        define fdopen(fd,mode) NULL /* No fdopen() */
      |                                 ^
/nix/store/rwfgxvwy9css9wh1jb0wl5rbyh31v0k3-clang-wrapper-18.1.8/resource-root/include/__stddef_null.h:26:16: note: expanded from macro 'NULL'
   26 | #define NULL ((void*)0)
      |                ^
In file included from ../../../../../src/3rdparty/chromium/third_party/zlib/zutil.c:10:
In file included from ../../../../../src/3rdparty/chromium/third_party/zlib/gzguts.h:20:
In file included from ../../../../../../../../h3skf4182jjbmli12nqggaj0x5kqq63l-apple-sdk-15.0/Platforms/MacOSX.platform/Developer/SDKs/MacOSX15.0.sdk/usr/include/stdio.h:61:
../../../../../../../../h3skf4182jjbmli12nqggaj0x5kqq63l-apple-sdk-15.0/Platforms/MacOSX.platform/Developer/SDKs/MacOSX15.0.sdk/usr/include/_stdio.h:314:7: error: expected ')'
../../../../../src/3rdparty/chromium/third_party/zlib/zutil.h:162:33: note: expanded from macro 'fdopen'
  162 | #        define fdopen(fd,mode) NULL /* No fdopen() */
      |                                 ^
/nix/store/rwfgxvwy9css9wh1jb0wl5rbyh31v0k3-clang-wrapper-18.1.8/resource-root/include/__stddef_null.h:26:16: note: expanded from macro 'NULL'
   26 | #define NULL ((void*)0)
      |                ^
../../../../../../../../h3skf4182jjbmli12nqggaj0x5kqq63l-apple-sdk-15.0/Platforms/MacOSX.platform/Developer/SDKs/MacOSX15.0.sdk/usr/include/_stdio.h:314:7: note: to match this '('
../../../../../src/3rdparty/chromium/third_party/zlib/zutil.h:162:33: note: expanded from macro 'fdopen'
  162 | #        define fdopen(fd,mode) NULL /* No fdopen() */
      |                                 ^
/nix/store/rwfgxvwy9css9wh1jb0wl5rbyh31v0k3-clang-wrapper-18.1.8/resource-root/include/__stddef_null.h:26:15: note: expanded from macro 'NULL'
   26 | #define NULL ((void*)0)
      |               ^

Same error when using llvmPackages_19.stdenv, but llvmPackages_17.stdenv seems to work, building now.

@emilazy Do you recognize this error?

@ofborg ofborg bot added the 6.topic: darwin Running or building packages on Darwin label Nov 28, 2024
@azuwis
Copy link
Contributor Author

azuwis commented Nov 28, 2024

nixpkgs-review result

Generated using nixpkgs-review.

Command: nixpkgs-review pr 359755


aarch64-darwin

✅ 2 packages built:
  • qt6.qtwebengine
  • qt6.qtwebengine.dev (qt6.qtwebengine.dev.dev)

@azuwis azuwis marked this pull request as ready for review November 28, 2024 23:14
@ofborg ofborg bot added 10.rebuild-darwin: 11-100 10.rebuild-linux: 0 This PR does not cause any packages to rebuild on Linux labels Nov 29, 2024
@@ -143,6 +152,9 @@ qtModule {
--replace "AppleClang" "Clang"
substituteInPlace cmake/Functions.cmake \
--replace "/usr/bin/xcrun" "${xcbuild}/bin/xcrun"
# Disable metal shader compilation, Xcode only
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nitpick. Can we make this substitution and the AppleClang one running on all platforms? Than we won't break macos if the package is updated.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

@github-actions github-actions bot added 10.rebuild-linux: 101-500 and removed 10.rebuild-linux: 0 This PR does not cause any packages to rebuild on Linux labels Nov 30, 2024
@somasis
Copy link
Contributor

somasis commented Dec 30, 2024

anything preventing this from being merged?

@emilazy
Copy link
Member

emilazy commented Dec 30, 2024

Well, merge conflicts for one :)

Sorry for forgetting about this.

@azuwis That‘s a known issue; you can look at e.g. pkgs/development/web/nodejs/v20.nix for the upstream Zlib patch we‘ve applied to handle it in previous cases. But I suspect Chromium has probably applied it upstream by now and you can get it from there instead to avoid messing with the paths? Patching would be preferable to using the old LLVM in any case.

There‘s also some other LLVM 19‐related V8 backports in there we might need. It’s possible there’s stuff elsewhere in Chromium too but hopefully upstream patches shouldn’t be too hard to find.

@wegank wegank added the 2.status: merge conflict This PR has merge conflicts with the target branch label Dec 31, 2024
So we won't break darwin build if the package is updated
@azuwis azuwis marked this pull request as draft December 31, 2024 12:40
@emilazy
Copy link
Member

emilazy commented Jan 1, 2025

Looks like the vendored libpng needs patching (or preferably we would just convince the build system to devendor it as we do on Linux).

@azuwis azuwis force-pushed the push-qlvztsusywtq branch from 3c92e65 to 3bed3d5 Compare January 5, 2025 16:18
@ofborg ofborg bot removed the 2.status: merge conflict This PR has merge conflicts with the target branch label Jan 5, 2025
@azuwis
Copy link
Contributor Author

azuwis commented Jan 5, 2025

With latest push, buildPhase and installPhase run fine, but fixupPhase hangs.

-- Installing: /nix/store/vw75y1awv5mlfxg6splckdkd3j8a9va1-qtwebengine-6.8.1/lib/cmake/Qt6
-- Installing: /nix/store/vw75y1awv5mlfxg6splckdkd3j8a9va1-qtwebengine-6.8.1/lib/cmake/Qt6/FindNodejs.cmake
-- Installing: /nix/store/vw75y1awv5mlfxg6splckdkd3j8a9va1-qtwebengine-6.8.1/lib/cmake/Qt6/FindNinja.cmake
-- Installing: /nix/store/vw75y1awv5mlfxg6splckdkd3j8a9va1-qtwebengine-6.8.1/lib/cmake/Qt6/FindGn.cmake
-- Installing: /nix/store/vw75y1awv5mlfxg6splckdkd3j8a9va1-qtwebengine-6.8.1/lib/cmake/Qt6/FindSnappy.cmake
-- Installing: /nix/store/vw75y1awv5mlfxg6splckdkd3j8a9va1-qtwebengine-6.8.1/lib/cmake/Qt6/FindGPerf.cmake
-- Installing: /nix/store/vw75y1awv5mlfxg6splckdkd3j8a9va1-qtwebengine-6.8.1/lib/cmake/Qt6/FindPkgConfigHost.cmake
-- Installing: /nix/store/vw75y1awv5mlfxg6splckdkd3j8a9va1-qtwebengine-6.8.1/lib/cmake/Qt6WebEngineCoreTools/Qt6WebEngineCoreToolsAdditionalTargetInfo.cmake
installPhase completed in 36 seconds
Running phase: glibPreFixupPhase
Running phase: fixupPhase
checking for references to /nix/var/nix-build-qtwebengine-6.8.1.drv-0/ in /nix/store/vw75y1awv5mlfxg6splckdkd3j8a9va1-qtwebengine-6.8.1...
/nix/store/h9lc1dpi14z7is86ffhl3ld569138595-audit-tmpdir.sh: line 23: patchelf: command not found
/nix/store/h9lc1dpi14z7is86ffhl3ld569138595-audit-tmpdir.sh: line 23: patchelf: command not found
/nix/store/h9lc1dpi14z7is86ffhl3ld569138595-audit-tmpdir.sh: line 23: patchelf: command not found
/nix/store/h9lc1dpi14z7is86ffhl3ld569138595-audit-tmpdir.sh: line 23: patchelf: command not found
/nix/store/h9lc1dpi14z7is86ffhl3ld569138595-audit-tmpdir.sh: line 23: patchelf: command not found
/nix/store/h9lc1dpi14z7is86ffhl3ld569138595-audit-tmpdir.sh: line 23: patchelf: command not found
patching script interpreter paths in /nix/store/vw75y1awv5mlfxg6splckdkd3j8a9va1-qtwebengine-6.8.1
/nix/store/vw75y1awv5mlfxg6splckdkd3j8a9va1-qtwebengine-6.8.1/.build/qtwebengine-everywhere-src-6.8.1/tools/scripts/make_archive.sh: interpreter directive changed from "#!/bin/bash" to "/nix/store/5wjj1f3pa1ndi3kvskm1fcs9kdy2xajf-bash-5.2p37/bin/bash"
/nix/store/vw75y1awv5mlfxg6splckdkd3j8a9va1-qtwebengine-6.8.1/.build/qtwebengine-everywhere-src-6.8.1/src/3rdparty/gn/infra/recipes.py: interpreter directive changed from "#!/bin/sh" to "/nix/store/5wjj1f3pa1ndi3kvskm1fcs9kdy2xajf-bash-5.2p37/bin/sh"
/nix/store/vw75y1awv5mlfxg6splckdkd3j8a9va1-qtwebengine-6.8.1/.build/qtwebengine-everywhere-src-6.8.1/src/3rdparty/gn/tools/update_reference.sh: interpreter directive changed from "#!/bin/sh" to "/nix/store/5wjj1f3pa1ndi3kvskm1fcs9kdy2xajf-bash-5.2p37/bin/sh"
/nix/store/vw75y1awv5mlfxg6splckdkd3j8a9va1-qtwebengine-6.8.1/.build/qtwebengine-everywhere-src-6.8.1/src/3rdparty/gn/tools/run_formatter.sh: interpreter directive changed from "#!/bin/sh" to "/nix/store/5wjj1f3pa1ndi3kvskm1fcs9kdy2xajf-bash-5.2p37/bin/sh"
stripping (with command strip and flags -S) in  /nix/store/vw75y1awv5mlfxg6splckdkd3j8a9va1-qtwebengine-6.8.1/lib /nix/store/vw75y1awv5mlfxg6splckdkd3j8a9va1-qtwebengine-6.8.1/libexec
signing /nix/store/vw75y1awv5mlfxg6splckdkd3j8a9va1-qtwebengine-6.8.1

@emilazy
Copy link
Member

emilazy commented Jan 5, 2025

autoSignDarwinBinariesHook should no longer be necessary, try dropping it.

@azuwis
Copy link
Contributor Author

azuwis commented Jan 5, 2025

It seems sigtool is processing every files in /nix/store/vw75y1awv5mlfxg6splckdkd3j8a9va1-qtwebengine-6.8.1/.build/ dir.

└─ nix -L build -f . qt6.qtwebengine                                                                                                                          
   └─ /nix/store/5wjj1f3pa1ndi3kvskm1fcs9kdy2xajf-bash-5.2p37/bin/bash -e /nix/store/v6x3cs394jgqfbi0a42pam708flxaphh-default-builder.sh
      ├─ find /nix/store/vw75y1awv5mlfxg6splckdkd3j8a9va1-qtwebengine-6.8.1 -type f -print0
      └─ /nix/store/6j9b3vzj6wppcyrbw96qxrk1qrx36dpz-sigtool-0.1.3/bin/sigtool --file /nix/store/vw75y1awv5mlfxg6splckdkd3j8a9va1-qtwebengine-6.8.1/.build/qtwebengine-everywhere-src-6.8.1/src/3rdparty/chromium/third_party/blink/common/shared_s...

@azuwis azuwis force-pushed the push-qlvztsusywtq branch 2 times, most recently from f17e5cb to f0f5204 Compare January 5, 2025 17:29
- Disable metal shader compilation
- Apply upstream zutil.h patch that fixes clang 18 and later
- Apply upstream libpng patch that fixes clang 18 and later
- Drop `autoSignDarwinBinariesHook` on aarch64-darwin, no longer necessary
@azuwis
Copy link
Contributor Author

azuwis commented Jan 5, 2025

I was wrong about the fixupPhase hang, the phase finally finished, just slow.

@azuwis azuwis force-pushed the push-qlvztsusywtq branch from f0f5204 to 85c1c3c Compare January 5, 2025 17:36
@azuwis
Copy link
Contributor Author

azuwis commented Jan 5, 2025


aarch64-darwin

✅ 2 packages built:
  • qt6.qtwebengine
  • qt6.qtwebengine.dev (qt6.qtwebengine.dev.dev)

@azuwis azuwis marked this pull request as ready for review January 5, 2025 23:42
@emilazy
Copy link
Member

emilazy commented Jan 5, 2025

Wonderful! I love how straightforward the diff is. Really nice that we can build big things like this again now.

I’ll try to remember to test the build on x86_64-darwin and then merge tomorrow; feel free to ping me if I don’t.

@azuwis
Copy link
Contributor Author

azuwis commented Jan 8, 2025

@emilazy Gentle ping.

Copy link
Member

@emilazy emilazy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It builds 🎉

@emilazy emilazy added the backport release-24.11 Backport PR automatically label Jan 8, 2025
@emilazy emilazy merged commit 3b1c809 into NixOS:master Jan 8, 2025
27 of 31 checks passed
@nixpkgs-ci
Copy link
Contributor

nixpkgs-ci bot commented Jan 8, 2025

Backport failed for release-24.11, because it was unable to cherry-pick the commit(s).

Please cherry-pick the changes locally and resolve any conflicts.

git fetch origin release-24.11
git worktree add -d .worktree/backport-359755-to-release-24.11 origin/release-24.11
cd .worktree/backport-359755-to-release-24.11
git switch --create backport-359755-to-release-24.11
git cherry-pick -x af397f4a337dcdea43583ab12ef31c0fcdbdcb31 85c1c3ca3f214ee465a63bd6a5a41a7a9ebe191b

@emilazy
Copy link
Member

emilazy commented Jan 8, 2025

Do you mind handling the backport? It’d be great if we got this fixed on 24.11.

@azuwis
Copy link
Contributor Author

azuwis commented Jan 11, 2025

Do you mind handling the backport? It’d be great if we got this fixed on 24.11.

See #372849

@azuwis azuwis deleted the push-qlvztsusywtq branch January 11, 2025 11:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Build failure: qtwebengine-6.8.0 fails to build.
6 participants