[Windows] Long paths issue with Ninja >=1.12

[axelnxp]

Hi,

I'm working with a cross-compile (arm gcc) CMake project which can generate long paths.
When building, I get errors like the following one:

Building C object xxx
Failed: xxx
arm-none-eabi-gcc.exe ...  fatal error: opening dependency file ...  No such file or directory

I enabled the long path support in the registry like documented by Microsoft.
I also updated my Ninja version to the latest one, 1.12 which seems to bring support of long paths. I also tried the builds from master from this link, without success.

I came accross several GitHub issues like #1900 which was closed mentioning there was still issues with this.
I tried to build from MinGW (Git bash env), cmd and PowerShell, without success.

I'm running on Windows 10 19045.

Thanks !

[jhasse]

What is your problem?

[axelnxp]

Hi @jhasse, sorry, the initial was not clear at all... I edited it with the actual error.
After checking more, I'm wondering if this is a ninja or arm-gcc issue.

[axelnxp]

I just did a test: building with ninja vs directly with arm-gcc.
So, when calling ninja, I get the error I mentioned previously, but when taking the exact build command and executing it by calling directly arm-gcc, it compiles.
So it looks like ninja is at fault here.

[digit-google]

It would be useful to give us something we can use to reproduce the problem, or more information about the paths themselves (i.e. are they properly printed in the error message vs what is in the Ninja build plan).

For the record, I have uploaded a PR to get rid of remaining Win32 long path issues in Ninja at PR #2410. Can you try the binary from the "Ninja binaries archive" link on https://github.com/ninja-build/ninja/actions/runs/8950181908 on your machine to see if this fixes your issue.

Otherwise, there is probably very little we can do without more information from you.

[axelnxp]

Hi @digit-google

Ok so I did something to reproduce the problem with a simpler and obfuscated env (I cannot share my actual project)
Some context:
The CMake source folder is the following:

/c/aaa/aaaaaa_aaaaaa/aaaaaaaaaa/aaaaaaaa/aaaaaa/aaaaaaaa/aaaaaa_aaaaaaaaaaa/build_cmake

This CMake project builds a library with a single C file located here (sorry for the name of the file, just wanted to make it stupidly long to reproduce the problem):

C:/aaa/aaaaaa_aaaaaaa/bbbbbbbbbb/bbb_bbb/bbb/bbbbb/bbbbbbbbb/bbb/bbbbbbbbb_bbbbb_bbbbbbbb_bbbbbbbbbbbbbbb_bbbbbbbbbbbbb_bbb_bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb.c

During the CMake configuration, a warning is displayed saying the path is still too long after hashing it:

CMake Warning in CMakeLists.txt:
  The object file directory

    C:/aaa/aaaaaa_aaaaaa/aaaaaaaaaa/aaaaaaaa/aaaaaa/aaaaaaaa/aaaaaa_aaaaaaaaaaa/build_cmake/build/CMakeFiles/mylib.dir/./

  has 117 characters.  The maximum full path to an object file is 250
  characters (see CMAKE_OBJECT_PATH_MAX).  Object file

    8e1e25dad068f8d0a92cb8c1e5e287b9/bbbbbbbbb_bbbbb_bbbbbbbb_bbbbbbbbbbbbbbb_bbbbbbbbbbbbb_bbb_bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb.c.obj

  cannot be safely placed under this directory.  The build may not work
  correctly.

Now, here is the output of Ninja (I'm using the ninja binary from #2410 ):

$ ninja -C build
ninja: Entering directory `build'
[1/2] Building C object CMakeFiles/mylib.dir/8e1e25dad068f8d0a92cb8c1e5e287b9/bbbbbbbbb_bbbbb_bbbbbbbb_bbbbbbbbbbbbbbb_bbbbbbbbbbbbb_bbb_bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb.c.obj
FAILED: CMakeFiles/mylib.dir/8e1e25dad068f8d0a92cb8c1e5e287b9/bbbbbbbbb_bbbbb_bbbbbbbb_bbbbbbbbbbbbbbb_bbbbbbbbbbbbb_bbb_bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb.c.obj
C:\MCUXpressoIDE_11.9.0_2144\ide\tools\bin\arm-none-eabi-gcc.exe   -Wall -Wno-expansion-to-defined -Wno-endif-labels -fmessage-length=0 -fdata-sections -ffunction-sections -std=gnu99 -MD -MT CMakeFiles/mylib.dir/8e1e25dad068f8d0a92cb8c1e5e287b9/bbbbbbbbb_bbbbb_bbbbbbbb_bbbbbbbbbbbbbbb_bbbbbbbbbbbbb_bbb_bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb.c.obj -MF CMakeFiles\mylib.dir\8e1e25dad068f8d0a92cb8c1e5e287b9\bbbbbbbbb_bbbbb_bbbbbbbb_bbbbbbbbbbbbbbb_bbbbbbbbbbbbb_bbb_bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb.c.obj.d -o CMakeFiles/mylib.dir/8e1e25dad068f8d0a92cb8c1e5e287b9/bbbbbbbbb_bbbbb_bbbbbbbb_bbbbbbbbbbbbbbb_bbbbbbbbbbbbb_bbb_bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb.c.obj -c C:/aaa/aaaaaa_aaaaaaa/bbbbbbbbbb/bbb_bbb/bbb/bbbbb/bbbbbbbbb/bbb/bbbbbbbbb_bbbbb_bbbbbbbb_bbbbbbbbbbbbbbb_bbbbbbbbbbbbb_bbb_bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb.c
C:/aaa/aaaaaa_aaaaaaa/bbbbbbbbbb/bbb_bbb/bbb/bbbbb/bbbbbbbbb/bbb/bbbbbbbbb_bbbbb_bbbbbbbb_bbbbbbbbbbbbbbb_bbbbbbbbbbbbb_bbb_bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb.c:1: fatal error: opening dependency file CMakeFiles\mylib.dir\8e1e25dad068f8d0a92cb8c1e5e287b9\bbbbbbbbb_bbbbb_bbbbbbbb_bbbbbbbbbbbbbbb_bbbbbbbbbbbbb_bbb_bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb.c.obj.d: No such file or directory
compilation terminated.
ninja: build stopped: subcommand failed.

I ran the command directly with arm-gcc, and now I get the same error, which contradicts what I observed yesterday, I'm a bit confused...

[jhasse]

Thanks for the investigation!

The original error message also hints at this being a bug in C:\MCUXpressoIDE_11.9.0_2144\ide\tools\bin\arm-none-eabi-gcc.exe. I'll close this for now, please reopen / comment if you find any evidence that this is not the case.

[JimWang151]

Hi,

I'm working with a cross-compile (arm gcc) CMake project which can generate long paths. When building, I get errors like the following one:

Building C object xxx
Failed: xxx
arm-none-eabi-gcc.exe ...  fatal error: opening dependency file ...  No such file or directory

I enabled the long path support in the registry like documented by Microsoft. I also updated my Ninja version to the latest one, 1.12 which seems to bring support of long paths. I also tried the builds from master from this link, without success.

I came accross several GitHub issues like #1900 which was closed mentioning there was still issues with this. I tried to build from MinGW (Git bash env), cmd and PowerShell, without success.

I'm running on Windows 10 19045.

Thanks !

hello bro.I met the same problem as you.

How you solved this issue finally?

Thanks.

[axelnxp]

Hi,
I'm working with a cross-compile (arm gcc) CMake project which can generate long paths. When building, I get errors like the following one:

Building C object xxx
Failed: xxx
arm-none-eabi-gcc.exe ...  fatal error: opening dependency file ...  No such file or directory

I enabled the long path support in the registry like documented by Microsoft. I also updated my Ninja version to the latest one, 1.12 which seems to bring support of long paths. I also tried the builds from master from this link, without success.
I came accross several GitHub issues like #1900 which was closed mentioning there was still issues with this. I tried to build from MinGW (Git bash env), cmd and PowerShell, without success.
I'm running on Windows 10 19045.
Thanks !

hello bro.I met the same problem as you.

How you solved this issue finally?

Thanks.

I haven't found a solution unfortunately, seems like the issue is not due to ninja but to armgcc.
Maybe armgcc doesn't support the long paths for windows.

[junbujianwpl]

When building electron for the windows with the ninja of 1.12.0, I met this

GetFullPathNameA(gen/third_party/blink/renderer/bindings/modules/v8/v8_union_cssimagevalue_htmlcanvaselement_htmlimageelement_htmlvideoelement_imagebitmap_offscreencanvas_svgimageelement_videoframe.h): 文件名或扩展名太长。

So just building electron on the windows will be a good way to test if ninja was ok for windows long path support.

[digit-google]

Oh, that seems to be a different issue, the only place where this error may come from is src/includes_normalize-win32.cc which calls the function with a fixe-size buffer of size _MAX_PATH. This should be fixable, let me try something ...

[digit-google]

Interestingly, my testing shows that even when using dynamically allocated buffers, GetFullPathNameA() fails for very long paths, while GetFullPathNameW() does not (for the same input, converted to std::wstring). E.g. the unit-test fails with:

GetFullPathNameA(aaaa/aaaaaaaaa/aaaaaaaaa/aaaaaaaaa/aaaaaaaaa/aaaaaaaaa/aaaaaaaaa/aaaaaaaaa/aaaaaaaaa/aaaaaaaaa/aaaaaaaaa/aaaa
aaaaa/aaaaaaaaa/aaaaaaaaa/aaaaaaaaa/aaaaaaaaa/aaaaaaaaa/aaaaaaaaa/aaaaaaaaa/aaaaaaaaa/aaaaaaaaa/aaaaaaaaa/aaaaaaaaa/aaaaaaaaa/
aaaaaaaaa/aaaaaaaaa/aaaaa): The filename or extension is too long.

I'll fix it, but this is clearly a case of a ANSI function not supporting long paths properly!

[digit-google]

I uploaded #2552 with a candidate fix.