dotnet restore (from nuget) doesn't complete on M1 mac

[tpitman]

I originally posted this in the dotnet/sdk github. Then they suggested I post in the NuGet/Home github. They suggested I post here.

dotnet/sdk issue: dotnet/sdk#29610 (comment)
NuGet/Home issue: NuGet/Home#12329

Describe the bug
When I execute the command docker-compose on an asp.net core solution, the dotnet restore step in the build process runs without completing until I ctrl-c the process to stop it.

Expected Behavior
dotnet restore should complete normally

Steps To Reproduce
Create an asp.net core project based on net7 release code.

Add docker support that includes a docker compose file.

Execute docker-compose at the command line.

Notice that the dotnet restore operation runs without completing.

Exceptions (if any)
none

.NET Version
7.0.1

Anything else?
VS4Mac on M1 MacBook pro

This works fine on my Intel MacBook pro for 2016.

[mthalman]

Without knowing what your Dockerfile looks like, I'm speculating that this is related to a regression in emulation support with NuGet described by NuGet/Home#12227. While that issue describes a specific error versus the hang that you're seeing, I've seen all sorts of unpredictable behavior (both errors and hangs) due to this regression. From a Docker perspective, background on this issue can be found at #4225 and a specific workaround is provided at #4225 (comment).

[kartheekp-ms]

@mthalman - How do we capture dump of a running process (that is executing dotnet restore) in docker container? I don't know how to capture dumps using dotnet-dump command on docker containers, hence asking.

[mthalman]

@kartheekp-ms - You should be able to use a dotnet/monitor container running as a sidecar to get the necessary information.

There is documentation for this on this blog post. @jander-msft - Is that the best doc currently available for this?

[jander-msft]

We have more recent samples for (usage in Kubernetes](https://github.com/dotnet/dotnet-monitor/blob/main/documentation/kubernetes.md) here. That can be adapted for usage in docker compose. Once it is set up, invoke the /dump route in order to capture a dump of the target process; the dump itself will be streamed back to the caller via the HTTP response.

[tpitman]

I found the proposed work around and it isn't working for my situation.

I have a aspnet web server project and several api projects that it relies on.

Here is the web server docker file (with the build stuff removed for brevity):

FROM mcr.microsoft.com/dotnet/aspnet:7.0 AS base
WORKDIR /app
EXPOSE 80
EXPOSE 443
`...` `FROM build AS publish` `RUN dotnet publish "WebServer.csproj" -c Release -o /app/publish -r linux-x64 /p:UseAppHost=false`
FROM base AS final
WORKDIR /app
COPY --from=publish /app/publish .
ENTRYPOINT ["dotnet", "WebServer.dll"]

Here is one of the api projects:

FROM mcr.microsoft.com/dotnet/aspnet:7.0 AS base
WORKDIR /app

FROM build AS publish
RUN dotnet publish "ClientPaymentApi.csproj" -c Release -o /app/publish -r linux-x64 --self-contained false /p:UseAppHost=false

FROM base AS final
WORKDIR /app
COPY --from=publish /app/publish .
ENTRYPOINT ["dotnet", "ClientPaymentApi.dll"]

When I build this the apis all seem to build fine, but the web server project gives this error:

error NETSDK1067: Self-contained applications are required to use the application host. Either set SelfContained to false or set UseAppHost to true. [/src/WebServer/WebServer.csproj]
------
failed to solve: process "/bin/sh -c dotnet publish \"WebServer.csproj\" -c Release -o /app/publish -r linux-x64 /p:UseAppHost=false" did not complete successfully: exit code: 1

I am not sure where to go from here. I see that the solution for the user in that other issue was to remove the -r linux-x64 and call out a different source image for the final output. I don't think the slim one will work because this is an aspnet application.

I tried to find images that might work for that, but could not determine the exact url.

Any thoughts?

[mthalman]

You should set --self-contained false now that you've added -r linux-x64 for the publish of WebServer.csproj. The -r option sets the RID which, by default, assumes a self-contained deployment. But since you don't want a self-contained deployment, you need to explicitly disable that with --self-contained false.

[tpitman]

When I set that option on the web server project I get this error:

error NETSDK1102: Optimizing assemblies for size is not supported for the selected publish configuration. Please ensure that you are publishing a self-contained app. [/src/Web/Web.csproj]

I also have some more information:

Since the api projects seemed to build images without errors (only the web server had issues), I decided to upload the api image to my azure registry and run it.

When I did I got an error when it tried to start that was simple that there was an exec format error.

I suspect the image ended up with an ARM container / image, so putting the -r tag may not have worked??

[tpitman]

Sorry to confuse you. I am pretty confused as well.

I think you had it right the first time. I do not need to run this on my M1 mac. I need to run it in linux/amd64 containers on Azure.

So it sounds like I just need to add that trim option and add the platform tag back into the docker compose file.

[tpitman]

I added the trim false and get a new error:

error MSB4044: The "ComputeBlazorPublishAssets" task was not given a value for the required parameter "DotNetJsVersion". [/src/Web/Web.csproj]

Please keep in mind that this project will build with docker-compose build on an Intel based Mac without any of these changes, so this new error has something to do with all these changes.

I add that just in case you were wondering if this project built on Intel computers.

[mthalman]

I'm not familiar with that error. So it seems like that project relies on PublishTrimmed being true. And for the NETSDK1102 error, it looks like you're hitting dotnet/sdk#20908. I don't know anything about the reason for that issue. So at this point I'm stumped on a workaround that you can use.

[tpitman]

Any suggestions on where to go from here?

[mthalman]

I would follow up on dotnet/sdk#20908 and ask about a workaround.

[tpitman]

Created a post here:

dotnet/sdk#20908 (comment)

[tobsai]

Apologies, I realize I'm a little late to this thread, but the behavior I'm seeing isn't related to the flags shown above. I've posted a repository here with a vanilla Dotnet 7.01 project running on Mac M1. The issue seems to be restricted to docker-compose which isn't able to pick up the platform flag or is dealing with it differently than docker build. Please let me know if you need additional detail/troubleshooting steps. The repo is just a step by step reproduction from this tutorial with a docker-compose file added on.

[tobsai]

Agreed. I think the author's original issue is different from what I'm seeing. I'll log a separate bug.

[tpitman]

So the great question I am left with is who can fix this? Should it be possible to build a linux/amd64 image on an Arm computer like M1? I am pretty sure you can build an arm image on an intel computer. Seems like the reverse should be possible.

[joergjo]

Just to be clear, I meant to say the issue isn't specific to Docker Compose. Trying to do build an ARM64 image on a AMD64 host (Windows 11) results in the aforementioned NuGet error, so I guess it's really a NuGet issue.

[tpitman]

So it doesn't hang, but also doesn't work. Bummer. Hopefully someone can find a solution for each situation...

[tpitman]

I did post this same issue in the Nuget/Home github, but they said they don't have much docker experience and pointed me back to here. They are going to look into it once they have someone with an M1 mac who can look at it. The people who responded said they didn't have access to M1 macs.

[tpitman]

Are you wanting the output from the dotnet command or something that docket would be logging at the system level? If the dotnet output I can tell you that nothing its output. I was able to terminal into the docker container and even with dotnet restore —verbosity detailed it just locks up and outputs nothing.

…

On Fri, Jan 20, 2023 at 6:31 PM Kartheek Penagamuri < ***@***.***> wrote: @mthalman <https://github.com/mthalman> - How do we capture dump of a running process (that is executing dotnet restore) in docker container? I don't know how to capture dumps using dotnet-dump command on docker containers, hence asking. — Reply to this email directly, view it on GitHub <#4285 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAWFTQ5ELCYMWJXSHBQZI7LWTM4AXANCNFSM6AAAAAATMK5YPY> . You are receiving this because you authored the thread.Message ID: ***@***.***>

[erdembayar]

@mthalman @tobsai @tpitman

Hi, I’m from NuGet.Client team, I got actual stack traces here.
It looks like docker image regression or qemu bug problem, not NuGet problem.
I see many of memory corruption/violation messages in stacktraces.
Please let me know if you need any more details for it.

[erdembayar]

I filed bug in dotnet/runtime about this dotnet/runtime#81232

[richlander]

I posted this statement about .NET docker images and x64 emulation on Apple M1s. That may be related.

#3848

[temga]

If anybody still is looking for solution, I have fix that using arm64 image for build and publish stage, and amd64 image as base for runtime. Dockerfile:

FROM mcr.microsoft.com/dotnet/aspnet:7.0-jammy-amd64 AS base
WORKDIR /app
ENV ASPNETCORE_URLS=http://+:5000
EXPOSE 5000

FROM mcr.microsoft.com/dotnet/sdk:7.0 AS build
WORKDIR /src
COPY ["mc_net_main/mc_net_main.csproj", "mc_net_main/"]
RUN dotnet restore "mc_net_main/mc_net_main.csproj"
COPY . .
WORKDIR "/src/mc_net_main"
RUN dotnet build "mc_net_main.csproj" -c Release -o /app/build

FROM build AS publish
RUN dotnet publish "mc_net_main.csproj" -c Release -o /app/publish /p:UseAppHost=false

FROM base AS final
WORKDIR /app
COPY --from=publish /app/publish .
ENTRYPOINT ["dotnet", "mc_net_main.dll"]

and run it on Mac M1 with following command:
docker build -t mc_main:v0.0.1 -f mc_net_main/Dockerfile --no-cache .

[ci-vamp]

Nothing listed in this thread works for me. I’m using Docker desktop (Rosetta feature enabled) on an M1 max.

No matter what I do dotnet restore always fails. However, oddly, if I just do a regular publish (without the --no-restore flag) then it restores, builds and executes properly.

Is the restore step of publish somehow different from the restore command?

[Hlaejanor]

Thank you! :-)

[richlander]

We posted a solution at https://devblogs.microsoft.com/dotnet/improving-multiplatform-container-support/

The trick is running the SDK natively with some variant of the following:

FROM --platform=$BUILDPLATFORM mcr.microsoft.com/dotnet/sdk:8.0-preview-alpine AS build

[vmandic]

No effect for my case, I always have to wait like 2 minutes to do the dotnet restore ie. to see it start.

My image is very simple:

FROM --platform=$BUILDPLATFORM mcr.microsoft.com/dotnet/sdk:7.0.400-alpine3.17 AS build
ARG TARGETARCH
ENV TARGETARCH $TARGETARCH # just to verfiy arm64 on my Mac M1 Air
RUN apk add --update nodejs yarn docker-cli docker-compose && rm -rf /var/cache/apk/

# for debug only so I can hook up with a shell and type dotnet restore inside
ENTRYPOINT ["tail", "-f", "/dev/null"]