Simplifying Nix #5154

sellout · 2024-06-27T23:12:25Z

Overview

This makes a number of changes – they make much more sense if you review commit-by-commit. Here are the highlights:

with Direnv, use flake should hopefully work for all users, now – no need to specify cabal-only-tools to get a working environment
a lot of extra packages have been trimmed from the outputs
the dual haskell.nix / non-haskell.nix environments have gone away, making it much easier to keep things in sync
CI should now cache our packages correctly
there is now ~100 fewer lines of Nix code, which is 29% of the Nix in this repo (and I think there is more that can be removed)

This is some preliminary work to support #5142.

Interesting/controversial decisions

It’s Nix – it’s all controversial.

Loose ends

nix flake show still wants to build GHC 8.10.7 for some reason, but nothing else seems to trigger that issue.

This `only-tools-nixpkgs` devShell generally paralleled the `cabal-only-tools` devShell, but avoiding haskell.nix. While I’m not a huge fan of haskell.nix, this just created duplication and gave us a shell with a somewhat different environment than the one used by `nix build`, etc. It also didn’t work for everyone. In removing that shell, it also sets the default devShell to be `cabal-only-tools`, which some people were already using to work around issues with the previous default.

There are various benefits to using a Nixpkgs release - more likely that things are cached - easier to update the input without breaking everything This also renames a lot of things in the flake: - `nixpkgs-unstable` to `nixpkgs-release` – partially because it’s not unstable any more, but also because both it and the nixpkgs from haskell.nix unstable, so it didn’t really clarify anything - `nixpkgs` to `nixpkgs-haskellNix` – to make it clear where it comes from - `unstable` to `release-pkgs` – the convention is to use `pkgs` for derivation attrsets, and the source switched from unstable to release - `nixpkgs-packages` to `tool-pkgs` – this holds our build tools, so that seemed clearer than “nixpkgs”

Overlays are for derivations, and this isn’t one. Putting it in an overlay also just gives us more levels of indirection to dig through to figure out where things are coming from.

`cabal-local` no longer triggers rebuilds of GHC, so now we can use the devShell that provides the same environment as our build.

This means those environments will also be cached in CI.

sellout · 2024-06-28T04:21:27Z

@ceedubs @mitchellwrosen – just to annoy you two, this PR removes cabal-only-tools, but on the plus side, bare use flake should work for you again.

Also, if anyone wants to try this out before it’s merged that’d be awesome.

neduard · 2024-06-28T12:30:23Z

Hey @sellout this is awesome! I was wondering if the flake could be simplified. I'm using NixOS so let me quickly give it a spin (will edit comment once I've done so)

ceedubs

Nice! I have wondered whether this flake could be simplified.

But I was under the impression that a lot of the complexity was due to supporting specific GHC/ormolu/etc versions that didn't necessarily line up with what was in nixpkgs/haskell.nix. Do you know? As long as @aryairani, @mitchellwrosen, and @ChrisPenner are good with the versions being used, then I think we are good.

sellout · 2024-06-28T14:02:26Z

But I was under the impression that a lot of the complexity was due to supporting specific GHC/ormolu/etc versions that didn't necessarily line up with what was in nixpkgs/haskell.nix. Do you know? As long as @aryairani, @mitchellwrosen, and @ChrisPenner are good with the versions being used, then I think we are good.

This does maintain the same versions of everything as before. One of the reasons for the multiple devShells previously was issues with the IOG cache that resulted in GHC having to be rebuilt by devs. That is fixed for the use cases we generally support. I think that keeping the versions in sync between the two devShells was then a source of complication. The removal of the second devShell is reflected in c163375 and 07c7ca4, where you can see is where we got most of the code removal from. It does look like GHC got unpinned, but the version of GHC used by the haskell.nix devShell comes from stack.yaml – it only needed to be included in the flake to ensure that the non-haskell.nix version also matched.

There are some real changes here, though. E.g., haskell-language-server no longer disables its various plugins. This way

we get an upstream cached version,
it’s more likely to match the configuration that non-Nix users have, and
it’s less Nix code.

I think that Nix can have good libraries with solid APIs, etc. but it’s not there yet and as a result things start looking complicated quite quickly. I.e., you really gotta justify every line of it.

And one benefit of a clear & documented (possibly literate?) flake is that it can also serve as a guide for non-Nix users to follow when setting up their own environment. So things like haskell.nix get hidden away, while version numbers, configuration options, supported platforms, etc. are presented up front.

neduard · 2024-06-28T16:15:23Z

Hmm I tried nix -L build '.#component-unison-cli:lib:unison-cli' locally and it seems to have built using ghc-9.2.8 (after compiling for a long time):

fd unison result/
result/lib/x86_64-linux-ghc-9.2.8/libHSunison-cli-0.0.0-9ZrQZ6gitIF8svWfJHSDe4-ghc9.2.8.so
result/lib/x86_64-linux-ghc-9.2.8/unison-cli-0.0.0-9ZrQZ6gitIF8svWfJHSDe4/
result/lib/x86_64-linux-ghc-9.2.8/unison-cli-0.0.0-9ZrQZ6gitIF8svWfJHSDe4/
...

Looking at CI it seems to also be pulling in ghc-9.2.8?

Any ideas on what might cause this?

sellout · 2024-06-28T16:29:38Z

Hmm I tried nix -L build '.#component-unison-cli:lib:unison-cli' locally and it seems to have built using ghc-9.2.8 (after compiling for a long time)

This is the expected version on trunk. GHC 9.6.5 support hasn’t been merged yet. For these same changes for that GHC, look at #5155.

As far as the long build – it didn’t rebuild GHC itself, right? 🤞🏼 (That was an issue we were having before) Soon our cache should have all of the latest hashes cached (after merging this, if not before).

ChrisPenner

Thanks Greg! I don't use the nix builds so it's got Cody's thumbs up it's all good by me!

neduard · 2024-06-28T17:12:44Z

it didn’t rebuild GHC itself, right?

I think it did given the time and since I recall (and I can't retest it now as https://ci.zw3rk.com is returning a 502), but it's okay; it sounds like a cache propagation issue. Thanks for clarifying regarding the ghc-version 👍

aryairani · 2024-07-11T19:59:00Z

this is part of #5142 now

sellout added 12 commits June 27, 2024 12:37

Don’t export packages for individual build tools

6153c71

Don’t hide unison-project in an overlay

ff62b21

Overlays are for derivations, and this isn’t one. Putting it in an overlay also just gives us more levels of indirection to dig through to figure out where things are coming from.

Remove dead code

07c7ca4

Remove reference to non-existent Cabal component

90d59bb

Actually cache UCM in CI

34745dc

Override tool versions directly in the package set

66f78e4

Minor simplification of haskell-nix-flake

d483188

Eliminate only-tools devShell

71ce523

`cabal-local` no longer triggers rebuilds of GHC, so now we can use the devShell that provides the same environment as our build.

Can now include the devShells in all package

967eb29

This means those environments will also be cached in CI.

Update the development docs for current Nix

73006b6

This was referenced Jun 28, 2024

Nix support for GHC upgrade #5155

Merged

Upgrade to LTS 22.26 GHC 9.6.5 #5142

Merged

sellout marked this pull request as ready for review June 28, 2024 04:19

sellout requested a review from a team as a code owner June 28, 2024 04:19

sellout requested review from aryairani and ceedubs June 28, 2024 04:19

ceedubs reviewed Jun 28, 2024

View reviewed changes

ceedubs approved these changes Jun 28, 2024

View reviewed changes

ChrisPenner reviewed Jun 28, 2024

View reviewed changes

sellout added 2 commits July 1, 2024 08:48

Add Cachix to the dev tooling

49f7d89

Use a newer runner for Nix caching on Linux

7c1a7b8

sellout marked this pull request as draft July 1, 2024 15:07

aryairani closed this Jul 11, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Simplifying Nix #5154

Simplifying Nix #5154

sellout commented Jun 27, 2024 •

edited

Loading

sellout commented Jun 28, 2024

neduard commented Jun 28, 2024

ceedubs left a comment

sellout commented Jun 28, 2024

neduard commented Jun 28, 2024

sellout commented Jun 28, 2024

ChrisPenner left a comment

neduard commented Jun 28, 2024

aryairani commented Jul 11, 2024 •

edited

Loading

Simplifying Nix #5154

Simplifying Nix #5154

Conversation

sellout commented Jun 27, 2024 • edited Loading

Overview

Interesting/controversial decisions

Loose ends

sellout commented Jun 28, 2024

neduard commented Jun 28, 2024

ceedubs left a comment

Choose a reason for hiding this comment

sellout commented Jun 28, 2024

neduard commented Jun 28, 2024

sellout commented Jun 28, 2024

ChrisPenner left a comment

Choose a reason for hiding this comment

neduard commented Jun 28, 2024

aryairani commented Jul 11, 2024 • edited Loading

sellout commented Jun 27, 2024 •

edited

Loading

aryairani commented Jul 11, 2024 •

edited

Loading