nixos/azure: add Gen 2 VM, aarch64 and accelerated networking support

[codgician]

Description of changes

[azure-agent]

• Removed x86 assertion as the latest waagent already works on aarch64.

[azure-common]

• Added option to enable support for accelerated networking, following official doc for custom vm image: virtualisation.azure.acceleratedNetworking = true.
• Remove reference of headless profile to make the console input work (and being able to select boot entries).

[azure-image]

• Added option for generating Gen 2 VM image (efi partition): virtualisation.azureImage = "v1" or "v2".
• Added option for specifying esp partition size: virtualisation.azureImage.bootSize

---

I have created an example flake for generating a test vhd for this change: github:codgician/azure-aarch64-nixos.

The result is promising:

Things done

• Built on platform(s)
  • x86_64-linux
  • aarch64-linux
  • x86_64-darwin
  • aarch64-darwin
• For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)
  • sandbox = relaxed
  • sandbox = true
• Tested, as applicable:
  • NixOS test(s) (look inside nixos/tests)
  • and/or package tests
  • or, for functions and "core" functionality, tests in lib/tests or pkgs/test
  • made sure NixOS tests are linked to the relevant packages
• Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
• Tested basic functionality of all binary files (usually in ./result/bin/)
• 24.11 Release Notes (or backporting 23.11 and 24.05 Release notes)
  • (Package updates) Added a release notes entry if the change is major or breaking
  • (Module updates) Added a release notes entry if the change is significant
  • (Module addition) Added a release notes entry if adding a new NixOS module
• Fits CONTRIBUTING.md.

---

Add a 👍 reaction to pull requests you find important.

[nixos-discourse]

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/prs-ready-for-review/3032/4451

[zimbatm]

thanks for your work!

Given the modules haven't been touched since 2016, I propose to merge your work once you confirm that everything is working as intended.

[zimbatm]

Assuming that the 👍 means it's all good

[nh2]

Thanks for having tested this with a gen v1, I wonder if there is a way to access systemd-boot's generation for gen2 during boot, I haven't seen any documentation about that, I believe most Azure images are using GRUB for gen v1 or v2.

@AkechiShiro Yes, it's possible.

For example, I use this config to achive with systemd-boot on an Azure V2 VM (on https://github.com/rudesome/nixos-on-azure/ instead of on this PR, since I'm on nixos-24.05, but the result should be equivalent):

    # Note that using `systemd-boot` breaks `switch-to-configuration boot`
    # (but not `nixos-rebuild`), see https://github.com/NixOS/nixpkgs/issues/82851:
    # > * For systemd boot, the new generation is not even accessible through the default boot entry.
    # The problem there is that `switch-to-configuration boot` only creates
    # bootloader entries that are system profiles, and does itself NOT create
    # system profiles.
    # This will only be fixed with the new `apply` script in NixOS >= 25.05.
    # Until then, we manually create a new system profile:
    # We must run manually during the deployment (before `switch-to-configuration boot`):
    #
    #     N=$(ls -1 /nix/var/nix/profiles | sed -nE 's/system-([0-9]+)-link/\1/p' | sort --numeric-sort | tail -n1) ; NEW_SYSTEM_LINK="/nix/var/nix/profiles/system-$((N+1))-link" ; ln -s "$STORE_PATH" "$NEW_SYSTEM_LINK" && nix-env -p /nix/var/nix/profiles/system --set "$NEW_SYSTEM_LINK"
    #     ls -la /nix/var/nix/profiles  # verify output
    #     $STORE_PATH/bin/switch-to-configuration boot
    #
    # Be aware of also this issue regarding `gcroots`:
    #     https://github.com/NixOS/nixpkgs/issues/73812

    # nixpkgs's `azure-common.nix` sets this to 0 for speediest boot times,
    # but we want to see the menu to be able to rollback from the serial console.
    boot.loader.timeout = (lib.mkOverride 99) 5;
    boot.loader.systemd-boot.enable = true;
    boot.loader.efi.canTouchEfiVariables = true;

[nh2]

@codgician You write in the PR description

Remove reference of headless profile to make the console input work (and being able to select boot entries).

But then you'r eadding profiles/headless.nix back here, so that conflicts.

Does the current config actually allow you to select GRUB boot menus?

Also, what does headless.nix have to do with GRUB entries?

There are no GRUB related options being set here:

https://github.com/codgician/nixpkgs/blob/94683f9e55d47996b7403791863143f7afad8a51/nixos/modules/profiles/headless.nix

The only thing that's disabled is the serial console of Linux -- which is also bad breaks the Azure Serial Console working on NixOS when Linux is booted, but should not affect GRUB menu selection.

Can you elaborate?

Another question:

How does this work at all when v2 is used, given that boot.loader.grub.efiSupport defaults to false?

---

I'm currently fixing it like this, for GRUB on V2 (which is EFI):

    boot.loader.systemd-boot.enable = false;
    boot.loader.grub = {
      enable = true;
      efiSupport = true;
      # Override needed because nixpkgs's `azure-common.nix` sets `device = "/dev/sda"`,
      # which renders into a mergeable list option that would include that AND `nodev`.
      devices = lib.mkOverride 99 [ "nodev" ];
      efiInstallAsRemovable = true; # see note [hetzner-canTouchEfiVariables] for an explanation
      # Enable Azure Serial Console to work with GRUB.
      # See https://learn.microsoft.com/en-us/troubleshoot/azure/virtual-machines/linux/serial-console-grub-proactive-configuration
      # Note previous serial output may not be cleared by GRUB, making
      # it hard to read the menu until you navigate with the arrow keys.
      extraConfig = ''
        serial --unit=0 --speed=115200 --word=8 --parity=no --stop=1
        terminal_input --append serial
        terminal_output --append serial
      '';
    };
    boot.loader.efi.canTouchEfiVariables = false;

Note the extraConfig to make the GRUB2 serial output work.

Screenshot of it working in Azure:

And I'm re-enabling the serial output of Linux like this:

    # Explicitly enable serial-getty on ttyS0, otherwise a login prompt will only
    # be shown on the first serial device (hvc0 in our case).
    # See: http://0pointer.de/blog/projects/serial-console.html (section Serial Terminals)
    systemd.services."serial-getty@ttyS0" = {
      enable = true;
      wantedBy = [ "getty.target" ]; # to start at boot
      serviceConfig.Restart = "always"; # restart when session is closed
    };
  };

[codgician]

For 1, you are right and thanks for the nice catch. I might have overlooked when creating this PR.

For 2, I've been using systemd-boot in my test flake. I did not elaborate much on the grub + UEFI scenario so I kept the pre-existing defaults. Thanks for your report and I will try to fix that for v2 generation.

I will create a new PR to address them asap.

[codgician]

I have drafted PR #359365 to address the issues you've mentioned and will publish it once further testings are done. Please feel free to review and leave your comments. Thanks!

[codgician]

I have published PR #359365 after testing via flake codgician/azure-aarch64-nixos. Now grub2 console output should work for both Gen 1 and Gen 2 VM.