Skip to content

Commit

Permalink
chore: add Nix instructions
Browse files Browse the repository at this point in the history
  • Loading branch information
realeinherjar committed Oct 21, 2023
1 parent cb80e62 commit 1b283bb
Show file tree
Hide file tree
Showing 2 changed files with 209 additions and 6 deletions.
33 changes: 27 additions & 6 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ Any bug may cost users real money. That being said, we deeply welcome people
contributing for the first time to an open source project or picking up Rust while
contributing. Don't be shy, you'll learn.

Communications Channels
Communication Channels
-----------------------

Communication about BDK happens primarily on the [BDK Discord](https://discord.gg/dstn4dQ).
Expand All @@ -31,8 +31,8 @@ facilitates social contribution, easy testing and peer review.
To contribute a patch, the workflow is as follows:

1. Fork Repository
2. Create topic branch
3. Commit patches
1. Create topic branch
1. Commit patches

In general commits should be atomic and diffs should be easy to read.
For this reason do not mix any formatting fixes or code moves with actual code
Expand All @@ -48,15 +48,36 @@ hesitate to split it into multiple small, focused PRs.

The Minimal Supported Rust Version is **1.57.0** (enforced by our CI).

Commits should cover both the issue fixed and the solution's rationale.
These [guidelines](https://chris.beams.io/posts/git-commit/) should be kept in mind. Commit messages should follow the ["Conventional Commits 1.0.0"](https://www.conventionalcommits.org/en/v1.0.0/) to make commit histories easier to read by humans and automated tools.

To facilitate communication with other contributors, the project is making use
of GitHub's "assignee" field. First check that no one is assigned and then
comment suggesting that you're working on it. If someone is already assigned,
don't hesitate to ask if the assigned party or previous commenter are still
working on it if it has been awhile.

Our CI can be replicated locally with Nix.
Just run `nix flake check`.
Our Nix setup can greatly increase developer experience
and onboarding of new contributors.
Please check the [`NIX.md`](NIX.md) for details
on how to install Nix and why we use it.

Commit policy
-------------

Commits should cover both the issue fixed and the solution's rationale.
These [guidelines](https://chris.beams.io/posts/git-commit/) should be kept in mind.

We enforce two commit styles in our CI:

1. Commits should be signed with GPG using a key with a valid email address.
1. Commit messages should follow the ["Conventional Commits 1.0.0"](https://www.conventionalcommits.org/en/v1.0.0/)
to make commit histories easier to read by humans and automated tools.
Commits starting with `Merge ...` which GitHub automatically generates
are exempt from this rule.

You can use Nix to automatically and easily check locally these rules before commit and push events.
Please check the [`NIX.md`](NIX.md) for details.

Deprecation policy
------------------

Expand Down
182 changes: 182 additions & 0 deletions NIX.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,182 @@
# BDK's Nix Guidelines

This document outlines:

1. What is Nix and how to install.
1. BDK's Nix-based tests and CI.
1. Developer best-practices with Nix.
1. How to run replicate CI tests locally.

## Nix

We use [Nix](https://nixos.org/) as the CI tool for BDK and also as a
development environment for the project.
Nix is purely functional.
Everything is described as an expression/function,
taking some inputs and producing deterministic outputs.
This guarantees reproducible results and makes caching everything easy.
Nix expressions are lazy. Anything described in Nix code will only be executed
if some other expression needs its results.
This is very powerful but somewhat unnatural for developers not familiar
with functional programming.

There are several resources to get started with Nix:

- [NixOS Wiki](https://nixos.wiki/).
- [Nix Reference Manual](https://nixos.org/manual/nix/stable/).
- [`nixpkgs` Manual](https://nixos.org/manual/nixpkgs/stable/).
- [Official documentation for getting things done with Nix](https://nix.dev/).

There's also the [`nix-bitcoin` project](https://nixbitcoin.org/).

To install Nix please follow the [official Nix documentation](https://nixos.org/manual/nix/stable/installation/installation.html).
You may need to enable Flakes support, check the instructions at [NixOS Wiki entry on Flakes](https://nixos.wiki/wiki/Flakes).

## Why are we using Nix

We want to make BDK's development easier.
This means two things:

1. Enhance developer experience.
1. Facilitate onboarding of new contributors.

Both of these can be accomplished with Nix.

BDK has many crates in the workspace,
and proper development and testing needs dependencies to be installed,
and environment variables to be set.
This can be daunting for new contributors.

BDK needs several Rust versions with different compilation targets.
This can be cumbersome to install, but also needs proper maintainability,
since these versions need to be updated frequently.

BDK enforces commit styles and commit policies (please check [CONTRIBUTING.md](CONTRIBUTING.md)).
This is difficult to enforce locally
and can be a source of frustration during contribution reviews,
potentially leading to a lot of wasted time and pushing new contributors away.

Finally, BDK has a lot of tests and checks.
It is difficult to replicate these locally.

All the above can easily be accomplished with Nix.
Nix is available for macOS, Linux, and Windows;
while also being easy to install.
Nix has a rich [community](https://nixos.org/community/) and one can find help
on the [Nix's Forums](https://discourse.nixos.org/),
[Nix's Discord](https://discord.gg/RbvHtGa),
and [Nix's Matrix channel](https://matrix.to/#/#community:nixos.org).
Additionally, Nix issues and errors can be searched in any search engine,
and in the [Nix's stackoverflow](https://stackoverflow.com/questions/tagged/nix+or+nixpkgs+or+nixos+or+nixops).

## BDK's Nix-based tests and CI

**TLDR: To replicate everything that our CI checks just do `nix flake check`**.

BDK's tests and checks are run using Nix.
If you want to run the tests locally, you can do so with:

```text
nix flake -L check --keep-failed
```

The `-L` flag prints full build logs on standard error.
This is good for debugging.
The `--keep-failed` flag keep temporary directories of failed builds.
This can be useful for caching intermediate results while fixing failed checks.

Under the hood `nix flake check` does the following:

- Checks for typos, GPG-signed and conventional commits, `rustfmt`, `nixpkgs-fmt` (`.nix` files).
- Runs `cargo clippy` in all workspace.
- Runs `cargo fmt` in all workspace.
- Checks dependencies for security advisory using [`rustsec/advisory-db`](https://github.com/rustsec/advisory-db).
- `cargo build` and `cargo test` in whole workspace using the latest Rust version.
- `cargo build` and `cargo test` in whole workspace using the MSRV Rust version.
- `cargo build` and `cargo check` WASM targets using the latest Rust version
on `bdk` and `bdk_esplora` crates.

The checks are by default cached by Nix and subsequent runs use the cached results,
respecting the Nix derivations' dependency graph.

All the necessary dependencies, e.g. `bitcoind` and `esplora`,
along with all necessary environment variables,
are installed and available in the environment that Nix's checks runs in.

Individual checks from the above list can be listed with `nix flake show`
and will be displayed under the `checks` tree.
You can run an individual check with:

```text
nix build -L .#checks.SYSTEM.CHECK --keep-failed
```

Where `SYSTEM` is the system you are running on,
and `CHECK` is the name of the check you want to run.
For example, in a MacOS Apple Silicon machine (`aarch64-darwin`)
and for the `clippy` check you can run:

```text
nix build -L .#checks.aarch64-darwin.clippy --keep-failed
```

We also group all the CI checks inside the `legacyPackages`.
These can be viewed in the `flake.nix` file under the `legacyPackages.ci` set.
You can run any of these with:

```text
nix build -L .#ci.CHECK --keep-failed
```

Where `CHECK` is the name of the check you want to run.
For example to run the `pre-commit-check`:

```text
nix build -L .#ci.pre-commit-check --keep-failed
```

You can check all the tests that our CI performs by inspecting both
the `flake.nix` and `.github/workflows/cont_integration.yml` files.

## Developer best-practices with Nix

We provide several development shells,
also called `devShell`s,
that assist developers with the necessary dependency and environment
to develop and test BDK.

To access these shells, you can run:

```text
nix develop .#SHELL
```

Where `SHELL` is the name of the `devShell` you want to use.
We provide the following shells:

- default: latest Rust, you can omit the `#SHELL` and just run `nix develop .`.
- `MSRV`: MSRV Rust version.
- `WASM`: latest Rust with WASM compilation support.

All of these `devShell`s handle all the necessary dependencies and environment variables needed.
They also provide pre-commit git hooks that checks either during commit or push events the following:

- GPG-signed commits (on a `git push` event).
- Conventional commit style commit messages (on a `git commit` event).
- `rustfmt` adherence (on a `git commit` event).
- `nixpkgs-fmt` adherence (on a `git commit` event).
- Typos in the commit content (on a `git commit` event).

Regarding the typos check, there might be some false positives.
In that case you can add a regex rule to filter out the typos in the `.typos.toml` file.
We already included all the false positives we've found so far with some explainable comments.
Hence, you should be able to follow the examples and add your own.
Additionally, you can find more information in [`crate-ci/typos`](https://github.com/crate-ci/typos).

As a final note on the `devShell`s, they can be conveniently automated with
[`nix-community/nix-direnv`](https://github/nix-community/nix-direnv).
`nix-direnv`, once installed, will:

- Enable shell completion in `devShell`s.
- Seamless integrate pre-commit checks even outside a `devShell`.
- Improved caching of `devShell`s.

0 comments on commit 1b283bb

Please sign in to comment.