diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 6a83ccf575..a4cff173c9 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -13,7 +13,7 @@ Any bug may cost users real money. That being said, we deeply welcome people contributing for the first time to an open source project or picking up Rust while contributing. Don't be shy, you'll learn. -Communications Channels +Communication Channels ----------------------- Communication about BDK happens primarily on the [BDK Discord](https://discord.gg/dstn4dQ). @@ -31,8 +31,8 @@ facilitates social contribution, easy testing and peer review. To contribute a patch, the workflow is as follows: 1. Fork Repository - 2. Create topic branch - 3. Commit patches + 1. Create topic branch + 1. Commit patches In general commits should be atomic and diffs should be easy to read. For this reason do not mix any formatting fixes or code moves with actual code @@ -48,15 +48,36 @@ hesitate to split it into multiple small, focused PRs. The Minimal Supported Rust Version is **1.57.0** (enforced by our CI). -Commits should cover both the issue fixed and the solution's rationale. -These [guidelines](https://chris.beams.io/posts/git-commit/) should be kept in mind. Commit messages should follow the ["Conventional Commits 1.0.0"](https://www.conventionalcommits.org/en/v1.0.0/) to make commit histories easier to read by humans and automated tools. - To facilitate communication with other contributors, the project is making use of GitHub's "assignee" field. First check that no one is assigned and then comment suggesting that you're working on it. If someone is already assigned, don't hesitate to ask if the assigned party or previous commenter are still working on it if it has been awhile. +Our CI can be replicated locally with Nix. +Just run `nix flake check`. +Our Nix setup can greatly increase developer experience +and onboarding of new contributors. +Please check the [`NIX.md`](NIX.md) for details +on how to install Nix and why we use it. + +Commit policy +------------- + +Commits should cover both the issue fixed and the solution's rationale. +These [guidelines](https://chris.beams.io/posts/git-commit/) should be kept in mind. + +We enforce two commit styles in our CI: + +1. Commits should be signed with GPG using a key with a valid email address. +1. Commit messages should follow the ["Conventional Commits 1.0.0"](https://www.conventionalcommits.org/en/v1.0.0/) + to make commit histories easier to read by humans and automated tools. + Commits starting with `Merge ...` which GitHub automatically generates + are exempt from this rule. + +You can use Nix to automatically and easily check locally these rules before commit and push events. +Please check the [`NIX.md`](NIX.md) for details. + Deprecation policy ------------------ diff --git a/NIX.md b/NIX.md new file mode 100644 index 0000000000..56dc25f120 --- /dev/null +++ b/NIX.md @@ -0,0 +1,182 @@ +# BDK's Nix Guidelines + +This document outlines: + +1. What is Nix and how to install. +1. BDK's Nix-based tests and CI. +1. Developer best-practices with Nix. +1. How to run replicate CI tests locally. + +## Nix + +We use [Nix](https://nixos.org/) as the CI tool for BDK and also as a +development environment for the project. +Nix is purely functional. +Everything is described as an expression/function, +taking some inputs and producing deterministic outputs. +This guarantees reproducible results and makes caching everything easy. +Nix expressions are lazy. Anything described in Nix code will only be executed +if some other expression needs its results. +This is very powerful but somewhat unnatural for developers not familiar +with functional programming. + +There are several resources to get started with Nix: + +- [NixOS Wiki](https://nixos.wiki/). +- [Nix Reference Manual](https://nixos.org/manual/nix/stable/). +- [`nixpkgs` Manual](https://nixos.org/manual/nixpkgs/stable/). +- [Official documentation for getting things done with Nix](https://nix.dev/). + +There's also the [`nix-bitcoin` project](https://nixbitcoin.org/). + +To install Nix please follow the [official Nix documentation](https://nixos.org/manual/nix/stable/installation/installation.html). +You may need to enable Flakes support, check the instructions at [NixOS Wiki entry on Flakes](https://nixos.wiki/wiki/Flakes). + +## Why are we using Nix + +We want to make BDK's development easier. +This means two things: + +1. Enhance developer experience. +1. Facilitate onboarding of new contributors. + +Both of these can be accomplished with Nix. + +BDK has many crates in the workspace, +and proper development and testing needs dependencies to be installed, +and environment variables to be set. +This can be daunting for new contributors. + +BDK needs several Rust versions with different compilation targets. +This can be cumbersome to install, but also needs proper maintainability, +since these versions need to be updated frequently. + +BDK enforces commit styles and commit policies (please check [CONTRIBUTING.md](CONTRIBUTING.md)). +This is difficult to enforce locally +and can be a source of frustration during contribution reviews, +potentially leading to a lot of wasted time and pushing new contributors away. + +Finally, BDK has a lot of tests and checks. +It is difficult to replicate these locally. + +All the above can easily be accomplished with Nix. +Nix is available for macOS, Linux, and Windows; +while also being easy to install. +Nix has a rich [community](https://nixos.org/community/) and one can find help +on the [Nix's Forums](https://discourse.nixos.org/), +[Nix's Discord](https://discord.gg/RbvHtGa), +and [Nix's Matrix channel](https://matrix.to/#/#community:nixos.org). +Additionally, Nix issues and errors can be searched in any search engine, +and in the [Nix's stackoverflow](https://stackoverflow.com/questions/tagged/nix+or+nixpkgs+or+nixos+or+nixops). + +## BDK's Nix-based tests and CI + +**TLDR: To replicate everything that our CI checks just do `nix flake check`**. + +BDK's tests and checks are run using Nix. +If you want to run the tests locally, you can do so with: + +```text +nix flake -L check --keep-failed +``` + +The `-L` flag prints full build logs on standard error. +This is good for debugging. +The `--keep-failed` flag keep temporary directories of failed builds. +This can be useful for caching intermediate results while fixing failed checks. + +Under the hood `nix flake check` does the following: + +- Checks for typos, GPG-signed and conventional commits, `rustfmt`, `nixpkgs-fmt` (`.nix` files). +- Runs `cargo clippy` in all workspace. +- Runs `cargo fmt` in all workspace. +- Checks dependencies for security advisory using [`rustsec/advisory-db`](https://github.com/rustsec/advisory-db). +- `cargo build` and `cargo test` in whole workspace using the latest Rust version. +- `cargo build` and `cargo test` in whole workspace using the MSRV Rust version. +- `cargo build` and `cargo check` WASM targets using the latest Rust version + on `bdk` and `bdk_esplora` crates. + +The checks are by default cached by Nix and subsequent runs use the cached results, +respecting the Nix derivations' dependency graph. + +All the necessary dependencies, e.g. `bitcoind` and `esplora`, +along with all necessary environment variables, +are installed and available in the environment that Nix's checks runs in. + +Individual checks from the above list can be listed with `nix flake show` +and will be displayed under the `checks` tree. +You can run an individual check with: + +```text +nix build -L .#checks.SYSTEM.CHECK --keep-failed +``` + +Where `SYSTEM` is the system you are running on, +and `CHECK` is the name of the check you want to run. +For example, in a MacOS Apple Silicon machine (`aarch64-darwin`) + and for the `clippy` check you can run: + +```text +nix build -L .#checks.aarch64-darwin.clippy --keep-failed +``` + +We also group all the CI checks inside the `legacyPackages`. +These can be viewed in the `flake.nix` file under the `legacyPackages.ci` set. +You can run any of these with: + +```text +nix build -L .#ci.CHECK --keep-failed +``` + +Where `CHECK` is the name of the check you want to run. +For example to run the `pre-commit-check`: + +```text +nix build -L .#ci.pre-commit-check --keep-failed +``` + +You can check all the tests that our CI performs by inspecting both +the `flake.nix` and `.github/workflows/cont_integration.yml` files. + +## Developer best-practices with Nix + +We provide several development shells, +also called `devShell`s, +that assist developers with the necessary dependency and environment +to develop and test BDK. + +To access these shells, you can run: + +```text +nix develop .#SHELL +``` + +Where `SHELL` is the name of the `devShell` you want to use. +We provide the following shells: + +- default: latest Rust, you can omit the `#SHELL` and just run `nix develop .`. +- `MSRV`: MSRV Rust version. +- `WASM`: latest Rust with WASM compilation support. + +All of these `devShell`s handle all the necessary dependencies and environment variables needed. +They also provide pre-commit git hooks that checks either during commit or push events the following: + +- GPG-signed commits (on a `git push` event). +- Conventional commit style commit messages (on a `git commit` event). +- `rustfmt` adherence (on a `git commit` event). +- `nixpkgs-fmt` adherence (on a `git commit` event). +- Typos in the commit content (on a `git commit` event). + +Regarding the typos check, there might be some false positives. +In that case you can add a regex rule to filter out the typos in the `.typos.toml` file. +We already included all the false positives we've found so far with some explainable comments. +Hence, you should be able to follow the examples and add your own. +Additionally, you can find more information in [`crate-ci/typos`](https://github.com/crate-ci/typos). + +As a final note on the `devShell`s, they can be conveniently automated with +[`nix-community/nix-direnv`](https://github/nix-community/nix-direnv). +`nix-direnv`, once installed, will: + +- Enable shell completion in `devShell`s. +- Seamless integrate pre-commit checks even outside a `devShell`. +- Improved caching of `devShell`s.