Skip to content

Commit

Permalink
workflow: Split prerequisites out to separate file
Browse files Browse the repository at this point in the history
Signed-off-by: Rune Morling <[email protected]>
  • Loading branch information
ermo committed Jan 1, 2025
1 parent ea222a8 commit cce1645
Show file tree
Hide file tree
Showing 2 changed files with 101 additions and 53 deletions.
89 changes: 36 additions & 53 deletions content/en/docs/packaging/workflow/packaging_workflow.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,44 +6,9 @@ description = "Building and testing `stone.yaml` recipe build artifacts"
+++


Before the system can build recipes, a few prerequisites need to be installed and a new directory
for storing local build artefacts needs to be set up.
Once the [prerequisites](../prerequisites) have been handled, it is time to learn how to install
newly built local moss-format .stone packages.

## Installing the `build-essential` package

We maintain a `build-essential` metapackage that should contain the basics for getting started
with packaging on Serpent OS.

```bash
sudo moss sync -u
sudo moss it build-essential
```

## Activating the Serpent OS helper scripts

The easiest way to create a local repository is to use the helper script distributed with the
Serpent OS recipe repository in the `tools/` directory.

```bash
mkdir -pv repos/serpent-os/
pushd repos/serpent-os
git clone https://github.com/serpent-os/recipes
popd
mkdir -pv ~/.bashrc.d/
ln -sv ~/repos/serpent-os/recipes/tools/helpers.bash ~/.bashrc.d/90-serpent-helpers.bash
```


## Adding /etc/subuid and /etc/subgid entries

Since `boulder` uses user-namespaces to set up isolated build roots, it is necessary to set this up
first:

```bash
sudo touch /etc/sub{uid,gid}
sudo usermod --add-subuids 1000000-1065535 --add-subgids 1000000-1065535 root
sudo usermod --add-subuids 1065536-1131071 --add-subgids 1065536-1131071 "$USER"
```

## Understanding moss-format repositories

Expand All @@ -57,14 +22,14 @@ The `stone.index` file is what both moss and boulder consult when they check whi
available to be installed into moss-maintained system roots.

Adding a moss-format repository is as simple as registering a new location from where to fetch
`stone.index` files.
`stone.index` files, which will be shown in detail later on this page.

### `moss` build roots

Every time a package is built, `boulder` calls out to `moss` to have it construct a pristine build root
directory with the necessary package build prerequisites installed.
directory (called a 'buildroot') with the necessary package build prerequisites installed.

The packages in this build root are resolved from one or more moss `stone.index` files, sorted in
The packages in this buildroot are resolved from one or more moss `stone.index` files, sorted in
descending priority, such that the highest priority repository "wins" when package providers are
resolved.

Expand Down Expand Up @@ -103,24 +68,26 @@ Boulder will need to have its list of "build profiles" be updated before it will
boulder profile list
# output
default-x86_64:
- volatile = https://packages.serpentos.com/volatile/x86_64/stone.index [0]
- volatile = https://packages.serpentos.com/volatile/x86_64/stone.index [0]

# add new local-x86_64 build profile
boulder profile add --repo name=volatile,uri=https://packages.serpentos.com/volatile/x86_64/stone.index,priority=0 local-x86_64
boulder profile update --repo name=local,uri=file://${HOME}/.cache/local_repo/x86_64/stone.index,priority=100 local-x86_64
boulder profile add \
--repo name=volatile,uri=https://packages.serpentos.com/volatile/x86_64/stone.index,priority=0 local-x86_64 \
--repo name=local,uri=file://${HOME}/.cache/local_repo/x86_64/stone.index,priority=100 local-x86_64
boulder profile list
# output
default-x86_64:
- volatile = https://packages.serpentos.com/volatile/x86_64/stone.index [0]
local-x86_64:
- volatile = https://packages.serpentos.com/volatile/x86_64/stone.index [0]
- local = file:///home/ermo/.cache/local_repo/x86_64/stone.index [10]
- local = file:///home/ermo/.cache/local_repo/x86_64/stone.index [100]
```


### Enabling `moss` to install local repository packages

Boulder builds from the `volatile` repository, hence it is wise to set it up as being disabled by
default:
Listing and adding moss-format repositories containing stone.index files done as follows:


```bash
moss repo list
Expand All @@ -136,16 +103,25 @@ moss repo list
- local = file:///home/ermo/.cache/local_repo/x86_64/stone.index [100]
```

In the above priority tower, moss-format packages would first get resolved via the `local`

#### Package resolution order

In the above priority tower, each moss-format package would first get resolved via the `local`
repository (priority 100), then from the `volatile` repository (priority 10), and finally from the
`unstable` repository (priority 0), the latter of which is the official upstream Serpent OS
moss-format `.stone` package repository.


#### Disabling moss-format repositories

Users of Serpent OS should generally _not_ have the `volatile` repository be enabled, because this
repository is where new `.stone` packages land right after being built, which means the repository
can potentially be in an undefined and volatile state when building large build queues (hence the
name).

Therefore, it can be useful to disable moss-format repositories without deleting their definitions
from the local system:

```bash
moss repo disable volatile
moss repo disable local
Expand All @@ -156,9 +132,12 @@ moss repo list
- local = file:///home/ermo/.cache/local_repo/x86_64/stone.index [100] (disabled)
```

However, when testing locally built packages, they _must_ be built against the `local-x86_64` boulder
build profile, which in turns relies on the `volatile` repostiory via the boulder `local-x86_64`
build profile.

#### Enabling moss-format repositories

However, when testing locally built packages, they _must_ be built against the `local-x86_64`
boulder build profile, which in turns relies on the `volatile` repository via the boulder
`local-x86_64` build profile.

Hence, when testing locally built packages, you may need to _**temporarily**_ enable the `volatile`
repository for moss to resolve from.
Expand All @@ -185,11 +164,15 @@ gotoserpentroot
chpkg nano
# bump the release number in the nano recipe
just bump
# check the difference between the local state and the upstream recipe state
git diff
# build the bumped nano recipe
just build
# move the build artifacts to the local repository
# check the difference between the local state and the upstream recipe state
git status
# move the newly built .stone build artifacts to the local repository
just mv-local
# list the build artifacts in the local repository
# list the build artifacts present in the local repository
just ls-local
```

Expand Down Expand Up @@ -234,7 +217,7 @@ repository in the previous system state.

## Ending notes

If you have made it this far, congratulations! You should know understand the basic workflow of
If you have made it this far, congratulations! You should now understand the basic workflow of
packaging and managing repositories with Serpent OS.

Tip: execute `just -l` to see a list of supported `just` 'recipes', which are common actions that
Expand Down
65 changes: 65 additions & 0 deletions content/en/docs/packaging/workflow/prerequisites.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,65 @@
+++
title = 'Prerequisites'
date = 2025-01-01T18:28:22+01:00
weight = 10
description = "Prerequisites for building packages on Serpent OS"
+++

To set up a Serpent OS system to be able to build package recipes, a few prerequisites need to be
installed, and a new directory for storing local build artefacts needs to be set up.

## Installing the `build-essential` package

We maintain a `build-essential` metapackage that should contain the basics for getting started
with packaging on Serpent OS.

```bash
sudo moss sync -u
sudo moss it build-essential
```

## Activating the Serpent OS helper scripts

The easiest way to create a local repository is to use the helper script distributed with the
Serpent OS recipe repository in the `tools/` directory.

Start by cloning the recipes/ git repository:

```bash
mkdir -pv repos/serpent-os/
pushd repos/serpent-os
git clone https://github.com/serpent-os/recipes

```

After the recipes/ git repositori has been cloned, symlink helpers.bash into `~/.bashrcd.d/`:

```bash
popd
mkdir -pv ~/.bashrc.d/
ln -sv ~/repos/serpent-os/recipes/tools/helpers.bash ~/.bashrc.d/90-serpent-helpers.bash
```

Finally, execute the following in a new a new terminal tab:

```bash
cd ~
gotoserpentroot
```

If the helpers script has been correctly loaded, the `gotoserpentroot` command should switch to
the directory containing the recipes/ git repository clone.


## Adding /etc/subuid and /etc/subgid entries

Since `boulder` uses user-namespaces to set up isolated build roots, it is necessary to set up
a subuid and a subgid file for the relevant users first:

```bash
sudo touch /etc/sub{uid,gid}
sudo usermod --add-subuids 1000000-1065535 --add-subgids 1000000-1065535 root
sudo usermod --add-subuids 1065536-1131071 --add-subgids 1065536-1131071 "$USER"
```

If `/etc/subuid` and `/etc/subgid` already exist, adapt the above as appropriate.

0 comments on commit cce1645

Please sign in to comment.