Skip to content

Spec for working with the OCaml compiler

Jump to bottom
Anil Madhavapeddy edited this page May 29, 2020 · 4 revisions

Opam is primarily a tool to manage installation of OCaml packages, but it is also used to manage compilers and different build environments. In opam 2.0 and higher, compilers are exposed as a normal package (so that a package can just depend on ocaml >= 4.08.0), but the OCaml compilers are special citizens of the opam ecosystem. Here are some scenarios we need to support well for opam 2.1:

Usecases

Trying a compiler patch

  • “A” works on a compiler feature. They push their changes to a git branch on a fork of the compiler. They call for testers.
  • “B” wants to try this patch. They use opam to set up a new switch which is based on this patched version.
    • B can install packages from the main opam repository (packages do not have to know about A’s fork).
    • Some packages can explicitly be available only for this forked compiler.
    • A reference (such as a branch name) is a moving target, so there are ways for B to know which particular commit their switch points to, and to update this reference.

Some example usecases:

  • Multicore (https://github.com/ocaml-multicore/ocaml-multicore) is a forked version of the OCaml compiler. This provides a way for users to test it. Library authors can use this to publish packages that depend on multicore features. (see instructions)
  • Each PR against ocaml/ocaml is a patched version. This provides a way to test each individual PR. There used to be a GitHub sync that was discontinued due to solver overload when added.

Trying a compiler prerelease

The OCaml developers complete a release cycle. The next version is almost ready. They call for testers.

End users can install this prerelease version through opam. This version is “by default” compatible with packages in the repository, though of course the actual build might fail.

This story is very similar to the previous one, considering the whole release cycle as a large patch. The main difference is that prereleases have stable names (e.g. beta1, beta2) rather than reference names.

An alternative way to look at this is that prereleases are normal compiler versions (like 4.10.0), except that they are not picked by default by the solver. Pypi, npm, rubygems, etc have similar features (this could be applied to packages as well but is out of scope for this story).

Using a locally built compiler to build local projects

Somebody develops a feature on the ocaml repository (compiler or standard library) and wants to use it in a local project. They can set up an opam switch that uses the corresponding compiler. This work does not have to be pushed anywhere. It is possible to install packages in this switch.

Depending on the kind of changes made on the compiler, two workflows are possible to sync the compiler (see https://github.com/gasche/opam-compiler-conf):

  • For changes that do not affect the compilation, it is possible to install the compiler in place without recompiling all installed packages.
  • In the general case, if compilation changes, it is possible to rebuild all affected packages.

Proposed workflow, from compiler author's perspective:

  • Hacking in ocaml, tree is configured and built
  • Do something to set-up a (local?) switch (at present opam switch create … and opam pin add --inplace-build)
  • Test packages
  • Hack on the tree
  • Something simple to update the opam view (e.g. opam update)
  • Some way to update but not rebuild everything (e.g. for an error message which is updated)

Status

  • Under discussion for opam 2.1 inclusion. Not committed yet.
  • Feature design in progress.
  • Authorship: @emillon @avsm

Implementation

Clone this wiki locally