Open-Source Project Management: Improving the Dune Release Process

How do you best manage open-source projects to make them easier to maintain and get more reliable updates out to users? Dune is the most widely used build system for OCaml, and its release cadence directly impacts a large share of OCaml developers.

At Tarides, we’re working on the fast-evolving Dune package management system, and we want to get new features and bug fixes out quickly. Getting them out to users depends on developing a predictable, low-friction, release process.

Over time, we’ve worked hard alongside the OCaml community to iterate on that process. In this post, we will share how we have recently identified gaps in the workflows, reduced the cognitive load for maintainers, and automated key parts of the pipeline.

Our Starting Point: The Minor Release Process

To give you a lay of the land, this is how a minor release (in the style of x.y.0.) was normally structured:

• In preparation for the release, we opened a tracking issue on GitHub that lists all blocking issues.
• During the alpha phase, we cherry-picked regression fixes from main and put them on a release-candidate branch.
• We put the release-candidate branch through multiple pre-release checks, including MirageOS build compatibility, opam packaging checks, and reverse dependency (revdep) build checks.
• Once the final release was ready, we generated a changelog and made a pull request to the opam repository.

This process worked well, but still involved significant manual effort and cognitive load. Some classes of failures were only being caught in the final stage when submitting to opam-repository. We set out to catch those failures earlier.

Improvements

We identified three key opportunities to address: package dependency specifications, reverse dependencies, and the release automations.

Catching Package Issues Early

We had run into packaging problems with Dune’s sub-packages in several minor releases. A Dune release involves not just the dune package but also sub-packages like dune-configurator, dune-rpc, and xdg. While we had CI checks for Dune itself, we had no systematic checks to verify that each sibling package could be installed in isolation with its own dependency set. For some examples of this problem in action, check out issues #13674 and #12947.

We also noticed problems with lower-bound dependency tests. Most opam files declare the minimum supported version of each dependency, and opam-repo-ci checks that it works. However, we weren’t catching the lower-bound failures. You may, justifiably, assume that the easiest fix would be to run opam-repo-ci on Dune early enough to discover them, but running opam-repo-ci on an OCaml package is not currently possible without non-trivial upstream changes.

Instead, we added the equivalent GitHub Actions workflow to opam that now lints all opam packages and installs each one in isolation.

We achieve this by using an opam trick where we pin all of the packages without installing them:

opam pin add "$pkg.dev" . -n

Pinning them in this way ensures that all the Dune packages are available to opam without being installed all at once. We can then build all the packages in isolation and get a report if there are any failures. Running the lower-bounds test on top of this requires just one more step using opam's built-in solver flags:

opam install --solver=builtin-0install \
  "--criteria=+count[version-lag,solution]" "$pkg.dev" -y

This catches any lower-bound issues at CI time, well before a release is attempted.

Testing Reverse Dependencies

Packaging and building Dune is only one part of the puzzle. As for releases, another equally important part is how the new release affects the rest of the ecosystem. This is especially critical for a package like dune, where a large chunk of the ecosystem depends on it to build.

Building every reverse dependency in full would take many hours and require significant computer power. As a compromise, we use a list of important reverse dependencies which we test as part of our pre-release checks. These include ocaml-lsp-server, a key platform tool whose breakage would be immediately felt by developers.

To improve how the list is selected, we wrote a small script to identify the packages with the most reverse dependencies in opam-repository. This GitHub CI action then uses opam to verify that these packages build cleanly with the release candidate.

Previously, we used Nix and nixpkgs to build reverse dependencies. We switched to opam for two reasons: firstly, it is consistent with the upstream opam-repo-ci workflow (which also uses opam), and secondly, not all packages in opam-repository are packaged in nixpkgs, so opam gives us more scope for testing.

We store the output of the reverse dependency opam-repo-ci build runs of every Dune release. We maintain a small script that compares the results of the current release with past ones to identify any new and unexpected failures. We plan to tidy up this script and publish it in a future update.

Automations

After implementing better CI coverage, we turned our attention to the release process itself. Even though tools like dune-release and opam-publish already automated part of the release, there was still a lot of manual effort required on behalf of the developer. For example, tasks like creating git branches for release candidates, coordinating regression cherry-picks onto the release branch, and keeping everything in sync were done ‘by hand’. To reduce the cognitive load and make the process more consistent, we automated these tasks.

We want to go further and automate more of the release itself! This work is still in progress, but we are focussing on workflow improvements like generating a changelog with the right version and committing it, and then automatically running dune-release to start the release process. The goal is a smoother workflow where contributors can focus on the content of a release rather than minutiae.

Stay in Touch

How do you manage your open-source projects? Have you used automations to improve your workflows? Reach out to us on Bluesky, Mastodon, Threads, and LinkedIn or sign up for our mailing list to stay updated on our latest projects. You can also connect with other OCaml users on Discuss to share your experience and feedback. We look forward to hearing your thoughts!