Multiple Component support for cabal repl

Following on from our work implementing support for compiling multiple units at once in GHC, we have now been extending the ecosystem to take advantage of this new support. This work has once again been made possible by Hasura. This work continues our productive and long-running collaboration on important and difficult tooling tasks which will ultimately benefit the entire ecosystem.

This post focuses on updates to the cabal repl command, allowing multiple components to be loaded at once into an interactive session. The work is being reviewed in Cabal MR #8726, and should be available in a future release of cabal-install.

Multiple Component Repl

When using cabal, most commands take a “target” which specifies which units you want to operate on. A command such as cabal build <target> will resolve all the units that the target <target> resolves to, and build all of them. The behaviour of the cabal repl command is different: you must specify a single unit to build.

Here are some common targets which you can specify when using cabal.

• all: Build all the locally defined components.
• exe:haskell-language-server: Build the executable called haskell-language-server
• lib:pkg-a lib:pkg-b: Build the local libraries pkg-a and pkg-b.
• src/Main.hs: Build the unit which src/Main.hs belongs to.

After enabling multi-repl, passing a target specification to cabal repl which resolves to multiple units will load all those units into a single repl session. For example:

cabal repl --enable-multi-repl lib:pkg-a lib:pkg-b

When the modules are compiled, the unit which they came from is listed next to the module name. The interactive herald in the build plan indicates that the library will be loaded into GHCi rather than being built like a normal package.

In order, the following will be built (use -v for more details):
 - pkg-a-0 (interactive) (lib) (first run)
 - pkg-b-0 (interactive) (lib) (dependency rebuilt)
Preprocessing library for pkg-a-0..
Preprocessing library for pkg-b-0..
GHCi, version 9.4.3: https://www.haskell.org/ghc/  :? for help
[1 of 2] Compiling Foo[pkg-a-0-inplace]
[2 of 2] Compiling Bar[pkg-b-0-inplace]
Ok, two modules loaded.

You will need to use at least ghc-9.4.1 in order to use multiple unit support. It’s advised to use ghc-9.4.5 or 9.6.1, in order to benefit from bug fixes.

Enabling Multi-repl

There are three ways to enable the multi-repl depending on how much you like it:

• Globally: Add multi-repl: True to your ~/.cabal/config file.
• Project-wide: Add multi-repl: True to your cabal.project file.
• Per-invocation: Pass --enable-multi-repl when invoking cabal repl.

A future cabal version is likely to enable multi-repl by default. For the time being, and due to the experimental nature of the command and lack of support in ghci for some features, the multi-repl feature is opt-in.

Closure Property for Multiple Home Units

For tools or libraries using the GHC API there is one very important closure property which must be adhered to:

Any dependency which is not a home unit must not (transitively) depend on a home unit.

For example, if you have three units p, q and r, and p depends on q which depends on r, then it is illegal to load both p and r as home units but not q, because q is a dependency of the home unit p which depends on another home unit r.

cabal will automatically enable loading of all units which are needed by the closure property (including non-local) packages. Given the previous example, if you specify on the command line cabal repl lib:p lib:q then lib:r will also be loaded into the same session as it is needed for the closure property.

Configuring and Promised Dependencies

The lowest-level interface which the Cabal library provides in order to build a package is the Setup.hs script. This consists of a normal Haskell file which depends on the Cabal library and can be executed in order to build the package. This is done, after compiling Setup.hs, via the following invocations:

./Setup configure
./Setup build

The configure phase checks to make sure that everything is in order so that when the build phase is run we know that all the environmental dependencies have already been provisioned by the user.

In the very old days, people would compile and run Setup.hs themselves in order to build a package, but these days, all the interactions with Setup.hs are managed by a higher-level build tool such as cabal-install, stack or nix. All of these tools ultimately call Setup.hs scripts.

The main technical change to enable the multi-repl was to modify the Setup.hs scripts to allow you to configure a package before all its dependencies are built. Now you can promise to Setup.hs that a certain dependency will be built by the time we attempt to build the unit. Since all units in a project are going to be built at the same time with one GHC invocation, they all need to be configured before anything else is built. So we just promise that all local packages will be built.

./Setup configure --promised-dependency=pkg-a

In addition to the configure and build commands, Setup.hs also provides a repl command which starts GHCi and loads a single component.

./Setup repl

This design is quite awkward because the Setup.hs scripts operate on a per-component basis. The interface is not aware of the existence of complicated multi-component projects, that is solely the domain of higher-level tools like cabal-install. Therefore, instead of starting the repl from the Setup.hs script, we need to start a multi-repl from cabal-install. However, the Setup.hs script is still responsible for computing the arguments we need to pass to GHC in order to compile that component. The solution is to allow the repl command to write its arguments into a file so that they can be collected later by cabal-install to correctly start a multi-component session.

./Setup repl --repl-multi-file=multi-args
# Arguments can be found in the `multi-args` directory.

This allows all the units in your project to be configured before any of them are built. After a project is configured, the Setup executable can be consulted to find out what options GHC would use to build the unit, and because we have promised to make sure things are built in the right order, we can supply these options to GHC in order to start a multi unit GHCi session.

HLS support for multiple home units

Zubin has already updated HLS to use native multiple home unit support for GHC-9.4.

The missing piece has been a mechanism to set up a multi component session which satisfies the closure property. Without such a mechanism, HLS would construct a multiple component session incrementally by adding units to a session as they are opened by the user. For a complicated project structure, users would have to very carefully load their files in the right order to get a session which worked correctly. Even worse, this doesn’t even work when a non-local package is needed to satisfy the closure property.

HLS consults cabal in order to set up a session: it invokes cabal repl and intercepts the final call to ghc which would start the repl. That command is then used as the options which are needed for the session in order to compile that unit.

Now that cabal repl supports creating a command line which specifies the options for multiple components at once, it makes sense to augment the HLS session loading logic to also understand these command lines in order to set up a whole multi-component session at once.

HLS now can understand and parse the kind of command line produced by a multiple component session. As a result:

• The correct session is initialised up-front. Loading any component in your local project will work seamlessly and quickly.
• The time taken to initialise a session is reduced, because no local dependencies are built before the session is started. All local components are configured before anything is built.
• Cabal ensures the closure property holds, even for non-local packages.

I have been testing this support when working on cabal and ghc, both projects with many local dependencies and the experience is much improved. In particular for cabal, the non-local hackage-security package is needed for the closure property but could never be loaded before. This made using HLS on cabal very error-prone because if you opened a file from the Cabal library and cabal-install library, you would break the session without a way to recover it. For ghc, it is a lifeline to be able to edit packages like template-haskell and see the changes ripple upwards through all the boot libraries and compiler.

Limitations

Now that there is a way to easily create and invoke a multi-repl session, users are probably going to run into limitations of the multi-repl.

Many features are not yet implemented because there is not a good way to change what the “active unit” of the repl session is. Some more careful thinking needs to be done to modify the GHCi interface in order to work nicely with multiple components in all situations.

At this time, the multi-repl is best used for interactive development situations where you want to use the repl to obtain fast-feedback about your project. We have made sure that the multi-repl works with ghcid for example.

Conclusion

Adding cabal repl support for multiple home units allows developers to easily interact with multiple home unit support in GHC. There are still limitations to the repl supported in multiple unit sessions, but as more users start using and wanting this feature we hope to expand the repl to work properly with multiple home units as well.

Well-Typed is able to work on GHC, HLS, Cabal and other core Haskell infrastructure thanks to funding from various sponsors. If your company might be able to contribute to this work, sponsor maintenance efforts, or fund the implementation of other features, please read about how you can help or get in touch.