GHC activities report: April–May 2021

Ben Gamari, Matthew Pickering, Andreas Klebinger, Zubin Duggal, Alfredo Di Napoli, Adam Gundry – Monday, 14 June 2021

all community ghc ghc-activities-report well-typed  

This is the sixth edition of our GHC activities report, which is intended to provide regular updates on the work on GHC and related projects that we are doing at Well-Typed. This edition covers roughly the months of April and and May 2021.

You can find the previous editions collected under the ghc-activities-report tag.

A bit of background: One aspect of our work at Well-Typed is to support GHC and the Haskell core infrastructure. Several companies, including IOHK and Facebook – and, new, GitHub (via the Haskell Foundation; see below), are providing us with funding to do this work. We are also working with Hasura on better debugging tools. We are very grateful on behalf of the whole Haskell community for the support these companies provide.

If you are interested in also contributing funding to ensure we can continue or even scale up this kind of work, please get in touch.

Of course, GHC is a large community effort, and Well-Typed’s contributions are just a small part of this. This report does not aim to give an exhaustive picture of all GHC work that is ongoing, and there are many fantastic features currently being worked on that are omitted here simply because none of us are currently involved in them in any way. Furthermore, the aspects we do mention are still the work of many people. In many cases, we have just been helping with the last few steps of integration. We are immensely grateful to everyone contributing to GHC. Please keep doing so (or start)!

Team

Currently, Ben Gamari, Andreas Klebinger and Matthew Pickering are working primarily on GHC-related tasks, and have recently been joined by Zubin Duggal, who started at Well-Typed in May. In addition, Alfredo Di Napoli, Alp Mestanogullari and Adam Gundry have been doing some work on GHC in the last two months, next to other projects they are working on.

New sponsor

We are delighted to announce that GitHub have joined the companies that sponsor our work on GHC, via an earmarked donation to the Haskell Foundation. Thanks to both GitHub and the Haskell Foundation for working out the details, and thanks to all the other companies that are making all of our work described here possible.

The GitHub contribution succeeds earlier funding we have had from Microsoft Research. The funding from Microsoft Research for doing work on GHC was the very first contract that Well-Typed as a company ever had, and lasted more than ten years. We would like to take the opportunity to also thank Microsoft Research and Simon Peyton Jones once more for all the support throughout the years.

Release management

We have been continuing to push ahead on three fronts:

  • Having over 60 backports since 8.10.4, the 8.10.5 release has been prepared and cut by Zubin. Due to a regression possibly affecting Haskell Language Server work has started on a follow-up 8.10.6 release.

  • Backports continue on the 9.0 branch, with further work being done to identify and fix a variety of runtime performance regressions seen in 9.0.1

  • Another alpha release has been cut from the 9.2 branch and work continues on sorting its remaining release issues.

Compiler error messages refactoring

  • Alfredo continued his work with Richard Eisenberg on the errors and warnings messages (now aptly rebranded as “Diagnostics”) and their refactoring. After several merged MRs (most notably !5116, !5410, !5533, !5509, !5207, !5172, !5580, !5569, !5528), the work reached an important milestone: now GHC has a diagnostic hierarchy in the form of Haskell types, with an umbrella GhcMessage at the root. Plugin authors can embed custom diagnostic messages via the presence of a GhcUnknownMessage constructor.

    Not all existing GHC messages have been ported yet, due to their sheer number, and MR !5719 should complete the conversion for the existing desugarer messages. A summary of the outstanding work can be found over at #19905.

Frontend

  • Andreas fixed some erroneous warnings about missed specialisations by GHC (#19586, !5359).

  • Matt has been continuing to work with Divam Naruala to refactor and modernise the driver code which powers --make mode. This month two big patches have been merged. Firstly, some transitive information about modules and packages was removed from interface files (!5688) which leads to modifying files causing less recompilation.

    Secondly, the recompilation logic in the driver was updated (!5661) to fix a large number of open tickets to do with recompilation checking. The patch also simplifies internal logic in the compiler so that modifying the driver in future should be easier.

    We are now moving onto some changes which would allow more parallelism when building programs.

Profiling and debugging

  • Andreas started looking into changing GHC’s ABI to use the machine stack register for the heap allocated Haskell stack. This is an old idea (#8272, ghc-proposals!17). If things work out this should allow common tools like perf/gdb to understands Haskell call stack. There is a branch in GHC already which can build a working compiler that both uses, and was built with the new ABI. However, there are still some bugs causing segfaults to squash before we can fully evaluate the new ABI for general use in terms of impact on runtime, code size and easier profiling/debugging using stock tools.

Compiler performance

  • Andreas changed the GHC-internal implementations for bit shifts (!5428). This should improve GHC performance ever so slightly by skipping some runtime checks.
  • Andreas also looked into the implementation for worker/wrapper and identified some inconsistencies in #19818 and #19766. There is now a small fix for the former issue (!5710), and GHC contributor Sebastian Graf is working on resolving the later.
  • Andreas also discovered an operation by the simplifier that turned out to be surprisingly costly (#19820). We might be able to mitigate this for a lot of cases in the future, but for now this has merely been documented.
  • Adam continued working on the performance of GHC when compiling programs making heavy use of type families (#8095). He is exploring a number of possible improvements: changing the representation of type family coercions to be more efficient (!5286), reducing the cost of coercion optimization (!5863), and making GHC rewrite type families only when needed (#18965).
  • Ben has been working on improving the performance of the compiler’s internal class- and family-instance environments. The performance of these data structures are critical to typechecker performance in the presence of a large number of instances, yet are currently quite naive. By moving to a trie structure we believe we can significantly improve typechecking performance in some programs (#19703).

Runtime performance

  • Tag Inference (!5614) is now in the review stage. This patch should provide ~1% improved runtime for most programs, which big improvements for certain special cases. This impact is particularly large in small loops like containers’ lookup operation.
  • Enabling strict dictionaries for -O2 (!2575) is waiting to be merged.
  • Ben has been working on the various runtime performance regressions reported in 9.0.1 (#19557, #19727, #19790).

Compiler correctness

  • Andreas identified #19768 which caused GHC master to crash when dumping core output in some cases. The issue was promptly fixed by Simon Peyton Jones.
  • Similarly a recent refactoring caused master to panic under rare circumstances. Andreas documented this in #19824 and a fix has been merged in !5722.
  • Ben has been investigating a rare compiler determinism bug caused by a surprising interaction between rule visibility and inlining behavior (#19725). With the culprit now clear, the current challenge is identifying an fix which maintains determinism while minimally affecting compilation time.
  • Ben diagnosed and fixed a code generation bug affecting unboxed sums with unlifted fields (#19645).

Compiler functionality and language extensions

  • The documentation about what runtime representation means for DataCons in the GHC API was a bit unclear. Andreas opened #19789 where this was then discussed and resolved in !5694.

CI and infrastructure

  • Ben has migrated the GitLab instance to a new machine and upgraded GitLab to 13.12.
  • Matt worked on adding more monitoring to CI pipelines and other internal infrastructure.
  • Matt has been working on improving the CI pipeline speed by speeding up some jobs on the critical path.