The new implementation
For about the last year we have been working on a new implementation of Cloud Haskell. This is the same idea for concurrent distributed programming in Haskell that Simon Peyton Jones has been telling everyone about, but it's a new implementation designed to be robust and flexible.
The summary about the new implementation is that it exists, it works, it's on hackage, and we think it is now ready for serious experiments.
Compared to the previous prototype:
- it is much faster;
- it can run on multiple kinds of network;
- has backends to support different environments (like cluster or cloud);
- has a new system for dealing with node disconnect and reconnect;
- has a more precisely defined semantics;
- supports composable, polymorphic serialisable closures;
- and internally the code is better structured and easier to work with.
The key packages on hackage are:
We will also release a backend for the Windows Azure cloud platform later this month.
I gave a talk at the Haskell Implementors Workshop last month with lots more details about the new implementation. The slides and video are available:
A tutorial on Communication Patterns in Cloud Haskell
Starting tomorrow we're going to do a series of blog posts about using Cloud Haskell. It's a mini-tutorial, with examples that you can try and exercises to extend the code.
We'll be focusing on patterns for distributing work amongst a number of machines in a network. We'll start with very simple distributed patterns and work up to map-reduce and a slight generalisation of map-reduce. We'll also look closely at performance and resources like memory and network latency.
A gentle introduction
If you want a little background reading to help you follow what we're going to be talking about, here's some recommendations:
- A Cloud Haskell Appetiser, part of the Parallel Haskell Digest series
- Slides from a talk I gave earlier this year which has more motivation for why this approach to distributed programming is interesting in the first place
- A video presentation by Simon Peyton Jones explaining the Cloud Haskell idea
- Towards Haskell in the Cloud, the original paper from last year by Jeff Epstein, Andrew Black, and Simon Peyton Jones
More technical details
For even more technical details, see the developer documentation in the source repo and wiki
- source repo and README
- distributed process overview
- Network.Transport lots of details on the network transport layer
- Draft of the formal semantics
There is also the parallel-haskell google group for discussion and release announcements.