Random Hacks: Map fusion: Making Haskell 225% faster

"Map fusion: Making Haskell 225% faster" by Aaron McDaid

Sat, 30 May 2009 16:00:33 +0000

Jim, GHC can’t automatically know that such a rule is valid. For example, (filter f . filter g) is not equivalent to (filter (f.g))

Each author’s implementation of functions called treeMap could be quite different, and it would be difficult to improve GHC such that it could correctly know which implementations could be optimized.

"Map fusion: Making Haskell 225% faster" by Lemming

Fri, 04 Apr 2008 02:42:12 +0000

I encountered the following problem which will hurt us in case of generalized monads as sketched in http://www.randomhacks.net/articles/2007/03/15/data-set-monad-haskell-macros and of data structure like StorableVector http://code.haskell.org/~sjanssen/storablevector that requires a constraint on the element types. There it makes a difference if you separately map and fold or if you do it in one go. In the first case you need a Storable constraint for intermediate result type, too. That is, these constraints make implementation issues observable via the type signature.

"Map fusion: Making Haskell 225% faster" by Sam Bronson

Tue, 17 Apr 2007 08:35:59 +0000

The ‘x’ would not be calculated until it is read… however, all sorts of intermediate heap objects are generated when you don’t actually fuse things, which means that the GC has to be run more. Fusing maps together also gives the compiler a chance to merge the mapped functions, etc.

I wonder what the codata for a tree like this looks like…

"Map fusion: Making Haskell 225% faster" by Jim

Tue, 27 Mar 2007 18:07:51 +0000

Why doesn’t GHC automatically have these optimizations thanks to lazy evaluation? It seems like the ‘x’ that treeFold sees shouldn’t be calculated until it is read, at which time it would simply apply the functions from treeMap. Does GHC internally produce a million node tree with “thunks” first or am I misunderstanding something?

"Map fusion: Making Haskell 225% faster" by Eric Kidd

Sat, 10 Feb 2007 21:05:11 +0000

An excellent point by Neel Krishnaswami:

For [fusion] to really work, the optimization has to be part of the performance model of the language, and when it is invoked has to be predictable by the programmer. (An old, familiar example of this is mandatory tail-call optimization in functional languages, which gives you asymptotic space savings and which is very predictable in its effect.)

Enforcing purity is actually an important part of the story for fusion, because it simplifies the performance model for the programmer. Even if you could do a very fancy whole program dataflow analysis to detect side effects, a programmer couldn’t easily reason about when the analysis succeeded or failed, so the optimization would be less useful.

Read the rest for more good insights.

"Map fusion: Making Haskell 225% faster" by Eric Kidd

Sat, 10 Feb 2007 20:54:48 +0000

It’s true, you could achieve quite a bit with manual annotations of purity. But to get any kind of consistent optimization, you’d need to use a lot of them, and do so correctly. You’d also need to write your code in a largely functional style, or have some scary loop-fusion support.

An alternate approach would be to use a compiler with robust inter-procedural analysis. LLVM, for example, can do a lot of optimizations along these lines.

But in both cases, we’re comparing my toy example, above, to a serious production compiler. The rabbit-hole goes much deeper than this.

Once you start treating programs like an algebra, you can do all kinds of crazy things: automatically infer rewrite rules using Wadler’s free theorems, optimize tricky concurrent code, exploit rank-2 types—the list goes on and on.

Now, for many programmers, functional purity might seem a ridiculously high price to pay. But there’s a surprisingly large number of functional languages out there already: query languages like SQL, OQL and LINQ; vertex and fragment shaders on your GPU; and even some hardware modeling languages (if you squint at them right).

All of these can benefit, today, from algebraic optimizers. But in each case, the price of admission is a proof of purity, whether performed by the programmer, inferred by the compiler, or enforced by the language.

I strongly suspect (but cannot prove) that supporting purity at the language level will take you much further than either of the other two approaches. And you don’t have to give up state to get there; at worst, you need better support for monads.

"Map fusion: Making Haskell 225% faster" by Paul Prescod

Sat, 10 Feb 2007 17:20:01 +0000

I disagree with the premise that you need the compiler to validate the side-effect-freeness of your functions. I would rather have the programmer make explicit assertions that there are no side-effects that matter.

I would buy the argument that it might be valuable to have a compiler that can recognize these optimization opportunities automatically more often, but the actual assertion was that you need purity to get the optimization at all. I don’t think that’s been proven.

Further discussion here.

"Map fusion: Making Haskell 225% faster" by Matthew

Sat, 10 Feb 2007 12:56:20 +0000

This kind of thing is why I think Haskell and lazy pure-functional programming is the future (of FP, and of programming, hopefully). Referential transparency allows this kind of logical separation of concerns and at the same time it also allows the specification of formal, provable, rules to transform programs.

The biggest benefit is that often the most naturally written code is also the best performing; while in strict languages there is usually an inversely proportional relationship between beautiful code and good performance.

The fact that GHC does so much of this work behind the scenes, and so effectively, is amazing already. Of course, there is plenty of room for improvement.

Map fusion: Making Haskell 225% faster

Eric Kidd — Sat, 10 Feb 2007 09:55:00 +0000

Or, how to optimize MapReduce, and when folds are faster than loops

Purely functional programming might actually be worth the pain, if you care about large-scale optimization.

Lately, I’ve been studying how to speed up parallel algorithms. Many parallel algorithms, such as Google’s MapReduce, have two parts:

First, you transform the data by mapping one or more functions over each value.
Next, you repeatedly merge the transformed data, “reducing” it down to a final result.

Unfortunately, there’s a couple of nasty performance problems lurking here. We really want to combine all those steps into a single pass, so that we can eliminate temporary working data. But we don’t always want to do this optimization by hand—it would be better if the compiler could do it for us.

As it turns out, Haskell is an amazing testbed for this kind of optimization. Let’s build a simple model, show where it breaks, and then crank the performance way up.

Trees, and the performance problems they cause

We’ll use single-threaded trees for our testbed. They’re simple enough to demonstrate the basic idea, and they can be generalized to parallel systems. (If you want know how, check out the papers at the end of this article.)

A tree is either empty, or it is a node with a left child, a value and a right child:

data Tree a = Empty
            | Node (Tree a) a (Tree a)
  deriving (Show)

Here’s a sample tree containing three values:

tree = (Node left 2 right)
  where left  = (Node Empty 1 Empty)
        right = (Node Empty 3 Empty)

We can use treeMap to apply a function to every value in a tree, creating a new tree:

treeMap :: (a -> b) -> Tree a -> Tree b

treeMap f Empty = Empty
treeMap f (Node l x r) =
  Node (treeMap f l) (f x) (treeMap f r)

Using treeMap, we can build various functions that manipulate trees:

-- Double each value in a tree.
treeDouble tree = treeMap (*2) tree

-- Add one to each value in a tree.
treeIncr tree   = treeMap (+1) tree

What if we want to add up all the values in a tree? Well, we could write a simple recursive sum function:

treeSum Empty = 0
treeSum (Node l x r) =
  treeSum l + x + treeSum r

But for reasons that will soon become clear, it’s much better to refactor the recursive part of treeSum into a reusable treeFold function (“fold” is Haskell’s name for “reduce”):

treeFold f b Empty = b
treeFold f b (Node l x r) =
  f (treeFold f b l) x (treeFold f b r)

treeSum t = treeFold (\l x r -> l+x+r) 0 t

Now we can double all the values in a tree, add 1 to each, and sum up the result:

treeSum (treeIncr (treeDouble tree))

But there’s a very serious problem with this code. Imagine that we’re working with a million-node tree. The two calls to treeMap (buried inside treeIncr and treeDouble) will each create a new million-node tree. Obviously, this will kill our performance, and it will make our garbage collector cry.

Fortunately, we can do a lot better than this, thanks to some funky GHC extensions.