I've recently been reading a lot of excellent essays on programming language design by Paul Graham. Paul and I agree about a number of things: (1) LISP is beautiful and powerful family of languages, even by modern standards, (2) all existing dialects of LISP are lacking a certain something, and (3) programmatic macros are a Good Idea.

What Are Programmatic Macros?

Programmatic macros, to put it simply, allow the programmer to add new control structures to a programming language. To do this, the programmer writes some code which runs in the compiler. This code transforms the new control structure into something the compiler already understands. A trivial example, in Common LISP:

(defmacro unless (condition &body body)
  ;; Transform (unless cond body...) into
  ;; (if (not cond) (progn body...)).
  `(if (not ,condition)
     (progn
       ,@body))

This defines a control structure unless in terms of the existing control structures if and progn.

Programmatic macros are similar to C preprocessor macros, but (1) they allow you to work directly with parse tree, instead of the raw textual source code, and (2) they allow you to run arbitrary code to produce the output. They're typically used to create domain-specific languages on top of an existing LISP dialect, which—like any domain-specific languages—make solving hard problems easier.

What are Hygienic Macros?

Hygienic macros are (essentially) macros which Do The Right Thing with local variable names. They're controversial because Doing The Right Thing makes it easier to write simple macros, and quite a bit harder to write extremely complex macros. Here's a very simple hygienic macro, in Scheme:

(define-syntax unless
  (syntax-rules ()
    ((unless condition body ...)
     (if (not condition) (begin body ...)))))

Unlike the above example, this definition of unless doesn't get confused by (say) a local redefinition of not. Similar safegaurds would apply to any local variables defined within the macro's expansion—they wouldn't get confused with local variables in body. In other words, the compiler knows a lot more about the macro expansion, and is doing some fairly complicated transformations behind the scenes.

Why I'm Convinced Hygienic Macros Are Better

After playing around with DrScheme—a truly amazing Scheme environment—I'm definitely convinced that hygienic macros are worth the added difficulties they inflict upon the authors of complex macros.

The other day, I implemented a begin/var macro in DrScheme because our users were getting sick of Scheme's cumbersome syntax for declaring variables. begin/var can be used as follows:

(define (silly-function)
  ;; Returns the list (15 20).
  (begin/var
    (var x 10)
    (set! x (+ x 5))
    (var y 20)
    (list x y)))

var works like Perl's my declaration—it declares a new local variable which can be seen until the end of the current scope. The implementation of begin/var requires 17 lines of slightly crufty macro code.

This macro was marginally harder to write in DrScheme than it would be in Common LISP. But because DrScheme's hygienic macro system has deep knowledge of how my macro works, my macro is extremely well-integrated into the environment: syntax highlighting works correctly, automatic variable renaming works correctly, cross-referencing works correctly, and so do all the other slick DrScheme IDE features. In other words, hygiene doesn't just protect you against simple bugs such as variable capture—it allows your code to formally analyzed by tools that simply could not exist for Common LISP.

Counterarguments (and Implications for Arc)

Non-hygienic macros might still be a good choice for languages which (1) are aimed at advanced macro hackers and (2) won't ever require particularly advanced IDE support. Paul Graham's Arc language definitely fits criteria (1), but I think his dreams of really excellent profiling tools may run counter to criteria (2).