Chapter 3. Syntactic sugar

Syntactic sugar ​

Core concepts ​

• Core calculus/language: The minimal set of constructs that the implementation handles directly.

• Surface language: The language programmers actually write in, which may have many conveniences.

• Syntactic sugar: Extra constructs in the surface language that make programming "sweeter" (more convenient). These are not handled directly but translated to core constructs.

• Desugaring: The process of removing sugar - translating Surface language programs down to the Core calculus/language.

• Desugarer: The program that performs desugaring. Technically a compiler from surface to core, but since core is a subset of surface, it's a special case.

• Macro: A Syntactic abstraction that rewrites program source into other program source before evaluation.

• Macro expansion: The process of applying macro transformations to produce core language code.

• Macro system: A facility that provides program rewriting capabilities directly to the programmer.

• Most languages have Syntactic sugar (built into the compiler).

• Very few languages have Macro systems (expose rewriting to programmers).

Why syntactic sugar matters ​

The redundancy problem: Many language constructs overlap in functionality.

Example - for and while in C:

for(x = 0; x < 10; x++) {
    sum += x;
}

// is exactly the same as:

x = 0;
while (x < 10) {
    sum += x;
    x++;
}

General pattern:

for(INITIAL; CONDITIONAL; UPDATE) {
    BODY;
}

// desugars to:

INITIAL;
while (CONDITIONAL) {
    BODY;
    UPDATE;
}

The implementor's dilemma:

• while implementation requires recursive calls, iteration, checking, bookkeeping, scope management.

• Without desugaring, all that work must be duplicated for for.

• Bug fixes must be applied to both implementations.

The solution: Implement while once, translate for into while.

Why have both constructs?

• Each is convenient for different purposes.

• for signals a certain stylistic pattern that would be harder to spot in raw while code.

• It adds to programmer vocabulary without adding to implementor pain.

Real-world desugaring examples ​

• and/or: Desugar into nested ifs.

• JavaScript o.x: Desugars into o["x"].

• x += y: Sugar for x = x + y.

• Python +: Desugars into method call __add__.

• Python dunder methods: __add__, __str__, etc. A whole "protocols" programming style evolved around these.

• List comprehensions (Haskell, Python): Desugar into function/method calls.

Awareness: If you don't notice desugaring, that's part of the point - you feel like working with a larger syntax than the implementor manages.

Implementation approaches to desurgaring ​

Approach 1 - AST rewriting:

• Parse program normally into full AST.

• Rewrite AST into a subset of itself (core AST).

Approach 2 - Macro systems (Parenthetical language):

• Two levels of parsing: coarse parenthetical level, then finer AST level.

• Rewrite at the parenthetical level before AST parsing.

• Internal AST only needs to cover core language.

Pitfall in real implementations: If you make an error using for but the error is reported in terms of while, you'd be confused. Modern desugaring systems (like Racket's) have special support to handle error reporting correctly.

Macros by example ​

Setup ​

Switch to #lang racket for macro examples. The restrictions and types of plait/plai-typed can interfere with macro writing.

#lang racket

Example 1: a strict conditional ​

Goal: Racket's if is truthy/falsy. We want a strict-if that accepts only Booleans.

First attempt - as a function:

(define (strict-if C T E)
  (if (boolean? C)
      (if C T E)
      (error 'strict-if "expected a boolean")))

Testing:

(strict-if true 1 2)   ;; => 1
(strict-if 0 1 2)      ;; => error (good!)

Pitfall: This fails because of eager evaluation:

(strict-if true 1 (/ 1 0))  ;; => error! Division by zero
(if true 1 (/ 1 0))         ;; => 1 (correct)

The whole point of a conditional is to avoid evaluating one branch. Functions evaluate all arguments first - we need macros.

Correct solution - as a macro:

(define-syntax strict-if
  (syntax-rules ()
    [(strict-if C T E)
     (if (boolean? C)
         (if C T E)
         (error 'strict-if "expected a boolean"))]))

Macro components:

• define-syntax: Declares a new piece of syntax (not a function).

• syntax-rules: Introduces a pattern-matcher. The () is required (for now, ignore its meaning).

• Pattern: (strict-if C T E) - matches input and binds C, T, E to corresponding source code.

• Output template: The expression that replaces the macro use, with pattern variables substituted.

Expansion example:

(strict-if true 1 (/ 1 0))

;; expands to:

(if (boolean? true)
    (if true 1 (/ 1 0))
    (error 'strict-if "expected a boolean"))

Now (/ 1 0) is only evaluated if the if chooses that branch.

DrRacket tip: Use the Macro Stepper to see step-by-step expansion. Set "Macro hiding" to "Standard" at bottom-left.

Awareness: The Macro Stepper is not an evaluator. It shows expansion steps, not evaluation steps. Runtime errors won't appear - only syntax errors.

Example 2: local binding ​

Goal: Define my-let1 that binds one variable:

(my-let1 (x 3) (+ x x))  ;; => 6

Why can't let1 be a function?

• Would try to evaluate (x 3) as an argument.

• (x 3) looks like an application, but x isn't bound.

• No meaningful "value" - its job is to bind x.

Terminology: New pieces of syntax are called constructs or special forms (forms with special binding/evaluation rules).

Solution - using Left-left-lambda:

(define-syntax my-let1
  (syntax-rules ()
    [(my-let1 (var val) body)
     ((lambda (var) body) val)]))

Key insight: What does let1 do?

• Binds a name to a value.

• Evaluates body in extended environment.

What else does this? Functions! A function:

• Binds formal parameter to argument value.

• Evaluates body in extended environment.

• Evaluates body when applied.

So let1 is an Immediately invoked function expression (IIFE).

Terminology: This pattern is called Left-left-lambda (for the left parentheses). JavaScript popularized it as Immediately invoked function expression (IIFE) due to scope problems in early versions.

Exercise pitfall - swapping parts:

(define-syntax my-let1
  (syntax-rules ()
    [(my-let1 (var val) body)
     ((lambda (var) val) body)]))  ;; WRONG! val and body swapped

(my-let1 (x 3) (+ x x))
;; expands to: ((lambda (x) 3) (+ x x))
;; x is unbound in (+ x x)!

Example 3: binding multiple locals ​

Goal: Bind many names at once, like Racket's let:

(my-let2 ([x 3] [y 4]) (+ x y))  ;; => 7

Pattern: The function takes multiple formals, applied to multiple actuals.

Syntax: Use ellipsis (...) for sequences of arbitrary length:

(define-syntax my-let2
  (syntax-rules ()
    [(my-let2 ([var val] ...) body)
     ((lambda (var ...) body) val ...)]))

Reading the pattern:

• ([var val] ...) matches zero or more var-val pairs.

• (var ...) in output produces all the vars.

• (val ...) in output produces all the vals.

Example 4: multi-armed conditional ​

Goal: Define my-cond with multiple question-answer arms:

(define (sign n)
  (my-cond
    [(< n 0) "negative"]
    [(= n 0) "zero"]
    [(> n 0) "positive"]))

Design:

• Peel off first question-answer pair.

• If question is true, evaluate answer.

• Otherwise, recur on remaining pairs (smaller instance of my-cond).

• If all questions exhausted, produce error.

Solution - recursive macro:

(define-syntax my-cond
  (syntax-rules ()
    [(my-cond)
     (error 'my-cond "should not get here")]
    [(my-cond [q0 a0] [q1 a1] ...)
     (if q0
         a0
         (my-cond [q1 a1] ...))]))

Key insight: [q0 a0] peels off the first pair. [q1 a1] ... captures all remaining pairs. The macro recurs on syntax.

More on macros ​

Definitional convenience ​

The underscore pattern: Don't repeat the macro name in the pattern - use _ instead:

(define-syntax unless
  (syntax-rules ()
    [(_ cond body ...)
     (if (not cond)
         (begin body ...)
         (void))]))

Usage:

(unless false
  (println 1)
  (println 2))

Awareness: _ and repeating the macro name do slightly different things in subtle situations. You should always use _.

Hygiene ​

The capture problem - consider:

(let ([not (λ (v) v)])  ;; redefine `not` to be identity!
  (unless false
      (println 1)
      (println 2)))

This seems to expand to:

(let ([not (λ (v) v)])
  (if (not false)  ;; uses the wrong not?
      (begin
        (println 1)
        (println 2))
      (void)))

If the outer not captured the inner not, the conditional would be inverted!

The solution - Hygienic macro:

• Variables are more than just names.

• Variables record binding information that keeps names introduced in different settings separate.

Conceptually, it's like using colors:

;; Original (red = user's not, blue = macro's not):
(let ([not (λ (v) v)])    ;; red not
  (unless false ...))

;; After expansion:
(let ([not (λ (v) v)])    ;; red not
  (if (not false)         ;; blue not (different!)
      ...))

The macro expander can use distinct fresh names internally: not1, not2, etc.

Hygiene is an analog of static scoping for macros:

• Macro writers can use any name without worrying about use context.

• Macro users can use any name without worrying about macro internals.

Pitfall: The C preprocessor does not have hygiene. Name capture is a real danger there.

Truthy/falsy idiom: or ​

First attempt:

(define-syntax or-2
  (syntax-rules ()
    [(_ e1 e2)
     (if e1 true e2)]))

Works for basic cases:

(or-2 true false)   ;; => true
(or-2 false false)  ;; => false
(or-2 false true)   ;; => true

Problem: Loses useful return values:

(member 'y '(x y z))  ;; => '(y z), a truthy value

(or-2 (member 'y '(x y z)) "not found")  ;; => true (lost the list!)

Second attempt - return the value:

(define-syntax or-2
  (syntax-rules ()
    [(_ e1 e2)
     (if e1 e1 e2)]))

Now (or-2 (member 'y '(x y z)) "not found") returns '(y z).

Macro definition hazard: duplication ​

Pitfall: The second or-2 has a subtle hazard:

(or-2 (print "hello") "not found")
;; prints "hello" TWICE!

Why? Macros are syntactic abstractions. (if e1 e1 e2) copies e1 twice into the output. This:

• Causes computation to happen twice.

• Blows up program size.

• Is especially problematic for side effects.

Solution - bind to a local variable:

(define-syntax or-2
  (syntax-rules ()
    [(_ e1 e2)
     (let ([v e1])
       (if v v e2))]))

Awareness: The let works because e1 is guaranteed to be evaluated anyway (it's the condition). If we put let where something might not be evaluated, we'd force evaluation incorrectly.

Hygiene for local variables ​

Question: Does hygiene protect local variables too?

(let ([v 1])
  (or-2 false v))

The macro introduces its own v. Does it capture the user's v?

With hygiene (what Racket does):

(let ([v0 1])
  (let ([v1 false])
    (if v1 v1 v0)))
;; => 1 (correct!)

Without hygiene (what would be wrong):

(let ([v 1])
  (let ([v false])
    (if v v v)))
;; => false (wrong!)

Key insight: Hygiene works for all names - built-in functions, local variables, everything. You can use names freely in macros just as in functions.

Generalizing to n-ary ​

Goal: Define orN that takes any number of arguments.

First attempt:

(define-syntax orN
  (syntax-rules ()
    [(_ e1 e2 ...)
     (let ([v e1])
       (if v v (orN e2 ...)))]))

Problem:

(let ([v true])
  (orN false v))
;; Error: orN: bad syntax in: (orN)

The pattern (_ e1 e2 ...) requires one or more arguments. When recursion reaches zero arguments, no pattern matches.

Solution - add base case:

(define-syntax orN
  (syntax-rules ()
    [(_) false]
    [(_ e1 e2 ...)
     (let ([v e1])
       (if v v (orN e2 ...)))]))

Exercise awareness: Why not just use (_ e ...) for zero-or-more? Try it - you'll find it doesn't work straightforwardly because you need to handle the sequence differently.

Summary table ​