Why there are no macros

A macro system promises to let a library extend the syntax of the language it is written in. TorbScript extends itself with functions instead, and this page argues for why that is not a smaller version of the same idea but a different one, forced by how names are resolved.

The decision#

The language is extended by functions, not by macros or annotations.

• Control structures are functions with closure or lazy parameters (do, unless, retry, test).
• DSLs are functions with receiver closures, and operators are traits.
• Code that has to be read instead of run - a query provider, assert, a validation rule - takes an Expression<Value> parameter, which hands the function the typed tree of an already-resolved, already-checked expression.
• There are no AST macros and no annotations.

fn unless(condition: Bool, action: () => Void) {
  if !condition {
    action()
  }
}

const items: List<Int> = []
unless items.isEmpty() { print "not empty" }
print "done"

Why#

Because name resolution in TorbScript needs types, and a macro would have to run before they exist. A receiver closure resolves a bare name against the type of self; a call resolves an argument label against a parameter list; extend decides which members are visible by which module imported what. All of that needs the declarations it resolves against to already be typed. An AST macro, by contrast, is defined to run on the syntax tree before any of that - it cannot know what a name will mean, because names have not been given meaning yet. Running macros after resolution would not be a macro system: nothing would be left to rewrite that the type checker had not already fixed in place.

Because Expression<Value> is the opposite of a macro, and covers what a macro is typically reached for. It reads code after it has been resolved and type-checked, alongside the ordinary value the same expression produces - see Reading code instead of running it. A query provider that wants to translate users.filter { _.age >= 18 } into SQL gets the typed tree of the closure, not the raw tokens a macro would see, and it cannot generate any code back into the program - only read what is already there.

Because control structures and DSLs already reach where most macro systems are reached for, without a second language inside the language. unless, retry and test all look like keywords and are ordinary functions with closure parameters; route "/health", to: "health" inside a configuration file is an ordinary command call on a method. Rust's macro_rules! and procedural macros exist largely to build exactly these two things - a custom control structure and a small DSL - and TorbScript reaches both through command calls and receiver closures instead.

Because a macro is a second syntax that every tool has to understand, and a function is not. A formatter, a highlighter and a language server all have to know a macro exists before they can format, highlight or complete code that uses it; a function needs none of that, because it is already the thing every tool understands.

What was rejected#

• AST macros (macro_rules!, procedural macros, Lisp-style defmacro). Rejected because they run before name resolution, which this language's resolution rules make impossible to reconcile - see the argument above.
• Annotations (@Annotation, #[derive(...)]) as a general mechanism. Rejected because an annotation is a request to run code the compiler does not check the shape of; what an annotation is typically reached for - Equals, Hash, Show, Encode, Decode - is generated by the compiler itself instead, following one fixed rule per trait rather than an open-ended plugin.
• A deriving-style syntax. Not adopted as a separate mechanism: with Equals, Hash at the declaration already says which traits are generated, so a second syntax for the same fact was not needed.

Consequences#

What a macro would generate is either already generated by the compiler, or written as a function. Equals/Hash/Show/copy need no annotation because the compiler derives them from the fields of a type automatically; a control structure a project wants is a function with a closure parameter, not a macro invocation.

A library cannot introduce new syntax, only new names. use brings a function, a type or a trait into scope, and calling it always looks like calling a function - a command call, a receiver closure, a trailing closure - never like a construct the language did not already have. A reader who knows the grammar of TorbScript can read any file, because nothing in it can rewrite that grammar.

One possible future step stays open without needing macros: compile-time functions that produce types (type Row = Schema.rowOf("users.sql")), evaluated by the same interpreter that already exists for compile-time constants - as an addition to declarative generics, never their replacement. Nothing about Expression<Value> or command calls has to change for that to be added later.

Related#

• Reading code instead of running it - what Expression<Value> hands a function, and what it cannot.
• Control structures are functions - do, unless, retry, and how to add one.
• Why a call is written as a command - the syntax that makes a function read like a keyword.
• Why there is no reflection - the same "no code that runs ahead of the type checker" argument applied to values instead of syntax.