Lamp Scheme

A Scheme implementation in Zig, built for embedding.

Two execution modes share one frontend: a register-based bytecode VM, and an AOT compiler that emits Zig source and hands it to LLVM. Both pass the same suites, and a differential corpus asserts they agree byte-for-byte.

• R5RS — 267/267, VM and AOT
• R7RS-small — 1066/1066, VM and AOT; one known failure remains, a 1-ULP libm difference in tan
• AOT — geometric mean 0.86x against Chez 10.4 on the R7RS benchmark suite

Other things worth knowing: the full real numeric tower — fixnums that promote to bignums on overflow and exact rationals, so (/ 1 3) is 1/3 and (exact 0.75) is 3/4 — hygienic syntax-rules macros, call/cc and dynamic-wind, proper tail calls, full Unicode via zg, a generational mark-sweep GC over typed arenas, a source-level debugger built for DAP integration, and native value types that compile to unboxed Zig structs.

Build

Requires Zig 0.16.

zig build                          # debug
zig build -Doptimize=ReleaseFast   # release
zig build test

Use

./zig-out/bin/lamp                    # REPL
./zig-out/bin/lamp script.scm [args]  # script; args land in *args*

> (define (fib n) (if (< n 2) n (+ (fib (- n 1)) (fib (- n 2)))))
> (map fib '(1 2 3 4 5 6 7 8 9 10))
(1 1 2 3 5 8 13 21 34 55)

To compile ahead of time, depend on lamp and use its build helper. The whole of your build.zig:

const std = @import("std");
const lamp = @import("lamp");

pub fn build(b: *std.Build) void {
    const target = b.standardTargetOptions(.{});
    // No .target here: this dependency supplies the Scheme compiler, which
    // has to run on your machine. Your target goes in .target below.
    const dep = b.dependency("lamp", .{});

    b.installArtifact(lamp.addSchemeExecutable(b, dep, .{
        .name = "myapp",
        .source = b.path("app.scm"),
        .target = target,
    }));
}

Then zig build. Cross-compilation (-Dtarget=), optimization mode, and linking your own Zig modules come from the build system rather than from flags lamp would have to re-invent one at a time — .native_types and .imports hang off the same options struct. Inside the lamp checkout, zig build aot-compile -Dsource=program.scm runs the same helper.

Libraries can live in files. (import (my utils)) looks for my/utils.sld then my/utils.scm under each search root:

lamp -I lib app.scm                  # repeatable, highest precedence
LAMP_LIBRARY_PATH=lib:vendor lamp app.scm

An embedder adds roots with vm.addLibraryPath(dir). With no roots configured nothing is loaded from disk, so import sees only what is registered from Zig or defined in the source.

--emit-zig prints the generated Zig instead of building it, which is useful for reading what the compiler produced:

lamp --emit-zig program.scm | less

That output is a module, not a program — it exports register and initGlobals and has no main — so it has to be linked against a host (src/aot_host.zig) and the lamp module graph. addSchemeExecutable is that wiring.

Forms the AOT path cannot compile natively — call/cc outside escape position, dynamic-wind, guard, case-lambda — fall back to embedded bytecode, so any program compiles.

Performance

Apple M1 Pro, identical algorithms and iteration counts across implementations. Full tables and the optimization history are in bench/HISTORY.md.

AOT vs Chez 10.4 (--optimize-level 2): geometric mean 0.80x, 15 of 29 benchmarks faster. Where it wins and loses:

Benchmark	Lamp	Chez	Ratio
mbrot	0.19s	1.22s	0.16x
sumfp	0.28s	1.45s	0.19x
fibfp	0.42s	1.69s	0.25x
cpstak	0.21s	0.56s	0.36x
destruc	1.71s	0.87s	1.96x
sboyer	1.07s	0.56s	1.89x
lattice	4.55s	2.47s	1.84x

Treat single numbers here with care: re-running the identical binary moves a benchmark by 4.7% on average and up to 22% (takl), because LLVM's layout choices shift under unrelated edits — the geometric mean itself has been seen between 0.80x and 0.87x across clean runs of the same build. Comparisons are only meaningful when both sides are rebuilt and re-run in the same session, and with nothing else running: a run started while another was still going tripled the AOT column while leaving Chez's untouched.

The suite carries 39 benchmarks and this table compares two of the three ways to run them — AOT and Chez. The bytecode VM runs all 39. Twenty-nine appear with both columns filled; nine show N/A under AOT, which cannot yet build them for four distinct reasons recorded in bench/check-vm-free.sh, and one (quicksort) shows N/A under Chez, which rejects its argument count.

Float and integer kernels win on unboxed f64 paths and zero-cost fixnum arithmetic (tag 000). The losses cluster in allocation-heavy list and symbol code (sboyer, destruc) and in higher-order dispatch (lattice, where every iteration pays an indirect call through a comparator).

VM vs Lua 5.4: geometric mean 0.39x, 14 of 16 benchmarks faster — largely the dedicated pair arena with headerless 16-byte cons cells.

AOT vs VM: 13x geometric mean over 20 benchmarks, from 4.3x to 54.4x.

Embedding

Construct a Memory and a Vm, then register only the libraries you want. Omitting a library makes it unavailable, which is how you sandbox untrusted code; only scheme.base is required.

const std = @import("std");
const lamp = @import("lamp");

var threaded: std.Io.Threaded = .init(allocator, .{});
defer threaded.deinit();

var mem = lamp.Memory.init(allocator, threaded.io());
defer mem.deinit();
var vm = lamp.Vm.init(&mem);
defer vm.deinit();

try vm.registerLibrary(lamp.lib.scheme.base.library);
try vm.registerLibrary(lamp.lib.scheme.write.library);
// or everything: try lamp.registerAll(&vm);

const result = try vm.interpret("(+ 1 2 3)");

Zig functions become primitives through makeNativeClosure + defineGlobal:

fn prim_timestamp(vm_ptr: *anyopaque, args: []const lamp.Value) anyerror!lamp.Value {
    if (args.len != 0) return error.ArityError;
    const mem = lamp.Vm.getMemory(vm_ptr);
    const ts = std.Io.Clock.now(.real, mem.io);
    return lamp.Value.makeFixnum(@intCast(ts.nanoseconds)) orelse error.Overflow;
}

try vm.defineGlobal("timestamp-ns", try mem.makeNativeClosure(&prim_timestamp));

Calling back the other way:

_ = try vm.interpret("(define (double x) (* x 2))");
const result = try vm.callAndRun(vm.globals.get("double").?, &.{lamp.Value.makeFixnum(21).?});

Nested evaluation from inside a primitive must go through callAndRun, not interpret — Vm.run is not re-entrant and will return error.ReentrantRun rather than corrupt the run in progress.

Native value types

Custom types register once and work in both modes. Under the VM they are GC-managed heap values read via nativeData(); under AOT the same Scheme compiles to direct Zig calls with no boxing and no GC, driven by a .zon descriptor naming each type's size and function signatures. Operators are overloadable, so +, -, *, / and display work on them:

(define v (+ (make-vec2 1.0 2.0) (make-vec2 3.0 4.0)))
(display v)         ; #<vec2 4.0 6.0>
(vec2-x (* v 2.0))  ; 8.0

examples/native-types/ is runnable end to end (zig build native-types-example) and documents the descriptor format. examples/callback/ covers VM, mixed, and pure-AOT build modes.

Libraries

Library	Provides
scheme.base	Core language — arithmetic, lists, strings, control flow
scheme.write	display, write, newline
scheme.read	read
scheme.char	Unicode predicates and case conversion
scheme.inexact	sqrt, sin, cos, exp, log, …
scheme.file	open-input-file, open-output-file, …
scheme.time	current-jiffy, jiffies-per-second
scheme.cxr	caar, cadr, caddr, …
scheme.eval	eval, environment
scheme.load	load
scheme.lazy	delay, force, make-promise
scheme.process-context	command-line, exit
scheme.case-lambda	case-lambda
scheme.repl	interaction-environment
lamp.debug	disassemble, debug-break
lamp.meta	library-list, library-exports, symbol-doc, symbol-signature

Scheme source embeds as proper R7RS libraries via @embedFile and define-library.

Docstrings

#doc attaches documentation to a definition. It is stored in the library table at compile time, dropped from bytecode, and queried through (lamp meta):

(define (square x)
  #doc "Return the square of x."
  (* x x))

(symbol-doc '(example math) 'square)  ; "Return the square of x."

Not implemented

Rationals and bignums (integers are 61-bit), complex numbers, make-parameter/parameterize, delay-force, define-values, syntax-case, and datum labels.

Dev tools

devkit (the release mechanism — see .devkit.conf), simgrep (duplicate-code search) and zigmap (codemaps) live in .tools/. None is a dependency:

scripts/setup-tools.sh      # clone devkit, clone and build simgrep + zigmap
scripts/setup-tools.sh -u   # pull first

Work happens on dev; main only advances via .tools/devkit/devkit promote, which merges, runs zig build test, and rolls back on any failure.

License

MIT. See LICENSE.