sicher
released this
2026-10-24 17:49:11 +02:00
| 4 commits to main since this release
- Replacing a native unit while the session runs:
Session.reloadNative(path) loads the rebuilt .dylib at path over the
unit loaded from it; the new definitions take over the names, and
procedures kept from the old one run its code. The old library is closed
by the first full collection after nothing needs its code
(Session.replacedNativeUnits counts them). loadNative of a path
already loaded is now error.NativeUnitLoaded.
Session.can_suspend: whether .stops = .suspend_ can suspend a
call here -- macOS on arm64. Elsewhere it is .hook, as before, and no
longer tries to make a fiber it cannot guard.- A .dylib loads into an engine built in any optimize mode: compiled
code read the runtime's heap at offsets Zig lays out differently in Debug
and in release builds, so a unit from a Debug lamp-native was refused
by a ReleaseFast engine ("compiled against a different runtime library").
What it reads is now in extern blocks with one layout.
- Faster string loops in compiled code:
string-ref on a string with
a character past Latin-1 (a wide string) and string-length run inline;
both went through the interpreter's handler. So do char-alphabetic?,
char-numeric?, char-whitespace?, char-upper-case? and
char-lower-case? for an ASCII character. A word count over a text with
non-ASCII in it ran 6x faster.
- A native used as a value is one procedure in compiled units too: a
unit compiled by lamp-native made its own closure for a native it
passed as a value, so eq? against the session's was #f (and a host
keying anything on the procedure missed it). A unit now takes the
session's, or leaves its own there for the session.
lamp-native for a dependent's build: lamp_dep.namedLazyPath("lamp-native")
is the compiler, for a build step that compiles its Scheme to a .dylib.lamp-native replaces its output instead of rewriting it: the new
file is written beside the old one and renamed over it. A running
session that loaded the old .dylib keeps its code intact, and loading
the path again opens the new one.- Native units as a .dylib:
lamp-native compile --unit --dylib unit.scm unit.dylib writes a signed dynamic library -- by itself, no
linker or Zig needed -- and Session.loadNative("unit.dylib") loads and
runs it on the session, as runNative runs a linked one.
- A native unit's entry takes the runtime's table: a unit object has
no undefined symbols now -- its calls of the runtime go through a table
Session.runNative passes it -- the first step to loading one as a
.dylib. A host declares the entry with a second parameter:
extern fn lamp_unit_main(host: *anyopaque, table: *const anyopaque) callconv(.c) c_int (Session.NativeEntry). Recompile units.
set! of a function defined earlier in a session: at the REPL (or
by load/eval), (set! f ...) after (define (f) ...) failed with
"UnknownVariable". It now assigns f, and functions compiled before
call the new value -- a procedure, or an error if it is none. Wrapping
the old one, (set! f (let ((old f)) ...)), works; a prelude function
is shadowed for the REPL, as define does.- A native unit's own function used as a value is the session's:
f
passed or returned by the unit's code was a private closure -- not
eq? to the session's f, and still the old function after the
session redefined f. It is now read from f's binding. And two
compile-time assumptions about the unit's top-level functions no longer
hold once the session can redefine them: that a function handed a
continuation never keeps it (re-entering it failed with "invoked after
its call/cc returned"), and that one captures no continuation (a list
built by recursion could be changed after it was returned).
- A function's own tail call sees a redefinition made during it: in a
session (the REPL, load, eval, an embedding), a function redefined by
something it called -- (define (f n) (g n) (f (- n 1))) with g
redefining f -- went on looping in the old definition; it now calls the
new one, as R7RS reads the name at every call. Native units likewise, for
tail calls and other calls of itself. Only where a real call can come
first: a loop over primitives, or fib, runs exactly as before.
- A native unit sees a redefinition of its own functions: a function
of the unit's top level redefined by the session (define, set!,
load) is the one the unit's other functions call from then on; they
called the compiled one. A library's functions in a unit are not
affected, as R7RS lets no importer redefine them. A small top-level
function is still inlined into its callers in the unit, behind a check
of its variable: a loop calling (define (inc x) (+ x 1)) runs as fast
as in a library (81 ms against 86), where without inlining it took 247.
- A native unit's function given a continuation keeps its parameter:
when every call in the unit passed it a call/cc's continuation, the
compiler dropped the parameter, and the session's calls of the function
failed with "wrong number of arguments".
- Several native units in one session:
session.runNative runs
each unit a host links (lamp-native compile --unit --entry NAME
names each one's entry). The units call each other's functions
directly -- 3.4 ns a call, against 2.6 ns inside one unit -- by name
or as values, and their guards, handlers and continuations work
across each other's code. A unit reads another's function from its
variable at each call, so a new definition reaches it.
- A native unit with nothing at its top level but definitions can be
called: the session's calls of its functions failed with "a native
unit's procedure before the unit ran".
- A native unit carries only the prelude it uses: a unit compiled
with lamp-native compile --unit held the whole prelude, 109 KB of code
for a one-line unit; it now holds what its code can reach (980 bytes
for that one).
- Continuations cross a native unit's boundary: the unit leaving a
for-each of the session's early, a callback the unit calls leaving
through it, and the reverse, go back to where the continuation was
captured, unwinding both sides' frames, guards and dynamic-winds
between; one may be re-entered from the other side while the call
that captured it runs. One resumed after that call returned is an
error that says so.
- A continuation called in a host native's callback goes back to where
it was captured; it could crash the VM, or resume inside the callback
and return to the native. call/cc inside such a callback is refused also when
the native was called from the top level, as it was everywhere else.
- A native unit and the session call each other to any depth: the
session's code may call back into the unit while the unit is calling
it, and a host native called from the unit may call closures of
either (session.call). A raise reaches the innermost handler or
guard on either side, with the object raised: the handler stack and
the dynamic-winds are now one for both. An apply of a list longer
than the VM's window to one of the unit's variadic procedures passed
one argument and upset the VM's next variadic call; it is spread as
it should be.
- A procedure of no parameters rejects arguments in the VM:
((lambda () 1) 5) returned 1 when the lambda was called through a
variable or passed as a value, and is now a "wrong number of arguments"
error, as it already was for every other arity.
- A native unit calls the session's procedures: a lambda passed to it,
a procedure such as list as a value, and a function the session
defined, by name. The session's VM runs them nested; an error there
reaches a guard in the unit, and a raise reaches a guard of the
session's below the unit. Calls between the unit's own functions are
unchanged: the session's take the path that used to refuse them.
- Steps in callbacks a host native makes. When Scheme calls a
native that calls back into the session once per entity (a frame
written in Scheme), a step that leaves a callback (step_out, or
running off its last line) now stops at the native's next callback,
then at the caller's next line, compiled or not. Before this, step_out
ran to the end of the frame, and a compiled caller never stopped.
Stepping over the native's line no longer stops inside its callbacks.
The abort fault now reads "aborted while stopped in the debugger".
- Breakpoints and steps in library code. Code from a library loaded
from the search path, from an inline define-library, or from an
included file now keeps its own file and lines. A breakpoint in
my/util.sld binds when the library loads, a step goes into it, and
backtraces and the editor show util.sld:6. A library file is named
by its real path. Before this, such breakpoints never bound and steps
ran through.
Session.runSuspendable(f, ctx): in suspend mode, a stop in any
callback the host's script phase makes -- on-message in its message
passes, on-collision -- suspends the phase itself, the host's own loop
with it. resumeStop goes on from the stopped instruction and returns
when the phase does. Nothing is skipped, repeated or reordered; a new
phase while stopped is error.SessionSuspended. resumeStop's error
set is now anyerror (a phase's own error comes back through it).
docs/EMBEDDING.md, "Stops in every callback, and order".dap.Adapter: the debug adapter without a transport, for a host
with its own threads and framing -- one request in, its responses and
events to a sink. dap.Server is now the socket and stdio framing over
it, used as before (server.launch_why is server.adapter.launch_why).
In suspend mode the editor's continue or step is the host's to carry
out (takeResume), from its frame loop; hostStopped/hostContinued
tell the editor of the host's own pauses, and logpoint messages are
output events. Conditions, hit conditions and logpoints come through
setBreakpoints; a breakpoint that binds or moves on a load is told
as a breakpoint event. Compound values open in the variables view
(lists, vectors, records, pairs; long ones capped), a host adds scopes
of its own (adapter.scopes_fn), and evaluate takes context
(hover reads a name without compiling) and runs at top level when no
frame is named while suspended. adapter.output(category, text) sends
a host's own output to the editor's debug console. The session's debug events add stopped and
continued (suspend mode).- Breakpoint conditions, hit counts and logpoints, in the session
(s.inner.setBreakpointWith(file, line, .{ .condition, .hit_condition, .log_message })), so a host without DAP has them too. A condition is
compiled once per breakpoint, not at every hit. A logpoint's message goes
to s.inner.onDebugEvent; it never stops.
- Breakpoints follow their source file: a hot reload keeps them, and
one set before its file is loaded binds when it is (setBreakpoint
returns null for it, and a breakpoint debug event follows). Before,
a reload's new code had none, and an unloaded file's were refused.
- Debugger stops that return to the host (experimental, arm64):
Options.stops = .suspend_ makes a call that stops return
error.Suspended; the host inspects the stop (Session.stopped) and
goes on later from its own frame loop with Session.resumeStop --
continue, step or abort. Meanwhile it may still evaluate, in a stopped
frame or at top level, and reload code. Session.debugScope says which
calls are debugged (a game's focused entity); a step that leaves its
call stops in the next call in scope. See docs/EMBEDDING.md, "Stops that
return to the frame loop".
- Fixed:
pause() did not stop in a compiled procedure the host then
called -- the step only ever saw interpreted code; the procedure is now
taken out of compiled code for the step. And a step no longer outlives
the host call it was taken in.
- Fixed: a
guard in a procedure the host called with Session.call
(or callWith) crashed -- a bus error once it caught something: the
call used the catch storage of the last run, in that run's finished
frame. A host call now starts as a run does, with the session's own
catch stack and an empty frame stack (a call after a stack overflow
also faulted again).
(- v) on a native type: native.Type's .neg option, functions
of one argument (fn (Vec2) Vec2). Without it (- v) needed a .sub
overload taking a number on the left.- Native-type values from the host:
callWith and the new
Session.box make a struct the session registered as a native type into
that type (it was silently a list of its fields), and decode reads one
back. A native's struct parameter or result may be another library's
type. A native may return a tagged union (its active payload), and read
a rest argument with a parameter's rules and refusals (c.as(T, i)).
- Fixed:
callWith could lose a Value argument to a collection while
it built a later one; an isa.Value argument also became a list.
- A host can write top-level variables:
Session.setGlobal(name, value), as (define name value) at the REPL, superseding a function of
that name; Session.slot(name) is a handle to read and write one
without a lookup. global(name) of a function no longer compiles a unit
on every call.
- Fixed: after
(define f #f) and a new (define (f) ...), f as a
value was still the old function (and global("f") returned it): the
old function's one closure was reused. Now the new function is.
- Natively compiled code can run inside an embedder's session
(experimental, static linking): lamp-native compile --unit and
Session.runNative; the unit's globals are the session's by name, and it
can call the host's natives (Session.writeNativeInterface,
--natives). The session can call the unit's procedures, its
functions by name and the closures it returns; the unit cannot call the
session's yet. See docs/EMBEDDING.md, "Compiled ahead of time".
- A library's own definitions stay its own: a program that leaves an
export out of its import ((except (my lib) f)) and defines its own f
no longer replaces the one the library's code calls (R7RS 5.6.1). A
library's body also sees another user library through its own import
sets (prefix, rename, only, except).
- A program may no longer redefine or
set! a name it imported
(R7RS 5.2): (import (scheme base)) (define (square x) ...) is refused,
naming the library, in both engines; leave the name out with (except (scheme base) square) or rename yours. The REPL and
Session.evalText still permit it. Without an explicit
(import (scheme base)) a program may define base's names as before.
- Native backend:
display and write inside with-output-to-string
are captured in every function: those the optimiser could not take
printed straight to the console, and their write did not label a
cyclic list.
- Reading a file uses far less memory:
cat, wc and tail peaked
at 290-320 MB and now take 12-39 MB (Chez: 49-97). A file is read into
one buffer of its size, and a large string that outlives a minor
collection no longer makes the heap grow instead of collecting it.
docs/BENCHMARKS.md has a new Memory section (Lamp against Chez).
- Memory use and the collector can be inspected:
Session.memoryStats()
(live and size per area, collections, pause times and their histogram)
and Session.onCollect for an embedder's profiler, Session.requestFull;
(gc-stats) in a new (lamp gc) library; lamp --stats and
LAMP_STATS=1 print a summary at exit, for natively compiled programs
too. See docs/EMBEDDING.md.
- Native backend: functions the optimiser cannot take (with
call/cc,
guard or apply) allocate their closures inline too (fibc 1.1x
faster).
- Native backend: a local procedure does float arithmetic unboxed when
the function that defines it works with floats, even with no float
constant of its own (simplex 1.1x faster).
- The collector stops minor collections sooner while a program's data is
growing: paraffins 1.6x faster with a third less peak memory, earley
1.15x (native), nboyer 1.06x. LAMP_GC_SUSPEND=<n> sets the threshold.
- VM: a suspended call's dead registers are no longer garbage-collection
roots, so memory a finished call left behind is freed (paraffins peaked at
over 1.6 GB and could run out; it now keeps what it uses).
- Fixed: a procedure called with its result going to a parameter slot could
look unused before the call (VM liveness analysis, which the native
backend's garbage-collection maps also use).
- Local procedures are compiled as ordinary functions wherever they are
defined -- in a let body, a lambda, a letrec, a named let used
from a lambda, or bound by let/let* -- instead of as closures (in a box
when recursive), the locals they read passed as arguments: direct calls,
which both engines can inline. Native: quicksort 1.7x faster, mperm 1.65x,
gcbench 1.4x (also on the VM), lattice 1.35x, earley 1.12x.
- Native backend:
vector-ref in a loop resolves its vector once
(pnpoly 1.35x, fft 1.1x faster), and loops are aligned, which removes a
22% swing in ack from code placement alone.
- A built-in procedure used as a value is called directly when the
argument count fits, instead of searching every native's name on each
call: the scheme benchmark 1.4x faster native, 1.3x on the VM.
- Native backend: calling a variadic function costs what Chez's does
(no rest arguments or one: 4x and 3x cheaper), and values used as a
procedure returns one value without building anything
((vector-map values v) 3.2x faster).
- Fixed (native backend): values held across
apply or
call-with-values could be freed by a collection inside the called
procedure.
- Native backend: a list walk tests each step once --
(null? l)
followed by (car l)/(cdr l) tests for a pair first instead of for
() and then for a pair (takl and ntakl 1.08-1.1x faster).
- Lists built by recursion in a
letrec lambda or a named let are built
by a loop too (primes 1.6x faster native, 2.2x on the VM; paraffins
1.25x).
map and for-each over a lambda or a known function run as a loop
(no closure, no indirect call per element), and map builds its result
forward where the function cannot capture a continuation: deriv about 1.4x
faster on the VM, 1.25x native.- Removed: the Zig AOT compiler (
addSchemeExecutable, the lamp-aot
module, zig build aot-compile, --emit-zig and the --aot-*,
--native-types, --dump-analysis and --audit-roots flags). Compile
ahead of time with the native backend (lamp-native); the old flags now
say so.
- Native backend: closures allocated inline (cpstak 2.6x, lattice and
graphs about 1.3x faster).
- A list built by recursion is built by a loop (
(cons x (f ...))
in tail position, when nothing in the body can capture a continuation):
divrec 1.5x faster native, 2.9x on the VM, and the VM no longer runs out
of registers building a long list this way.
- Native backend: no write barrier for storing a fixnum or immediate
(array1 1.6x faster).
- Native backend: fibfp 1.45x faster (constants in guard states take no
slot, raw floats kept in registers up to a call). Fixed a fallback that
could resume a float argument from an unset register.
call/cc used only as a return is compiled away (ctak about 55x
faster, VM and native); the native backend compiles a continuation that
cannot outlive its call/cc as an escape (fibc 1.9x).- Native backend: deep self-recursion no longer mispredicts every
return (a function's one self call is kept off the CPU's return stack):
divrec and primes about 1.55x faster.
- Fix: a call or
vector with many arguments in a busy function. When
its arguments ran past register 63 they silently overwrote live values (a
wrong answer); such a call is now compiled to fit, and such a vector is
built element by element.
- Native backend: Ackermann-shaped functions run as one loop (a self
call as an argument of a self tail call, integers only) over a stack of
their own (ack 5.5x faster).
- Native backend and VM: plain ASCII file output is written straight
out, and native read-char/write-char are inline (cat about 4x, wc
2.4x faster).
- Native backend: type predicates,
eqv?, vector/record allocation and
string/bytevector stores inline (gcbench, peval, dynamic about 2x
faster).
- Native backend: character I/O about 2x faster (cat, wc).
- Native backend: integer loops run without type tests (sum 2x faster).
guard re-raises as R7RS says. With no clause matching, the object
is re-raised by raise-continuable from the raise itself, so the winds
around it are re-entered, and an outer handler's value returns to a
raise-continuable. Clause tests run outside those winds. (An error a
primitive signals, like (car 5), is re-raised from the guard; the AOT
re-raises every one from the guard.)- Native backend: float loops that start from an argument or a captured
variable run unboxed (sumfp 6x faster), and closures speculate too.
- Fix:
apply and call-with-values call in tail position. As R7RS
requires: a loop through either used to fill the stack after about a
million iterations.
- Fix:
exact-integer-sqrt in a full heap. A program doing heavy
bignum arithmetic could stop with "%isqrt: no room" (the pi benchmark at
full size); the root now collects and grows the heap as * does.
- Fix:
guard runs dynamic-wind's exits. A raise or an error inside
a dynamic-wind that a guard caught skipped the wind's after thunk; it
now runs before the guard's clauses, as R7RS says.
- Fix:
(flush-output-port) with no port. The VM refused it at
compile time; it now flushes the current output port, as R7RS says.
- Embedding:
decode takes typed shapes. Slices from lists, structs
and tuples from lists of their fields (a 2-tuple from a dotted pair too),
native.Pair, optionals and tagged unions -- the first variant that fits
-- with refusals that name the element: "element 2: expected a symbol or
an integer from 0 to 15, got 99".
- Embedding fix:
evalText and evalSource from inside a native. A
unit evaluated by a native of the same session -- a REPL key -- ran over
its caller's registers, so a loop around the native saw the unit's
result in its counter. It now nests as call does; so does global
when it compiles a lookup.
- Natives: a library entry can carry a signature and doc.
.{ "move-left", f, "(move-left [count])", "Move the cursor left." };
docOf, symbol-doc and symbol-signature answer them.
- Embedding: names for completion.
Session.definedNames lists what a
session's units defined at top level, and lamp.embed.keywords the
special forms; libraries and docs come from (lamp meta) through
decode.
- Embedding: a session's output can go to a host writer.
Options.output and Session.setOutput(w) send display, write and
the rest to a *std.Io.Writer -- a REPL buffer, a game console, or
anything but stdout in a Zig test binary, whose stdout is the build
runner's protocol.
- Embedding:
Session.callWith builds a call's arguments from Zig.
s.callWith(f, .{ "query", 3 }) -- strings, numbers, symbols, lists,
pairs -- where call needed ready-made values and a host had no way to
make a string.
- Natives: a library may have hundreds of natives. Adding a large one
stopped compiling with "evaluation exceeded 1000 backwards
branches", a limit only Lamp could raise; it now scales with the library.
- Embedding: a native may call back into its own session.
Session.call from inside a host native runs nested, as eval does,
instead of overwriting its caller's registers; a fault inside reaches the
caller (and its guard) or is cleared by the native with errorMessage.
- Natives:
native.Pair(A, B) results are dotted pairs. (cursor) can
answer (line . col) and a list of pairs is an alist; a plain struct is
still a list of its fields.
- Natives: an optional parameter takes
#f as null. A ?T parameter
is null when its argument is #f, as when it is missing -- a key binding
can always pass its count or #f. For ?bool, #f is still false.
Before, an explicit #f was refused.
- Embedding:
Session.decode reads Scheme data into Zig. s.decode(T, v, arena) copies a value -- a native's argument, a callback's result --
into Datum (any value: lists, dotted lists, strings, symbols, numbers,
and procedures or native values tagged, not copied) or a typed leaf
([]const u8, native.Symbol, integers, floats, bool, Held), with
Lamp's error text when it does not fit.
- Embedding: a host can keep any value alive.
Session.hold(v) returns
a handle whose value every collection keeps, held(h) reads it and
release(h) lets it go -- for a lambda a script handed to a native, say.
Before, the only way to keep a value was to put it in a global.
- Embedding: a native used as a value is one procedure. Each mention
of a native in value position used to make a fresh closure, so (eq? char-upcase char-upcase) was #f; now a session makes one per
native. Session.global also finds a native the session imported from a
library, so a host can keep its natives' values and recognise them by
identity.
- VM: float loops run unboxed. A loop that does float arithmetic keeps
its floats in machine registers for as long as it runs, instead of
allocating a heap object for every result: sumfp 3.6 -> 0.78 s,
mbrot 4.7 -> 1.4 s. LAMP_JIT_NO_FLOAT_LOOPS=1 turns it off.
- VM: recursive float functions pass floats unboxed. A function whose
arguments and result are floats and that calls itself (fibfp) gets a
second compiled form taking and returning them in machine registers:
fibfp 2.3 -> 0.59 s, sumfp 0.78 -> 0.47 s. LAMP_JIT_NO_FLOAT_ENTRIES=1 turns it off.
- VM: small functions are inlined at their call sites. A call of a
short function (an accessor, a square) runs its body in place behind a
one-compare guard, so a redefinition, a breakpoint or a step into it still
takes effect at the next call; backtraces and error messages still name
the inlined function. Over all benchmarks 2.7% faster (geomean), ray
20%, fibc, conform, simplex, destruc, nucleic, mperm 13-15%.
LAMP_NO_INLINE=1 turns it off.
- VM: recursive fixnum functions call themselves natively. A function
whose arguments and result are fixnums and that calls itself or loops
(fib, tak, ack) gets a second compiled form that keeps its values in
machine registers and calls itself directly; overflow falls back to the
ordinary code, which promotes to a bignum. sum 1.9 -> 0.96 s, fib
3.2 -> 1.9 s, tak 1.5 -> 0.97 s, ack 6.4 -> 4.9 s.
LAMP_JIT_NO_INT_ENTRIES=1 turns it off.
- VM fix: compiled
+ or - that overflowed into one of its own
operands' registers returned a wrong fixnum instead of a bignum -- e.g.
(- acc 1152921504606846975) updating acc in a loop. The interpreter
was right.
- AOT:
read no longer records source lines for the data it reads --
bookkeeping meant for the compiler's error messages, a hash insert per
pair. read1 0.64 -> 0.40 s.
write no longer clears 20 KB of scratch per call. Its cycle scan
set up its tables for every value, a number included; now only pairs
and vectors are scanned. The ray benchmark, which writes a number per
pixel: VM 2.45 -> 2.34 s.
Downloads