NEP 29: Typed events with reusable storage#

Created: 2026-09-06 · Status: Implemented

Summary#

nupp.events is an addressed, synchronous event bus whose events are ordinary record and struct declarations marked @derive(events.Event). Emitting an event by type constructs it into storage the bus already owns, and only when something is observing; emitting an existing instance borrows it without allocating. Observers receive the event as a call-scoped borrow they can mutate but not keep. State beyond the event comes from an ordinary capture or an event field.

The design needs five things the compiler did not have: an initializer split out of every constructor, a comptime view of a declaration's construction contract with parameter names, closure literals that adopt parameter modes from the type they are checked against, pack binders that forward the contracts of the arguments that bound them so a protected call can carry an exclusive view, and a derive recipe capability that binds an event to its initializer. Those are decided here alongside the library, because the library is not expressible without them. The first consumer is Tecs, whose Nupp rewrite lost the pooled and arena-backed events its Teal router had.

Goals#

• An event is a declaration, not a registration call: its fields, defaults, constructor and layout are the ones the language already checks.
• Emission by type is fully typed at the call site, including named arguments and defaults, and costs nothing when nobody is observing.
• Warmed emission of a scalar event to non-suspending observers performs no allocation the bus owns: no envelope, no closure, no vararg table, no fresh payload.
• Registering and clearing observers at entity churn rates allocates nothing in steady state, and an unobserved address costs nothing at all, so a world of millions of entities pays for the observers it has and not for the entities it could have.
• An observer may mutate the payload and may suspend, and cannot retain the payload or a view of it past its own return.
• A source owns observer state without becoming part of the observer contract.
• Struct events keep their fixed layout on every backend that has physical storage, and are refused rather than degraded where none exists.
• Every storage and dispatch rule is a checker fixture or a runtime test before the dispatcher exists.

Non-goals#

• Cross-worker or cross-runtime transport. A bus is local to one Lua state; NEP 16 owns what crosses one.
• Running dispatch inside an @aot body. A kernel is nosuspend and calls no callbacks (NEP 9), so delivery always runs in Lua and only a payload may reach a kernel.
• Constructor overloads on an event. One declaration has one construction contract in this version; the reason is in the specification.
• A static proof that emission does not allocate. A dispatcher calls callbacks it cannot see, which widens its effect summary to top (Effect contracts), so the allocation claim is a measured gate rather than a noalloc region.
• A frame. Storage epochs end when a source is quiescent; an engine that wants a frame boundary owns an allocator and resets it.

Motivation#

Tecs's Teal router let a game declare an event record, write an init, and emit by type. The world checked for observers first, took a table from a pool or a row from a per-world FFI arena, ran init on it, delivered it, and gave it back. Emission with no observer cost two hash lookups; emission with one cost no allocation at all. The Nupp rewrite kept the addressing and lost the rest: an event definition is now a callable that allocates an envelope record around a payload its construct allocated, on every emit, whether or not anyone is listening.

Putting that back as library code does not work, and the reasons are what shape this proposal.

A library cannot run a constructor against storage it already holds. new T(...) lowers to one generated function that allocates self, seeds the field defaults, runs the body, and returns the instance. Nothing can call the body on a pooled table or an arena row, so a pool has to reimplement construction, which is how the Teal router came to generate an unrolled init from field names as a string. Nupp's derives are deliberately not source generation, so the initializer has to be something the compiler emits.

A library cannot type the constructor's arguments. emit(address, Damage, amount = 10, source = player) needs the parameter list of Damage's construction, with names so the named arguments bind and with defaults so the omitted ones fill. Comptime nupp.types answers fields, parameters of a function type, and packs, but nothing about how a declaration is constructed, and a computed pack carries types without names.

A library cannot state the borrow it wants from an observer. The contract is "borrowed for the call": the observer may read and write the event and must not store it. That is a borrows parameter, and a borrows parameter is exactly what an observer written as |event| -> ... cannot have. A short-function parameter takes a type and no mode, a function literal's modes are inferred from its body and collapse to plain for anything that is not a pointer, and callable subtyping requires modes to match exactly. So a slot typed function(borrows event: E) refuses every closure a user would write, and a slot typed function(event: E) lets the closure put event in a table with no diagnostic. scoped is the one construct that admits borrowed captures, and it is restricted to callbacks invoked during the call that received them, which a stored observer by definition is not.

Two emit overloads cannot be told apart for a struct. The obvious surface is one emit taking either Type<E> and arguments or an E instance. A record's declaration witness is a distinct Type<T> and its instances are not, so the overloads are disjoint. A struct's witness is the bare nominal, so emit(address, Contact) and emit(address, contact) are the same call to the checker and report NUPP2126.

Observer state needs one owner. Keeping registrations and reusable storage on the source gives registration, dispatch, and cleanup one lifetime. A separate bus object would add another ownership root without adding an event capability.

Overview and specification#

Syntax#

local events = require("nupp.events")

@derive(events.Event)
local record Damage
    amount: number
    source: integer
    kind: string = "physical"
end

@derive(events.Event)
@event(name = "tecs.physics.Contact")
local struct Contact
    entityA: number
    entityB: number
    impulse: float
end

local bus: events.MessageBus<integer> = events.newMessageBus()

bus:observe(enemy, Damage, |event| -> print(event.amount, event.kind), "log")
bus:observeOnce(enemy, Damage, |event| -> print(event.amount))

bus:emit(enemy, Damage, amount = 10, source = player)
bus:emit(enemy, Contact, a, b, 0.5)

local damage = new Damage(amount = 10, source = player)
bus:deliver(enemy, Damage, damage)

Emission by type and delivery of an instance are two names rather than one overload, for the struct reason above. deliver takes the declaration beside the instance, because a struct instance carries no witness the runtime can read on every backend; it performs no acquisition and no release, and is what a caller uses for an event it owns.

The construction contract#

Every record and struct has one construction contract: the parameters of its single declared constructor, or its stored fields in declaration order when it declares none. This proposal makes it visible at comptime as nupp.types.construction(T), a pack whose slots carry a type and a parameter name, with a slot optional when its field has a default or admits nil.

A computed tail written ...: unpackof nupp.types.construction(E) is then a parameter list with names, so a call binds named arguments to it. The generics the tail depends on are bound from the positional prefix before a name is looked up, which is one extension to how a computed tail is checked and none to how it is lowered: named arguments erase to positional Lua arguments as they do everywhere. A default is not filled at the call site, because a positional call to a computed tail never passes through the checker's default filling; the initializer applies it where the argument is nil, which is the one place every call reaches.

An event admits one contract because a computed tail is a parameter list as soon as it reduces, and there is no way for it to be an overload set. A declaration with several constructors is refused by the derive with the reason; a game that wants two ways to build an event writes two events, which a bus treats as two identities anyway.

The initializer#

Lowering of a declared constructor splits into two generated members. The initializer takes the instance first and fills it; the constructor allocates and calls the initializer. A record with no constructor keeps the inline literal that new lowers to today and additionally gets an initializer that assigns each contract slot to its field:

function Damage.__nuppInit1(self, amount, source, kind)
    self.amount = amount
    self.source = source
    self.kind = kind
    return self
end

function Damage.__nuppCtor1(amount, source, kind)
    return Damage.__nuppInit1(setmetatable({}, Damage), amount, source, kind)
end

A field-list initializer applies a field default where its argument is nil, which is why it is written as if a3 == nil then self.kind = "physical" rather than a bare assignment; a constructor's initializer writes the defaults its constructor would have seeded and then runs the body. A struct's initializer writes fields of the cdata it is handed; its constructor remains the ctype call. Completeness, effects, overload selection over the contract, and every constructor diagnostic are unchanged, because the body is the same body checked the same way. The initializer is reachable only through a derive recipe; it is not a member a program can name.

Two things the initializer's contract refuses. A constructor whose body transfers an affine owner into self, or lets self escape before the body returns, cannot be run against reused storage, because the next lease would find the owner already moved or the escaped reference already aliased. Both are refused on an event declaration, by the derive, naming the field or the escape.

The derive#

events.Event is a provider whose recipe claims the events.Emittable interface and returns data and one request: the event's registered name, its representation, and initializer = true. The request is a new result capability beside forward.v1, since a forwarding recipe can pass only the receiver, an argument, the registry entry, a field, or a constant, and what this needs is not a member but a compiler-minted function. The registry entry the derive already writes for every derived type records the initializer's hidden name, so the runtime reaches it through the type witness it was handed, for a record via the witness table and for a struct via the metatype's index table, the same route derived methods take today.

The runtime assigns each registered event an integer identity the first time a bus, events.id, or the derive's own registration asks for it, from a counter local to the Lua state. It is stable for the life of that state and never persisted, which matches the rule extension slots already live under. @event(name = "...") overrides the registered name; the default is the declaration's qualified path. Tecs pins six ECS names and twenty-two platform names as an external surface, which is why the override exists.

The derive admits a concrete record or a fixed-layout struct. A generic declaration is refused, for the reason NUPP2806 refuses one for JSON: type parameters erase (NEP 22), so every specialization shares one runtime table and would share one event identity.

Observers and their borrows#

An observer's type is function(borrows event: E). This proposal adds one rule to closure checking: a function literal or short function checked against an expected function type adopts, for each parameter it leaves without a mode, the mode of the corresponding expected parameter. The expected type already flows into a short function's parameter types; this carries the modes with it. A parameter written with an explicit mode keeps it, and the exact-mode subtyping rule is unchanged, so a closure whose body does something the adopted mode forbids is refused where it is written rather than where it is passed.

With event a borrows parameter, the checks that already exist do the rest: storing it in a table is NUPP2603, returning it or assigning it outward is NUPP2608, and a rooted view derived from it cannot leave the call. Mutation is permitted because a shared borrow proves non-invalidation, not non-mutation. Other state is either captured by the callback or carried by the event itself.

Sources#

export interface Source<A>
    observers: events.Observers<A>
end

export function emit<S is Source<A>, A, E is Emittable>(
    exclusive source: S,
    address: A,
    event: Type<E>,
    ...: unpackof nupp.types.construction(E)
): nil

The reusable implementation is a set of functions generic over the source, each taking it exclusively and reaching observer state through the observers field. MessageBus<A> is a record with that field whose methods forward to them; Tecs's World adds the field and the same forwarding methods. There is no bus object beside the source to overlap it. The observer table is an ordinary field reached through the source while dispatch updates registration bookkeeping.

Addresses are table keys compared by identity. Tecs supplies its packed entity ids with their generation, so a recycled slot is a new address, and zero is a Tecs convention for the world rather than anything the bus knows.

Registrations#

Observer state is a map from address to a map from event identity to one flat array per pair, and nothing exists for an address or a pair until something observes it. An entity nobody observes has no entry, which is what lets a world of four million entities with a few thousand observed ones hold a few thousand entries. Emission at an unobserved address is the source's registration count, which short-circuits everything when it is zero, and then two hash lookups that find nothing.

A registration is three slots of that flat array, its name or false, the callback, and whether it fires once, interleaved, with the live count kept beside the array rather than in its length. There is no record per registration. The Teal router chose this layout and the reason survives it: at a million registrations a record each is on the order of a hundred megabytes where a few slots each is on the order of forty, and the dispatch loop reads an array slot per observer instead of a field of a table it had to fetch first. The count lives beside the array so that a removal can leave a hole to be swapped out later without the length lying about how many observers a delivery should visit.

A removal during delivery and a consumed once registration are the same operation: the callback slot is overwritten with a tombstone, a function that does nothing. A nested or interleaved delivery that reaches the slot calls the tombstone, which is how "cannot be invoked again" is guaranteed without a flags array or a check in the loop. Compaction swaps tombstones out after the outermost delivery leaves, in one pass over the arrays a removal touched.

The per-address maps, the per-pair arrays, and the deferred-compaction list come from pools the source owns and go back to them when they empty. Under entity churn an observer registered at spawn and cleared at despawn would otherwise allocate two tables per entity per lifetime, which at the entity counts above is the one steady-state allocation the bus itself would own. A detached list is returned to its pool only after the last delivery holding it has exited, since an in-flight loop is reading it.

Storage#

Storage is a general facility that events consume. nupp.mem.pool is a table pool that clears on release and reserves a capacity; nupp.mem.arena is a paged allocator over the cstorage capability whose pages never move and whose reset rewinds every used page's cursor. Both are usable without the bus. nupp.events is classified as requiring cstorage, so a backend with no physical storage refuses the module at build time instead of substituting table-backed structs: the browser and Wasm backends have it, portable does not.

An allocator is acquire and release over one representation. A source creates its defaults lazily, a pool per record event and an arena per struct event; setAllocator installs a caller-owned one, whose lifetime the caller guarantees. Dispatch runs the initializer and the observers under one pcall of a fixed function with the source, the list, and the storage as its arguments, and releases the storage when the call returns whether it returned, raised, or was cancelled while an observer was suspended; the failure is raised again after the lease is back. No closure is built per emission. That is what the pack-forwarding feature is for: a protected call's A... binder takes the mode of each argument that bound it, so the source reaches the delivery loop as the exclusive view the emitting call held, and the callee is held to declaring it. Without it the only protected call the checker admitted with an exclusive argument was one wrapped in a closure, which is the per-emit allocation the current Tecs world pays.

Reclaimed record storage is cleared before it is leased again; leased struct storage is zero-filled the way a bare struct binding is. Nested emission of the same type acquires distinct storage because a lease is not released until its delivery leaves. A default arena rewinds when its source becomes quiescent, meaning the outermost delivery has left; an explicit arena rewinds when its owner says so, and a rewind, close, or replacement that would invalidate a live lease raises before changing anything.

Dispatch#

Delivery is synchronous. An observer that suspends suspends the emission with it, and storage stays leased until it returns or is cancelled. Observers run in array order and see each other's mutations. A delivery reads the array length on entry, so an observer added during it joins the next emission, including a nested one. Removal during delivery tombstones the registrations it resolved when it was asked for and compacts by swap after the outermost delivery leaves; removal by callback tombstones every match, removal by name the first. A once registration is tombstoned before its callback is invoked, so nothing nested or interleaved reaches it again. Clearing an address or resetting the source detaches its lists; a delivery holding a detached list finishes it, and the list is recycled after the last such delivery exits. reset clears registrations and nothing else.

A caller who wants to remove by callback keeps the value it registered. A short function is a fresh object each time its expression runs, so an inline literal is unremovable by identity, and the documentation says so.

Consumers in Nupp#

A survey of Nupp itself for hand-written observer patterns found nothing that should move onto the bus. Every callback list in the tree is one of three shapes: one-shot waker lists plumbed into the suspension contract, which carry no payload and answer a woken count the readiness pump sums; single optional hooks such as the incremental checker's observer, the log sink, and watch mode's two loader callbacks; and method routers keyed by name whose entries return a value, such as the LSP handler table and the service registry. The closest match is the HTTP transport's waiter channels, addressed by transfer and keyed by four event kinds, and even there dispatch answers a count and the channels are members of a transport provider interface. What the task and worker waiter lists could use is the storage half of this proposal, the pooled interleaved array, not the bus.

Hot reload is the one place a bus would be an addition rather than a migration. The runtime is pull-based by NEP 30's rule that a consumer asks for changes rather than being called, so nothing observes a committed generation today; a program that wants to be told would observe a GenerationCommitted event at the runtime's address. That is a change to NEP 30's contract and is left to a proposal of its own, with this bus as the mechanism it would name.

What lands, and in what order#

The compiler half is the initializer split, nupp.types.construction, named slots in computed tails bound from the positional prefix, mode adoption in closure literals, pack binders forwarding contracts, and the initializer recipe capability. It landed first, with nupp.mem.pool and nupp.mem.arena, which use nothing new. nupp.events waited for a release: the stage-zero rule (NEP 28) reaches every module under src, because a cold checkout's first build is a plain build by the pinned release that type-checks the whole include set, so a standard-library module written against nupp.types.construction and pack forwarding fails that build until a tag carrying them is published and the pin moves. Tecs pins Nupp by revision and migrates against the revision the module landed in.

Building the initializer found that a struct with a declared constructor passed the checker and failed at run time, since codegen never emitted the member new resolved to; that was fixed first, as its own change.

The gates before the library lands are checker fixtures for every rule above, written against the current fixture suites; runtime tests for every storage and dispatch rule, including the historical Teal cases; a standalone fixture reproducing the Teal router, pool and arena at the revision that deleted them, measured five times interleaved against the new bus on the same machine; and nupp bc --check plus a trace-IR count on the emission path, so no closure, vararg table, or reflective lookup enters it unnoticed. Warmed LuaJIT at the deliverable optimization level is the measured native path; the browser is measured separately through the Wasm project harness.

Risks and assumptions#

• Mode adoption changes what existing closures mean. A literal passed today to a slot with a borrows parameter is refused, so no working program adopts a mode it did not have; the risk is a literal that was refused for a mode mismatch and is now checked under a stricter body rule, reporting a different diagnostic. That is the intended change.
• Named slots in a computed tail bet that names belong to packs. If a pack turns out to need to stay nameless, emission by type falls back to the derived static in the alternatives, which costs a generic member recipe.
• A plain table field is assumed not to be a region. The dispatcher reaches source.observers while holding the source exclusively. The fixture checks that ownership shape directly.
• The initializer is a second entry to every constructor body. Anything that reasons about a constructor as one whole, such as the refusal of @aot on one, has to see both.
• Pack forwarding widens what a generic tail admits. An owner or a borrow handed to A... was refused outright; it is now accepted when the callee that receives the pack declares the matching mode, and refused where it does not. A program the old rule accepted is unchanged, since a plain argument still binds a plain slot.
• Two steps is the honest cost. The library cannot land beside the compiler features it uses; it lands one release later, and until then the branch holding it is the work.

Alternatives considered#

Keep the current definition-object design and add pooling underneath. The envelope and the construct callback are the allocations; pooling the payload still leaves construction outside the type system and the borrow of the payload unstated. The Teal router already showed where that leads, with a string-generated init.

One emit overloaded on Type<E> versus an instance. Reads well for records and is ambiguous for every struct, because a struct's witness is its nominal. Giving struct witnesses a real Type<S> would touch every Type<T>-directed API that relies on the nominal unifying with the witness. Two names cost nothing.

A derived static, Damage.emit(source, address, ...). The derive knows the contract and could declare the member with a closed signature, which needs no pack changes. It fails on the source: the member has to be generic over the address type and the source type, and the recipe language admits no generic member. Making it admit one is as much compiler work as naming pack slots, and buys less, since nupp.types.construction serves every other construct-into API as well.

Infer observer modes from the body. Today's rule for function literals, extended to short functions. It cannot express "borrowed for the call" for a value that is not pointer-shaped, and it decides the mode after the body has been checked as plain, which is the wrong order for refusing an escape.

A scoped observer. scoped is the contract for a callback consumed during the call that receives it. An observer outlives its registration call by construction, so the contract would have to be widened to stored callables, which is a different and larger change than adopting a mode.

A bus object owned by the source. The natural library shape, and the one that overlaps: exclusive world.events under exclusive world is NUPP2607. Functions generic over a Source<A> interface keep one root.

A frame-scoped arena owned by the engine, reset at a host boundary. Once observers cannot retain a payload, nothing can read one after the outermost delivery leaves, so a rewind at quiescence retains exactly what a frame reset would, minus the concept. An engine that wants a longer epoch installs an explicit arena.

A record per registration. The shape the current Tecs world has, and the easier one to read. It costs a table per observer, a field fetch per observer in the dispatch loop, and a flags field or a side list to mark a removal in flight, and none of that scales with entity count in a direction anyone wants. The interleaved array is what the Teal router had settled on for the same reasons.

Observer tables allocated on demand and left to the collector. Correct, and free of bookkeeping, and it makes subscription churn the one thing the bus allocates for in steady state. The pools cost a few lines and were already there once.

Assigning event ids at compile time. Ids would then have to be stable across incremental rebuilds and separate compilations, which is a wire identity in disguise. A per-state counter assigned on first use is what the runtime's other slot identities already do.