# Schema-driven serde

Serde separates a value's logical shape, its physical representation, and its
wire format:

- `Schema` describes members, scalar kinds, requiredness, and defaults.
- `Binding<T>` joins that schema to a record, struct, or dense dynamic value.
- a codec profile decides format policy such as JSON field names and unknown
  member handling.

The separation lets a generated record and a run-time client use the same codec
without making the standard library understand a service model such as Smithy.
Format-specific work is prepared and cached rather than generated once per
type, protocol, and format.

## Derived bindings

`@derive(nupp.derive.Serde)` records format-neutral materialization data. It
does not add serialization methods to an instance.

```nupp:playground
@derive(nupp.derive.Serde)
local record User
    id: uint32
    name: string?
end

local binding = nupp.serde.of(User)
local prepared = nupp.serde.json():prepare(binding)
local text = prepared:encode(new User(id = 41, name = "Ada"))
local restored, problem = prepared:decode(text)
local output = require("nupp.text.buffer").new()
prepared:write(new User(id = 42), output)

assert(text == [[{"id":41,"name":"Ada"}]])
assert(problem == nil)
assert(restored and restored.name == "Ada")
```

The declaration name is the `Type<T>` witness accepted by `serde.of`. This is
also true for a fixed-layout struct:

```nupp
@derive(nupp.derive.Serde)
local struct Vec3
    x: float
    y: float
    z: float
end

local binding: nupp.serde.Binding<Vec3> = nupp.serde.of(Vec3)
```

Struct derivation reads fields individually. It does not serialize padding,
endianness, or the memory image. Pointer fields are rejected because the field
type alone establishes neither their extent nor their ownership.

## Dynamic schemas

A dynamic client builds and freezes the same logical schema, then binds names
once into dense indexed storage:

```nupp
const serde = nupp.serde
local builder = new serde.SchemaBuilder()
builder:structure("example.User")
builder:required("id", serde.uint32)
builder:defaulted("active", serde.boolean, true)
builder:optional("name", serde.string)

local schema = builder:freeze()
local binding = serde.dynamic(schema)
local value = binding:bind{id = 41, name = "Ada"}
local text = serde.json():prepare(binding):encode(value)
```

`bind` rejects unknown members, missing required members, wrong scalar kinds,
and out-of-range fixed-width integers. Code constructing many values can retain
a `Member` and avoid name resolution:

```nupp
local id = schema:expectMember("id")
local value = binding:newValue()
value:set(id, 41)
assert(value:get(id) == 41)
```

A schema may have several bindings at once. A nominal record and a dynamic
value can therefore share logical member identity while retaining different
physical access plans.

## Typed metadata

`MetadataKey<T>` is a typed identity for data supplied with a schema. A dynamic
client can attach root or member metadata while constructing its model:

```nupp
const serde = nupp.serde
local serviceName: nupp.serde.MetadataKey<string> = serde.metadataKey()
local builder = new serde.SchemaBuilder()
builder:structure("example.Credentials")
builder:required("user", serde.string)
builder:required("password", serde.string)
builder:metadata(serviceName, "example")
builder:memberMetadata("password", serde.debugRedact, true)

local schema = builder:freeze()
assert(schema:metadata(serviceName) == "example")
assert(schema:expectMember("password"):metadata(serde.debugRedact) == true)
```

Every key receives a dense process-local index. The index is never serialized
or treated as stable across runs; it makes both derived and dynamic metadata a
direct indexed lookup after the schema is frozen.

## Debug preparation

`Debug` is a schema consumer rather than a separate generated traversal. Its
prepared plan combines member metadata such as `debugRedact` and `debugSkip`
with the binding's record, struct, or dynamic-slot access once, then caches the
result on the binding:

```nupp
local prepared = serde.prepareDebug(binding)
local text = prepared:format(value)

local output = require("nupp.text.buffer").new()
prepared:write(value, output)
```

`format` returns the conventional Debug string. `write` appends directly to a
caller-owned FIFO byte buffer, which avoids allocating that complete result
and is the appropriate path for logging and larger composed diagnostics. A
derived `value:debug()` lazily retains the prepared operation on its type entry;
it does not resolve schema extensions for each field or each call.

## JSON preparation

`json()` creates an immutable profile. `prepare(binding)` memoizes the combined
schema and physical plan on that codec:

```nupp
local codec = nupp.serde.json{
    unknownMembers = "ignore",
    fieldNames = function(member: nupp.serde.Member): string
        return member.name == "id" and "userId" or member.name
    end,
}
local prepared = codec:prepare(binding)
```

The field-name function runs during preparation, not once per value. For flat
scalar structures, the native plan retains pre-encoded output keys, compares
input key bytes directly, tracks required members, and traverses the complete
root in one native call. Known input keys are not materialized as Lua strings;
ignored values are validated and skipped without constructing a document.
`write(value, buffer)` appends the complete root to caller-owned storage in one
buffer operation; encoder scratch is pooled per worker thread.

Nested records, lists, maps, optionals, and documents use the same schema and
binding semantics and are traversed by the recursive prepared implementation.
Preparation remains the API boundary for adding more format-specific
optimizations without changing callers.

`nupp.codec.json.newCodec` is a compatibility entry point for the same codec.
`nupp.serde.json` is the typed primary API.

## Typed extensions

Schemas, bindings, and runtime reflection descriptors are extension hosts.
`nupp.reflect.extensionKey` creates a typed provider identity, and a host computes its
value once:

```nupp
local calls = 0
local displayName = nupp.reflect.extensionKey(function(schema: any): string
    calls = calls + 1
    return schema.name or "anonymous"
end)

local first = schema:extension(displayName)
local second = schema:extension(displayName)
assert(first == second and calls == 1)
```

Metadata is supplied by a model builder or derive. Extensions differ by
computing a derived value lazily from their host. Successful extension values
and failures are cached, and recursive initialization reports an error. An
extension key is an anonymous [`nupp.store.Key`](standard-library.md), and hosts
cache extension state by its id rather than by key-object identity. Ids are
acceleration values: their numbers may change with module initialization order
and are never persistent metadata identifiers. JSON uses schema extensions for
profile layouts and binding extensions for physical access, so format facts do
not leak into the logical schema.

## Persisting a store

A [`nupp.store.Store`](standard-library.md) holds live values under typed keys,
and its keys persist by name. `serde.key` declares a named key whose value
type comes from the binding that will carry it, so no annotation is needed,
and `saveStore` and `loadStore` move the store through plain values:

```nupp
const serde = nupp.serde

@derive(nupp.derive.Serde)
local record Settings
    volume: number
    fullscreen: boolean
end

local settings = serde.key("game.settings", serde.of(Settings)) -- Key<Settings>

local store = nupp.store.newStore()
store:set(settings, new Settings(volume = 0.5, fullscreen = false))
local saved = serde.saveStore(store)
assert(saved["game.settings"].volume == 0.5)
local text = nupp.codec.json.encode(saved)

local restored = nupp.store.newStore()
serde.loadStore(restored, nupp.codec.json.decode(text) as {[string]: any})
local back = restored:get(settings)
assert(back ~= nil and back.volume == 0.5)
```

`saveStore` walks the occupied keys and encodes each value through its
binding, under the key's name, into the document a JSON codec with default
field names would write. An occupied key that is anonymous or was declared
with `nupp.store.newKey` rather than `serde.key` raises, naming the key: a save
that silently drops state is worse than one that fails. `loadStore` looks each
name up in this runtime state's registry, raises for one that is not
registered or has no binding, decodes the value through that key's binding,
and writes the store. Keys the saved table does not mention are left as they
are. A value is therefore checked against the schema this program gave the
name, and a save written under another meaning of a name fails in decode
rather than loading as something else.

## Relationship to the JSON derive

`nupp.derive.JSON` remains available and retains its existing record methods
and `@json` policy. It is not silently redirected through Serde. Serde is the
language-wide abstraction for new codecs and dynamic clients; compatibility
derives can migrate only after their complete format behavior and diagnostics
have matching prepared implementations.

See [NEP 14](../../../neps/0014-schema-driven-serde.md) for the design reasoning and
the alternatives it rejected.