nupp.codec.valuebuilder

valuebuilder.newconstructor#

function valuebuilder.new(nullValue: any): any

Starts a direct stream of values.

This is the streaming shape. Open a container with nupp.codec.valuebuilder.openArray or nupp.codec.valuebuilder.openObject, add values to it, nupp.codec.valuebuilder.close it, and take the single root back from nupp.codec.valuebuilder.finish. A stream must publish exactly one root: adding a second value at the top level raises, and so does finishing with none.

The ordinary implementation keeps growable Lua state and is the aot = "off" oracle. VM-aware AOT keeps the same state in a bounded C stack object and roots every unfinished container on the Lua stack; it admits at most 1,024 open containers. Use nupp.codec.valuebuilder.newSized to choose that bound, and to reserve scratch for transformed strings, when the parser knows better.

local valuebuilder = nupp.codec.valuebuilder

--- Builds `[true, null]` without a source document.
--- @raises when the stream is left incomplete
@aot
local function pair(nullValue: any): any
    local builder = valuebuilder.new(nullValue)
    valuebuilder.openArray(builder, 2)
    valuebuilder.boolean(builder, true)
    valuebuilder.null(builder)
    valuebuilder.close(builder)

    return valuebuilder.finish(builder)
end

Arguments

Returns

valuebuilder.newByteScratchconstructor#

function valuebuilder.newByteScratch(capacity: uint32): any

Allocates a bounded byte work buffer local to one decode.

This is where a codec assembles bytes that are not a range of the source, an unescaped string or a decoded field, before publishing them. Write with nupp.codec.valuebuilder.setScratchByte, read with nupp.codec.valuebuilder.scratchByte, empty it between values with nupp.codec.valuebuilder.resetByteScratch, and publish a finished range with nupp.codec.valuebuilder.stringScratch or nupp.codec.valuebuilder.keyScratch.

When the bytes are already a contiguous range of the source, skip all of this and use nupp.codec.valuebuilder.string or nupp.codec.valuebuilder.key, which read the source in place.

Arguments

Returns

valuebuilder.newFixedByteScratchconstructor#

function valuebuilder.newFixedByteScratch(capacity: uint32): any

Allocates a fixed byte work buffer, every byte of it readable at once.

The byte counterpart of nupp.codec.valuebuilder.newFixedWordScratch, and the difference from nupp.codec.valuebuilder.newByteScratch is the same one. That buffer grows: a byte is readable once something has written it, so the bound every access is checked against is a length that moves. This one is zero from the moment it exists, so every byte below the capacity is readable and writable immediately, in any order.

Reach for it where the buffer is a block of a size the source knows -- a digest's final block, a fixed-width field, a lookup -- and for the appending one where the size is the input's. The checks are the same and refuse the same indexes; what changes is that the bound is a number rather than a field to load, and that an @aot entry can stand the storage on the C stack instead of allocating it. bench/sha256 measured those two together at about a hundred nanoseconds of a two-hundred-and-ninety nanosecond digest.

There is no emptying it: nupp.codec.valuebuilder.resetByteScratch refuses a fixed buffer, because a buffer with no fill state has none to discard.

An @aot entry requires the capacity to be a literal, since a bound that is not known is not one anything can reason about. Ordinary Nupp takes any.

Arguments

Returns

valuebuilder.newFixedWordScratchconstructor#

function valuebuilder.newFixedWordScratch(capacity: uint32): any

Allocates a fixed uint32 work buffer, every word of it readable at once.

The difference from nupp.codec.valuebuilder.newWordScratch is what "initialized" means. That buffer grows: a word is readable once something has written it, so reading ahead of the writes raises, and the bound every access is checked against is a length that moves. This one is zero from the moment it exists, so every word below the capacity is readable and writable immediately and the bound never moves.

Reach for it where the buffer is a table of a size the source knows -- a message schedule, a state vector, a lookup -- and for the appending one where the size is the input's. The checks are the same and refuse the same indexes; what changes is that a fixed buffer's bound is a number the C compiler has rather than a field it must load, which is what lets it discharge the comparison itself in a counted loop. bench/sha256 measures nupp.digest.internal.sha256 at 0.849x the hand-written C before and 0.963x after.

An @aot entry requires the capacity to be a literal, since a bound that is not known is not one anything can reason about. Ordinary Nupp takes any.

Arguments

Returns

valuebuilder.newPullconstructor#

function valuebuilder.newPull(nullValue: any, maxDepth: uint32, stringCapacity: uint32, arrayMarker: any, objectMarker: any, shape: any, arrayShapeMarker: any, serdeMarkers: any): any

Starts a bounded stream that materializes only one JSON pull shape.

This is an internal AOT boundary. The native builder still observes every parsed token so skipped subtrees are validated, but it allocates values only where shape selects them. arrayShapeMarker is the private key used by the JSON provider's nupp.codec.json.arrayOf shapes.

Arguments

Returns

Raises

• outside an AOT-compiled function

valuebuilder.newSerdeconstructor#

function valuebuilder.newSerde(nullValue: any, maxDepth: uint32, stringCapacity: uint32, arrayMarker: any, objectMarker: any, shape: any, arrayShapeMarker: any, serdeMarkers: any): any

Starts the schema-specialized form of a bounded native value stream.

This has the same source contract as nupp.codec.valuebuilder.newPull, but its distinct intrinsic identity lets AOT erase pull-only dispatch and emit serde stage two independently.

Arguments

Returns

Raises

• outside an AOT-compiled function

valuebuilder.newSizedconstructor#

function valuebuilder.newSized(nullValue: any, maxDepth: uint32, stringCapacity: uint32, arrayMarker: any?, objectMarker: any?): any

Starts a stream with authored bounds for native frame and transformed-byte scratch storage.

Same contract as nupp.codec.valuebuilder.new; only the storage differs. The ordinary implementation keeps normal growable Lua state and ignores both bounds, because plain Lua has nothing to preallocate. AOT uses them to allocate two rooted regions once: the first 16 frames stay inline and deeper streams lazily spill to Lua-rooted storage, and the byte region is allocated on the first escaped string and then reused. Publication still copies exactly once into a normal Lua string.

Reach for this over nupp.codec.valuebuilder.new when the document may nest deeper than 1,024 containers, or when the parser already knows the longest string it can produce and would rather not grow into it.

Arguments

Returns

valuebuilder.newWordScratchconstructor#

function valuebuilder.newWordScratch(capacity: uint32): any

Allocates a bounded uint32 work buffer local to one decode.

AOT lowers this to Lua-owned userdata, so its address remains stable across value allocations without crossing the public API as a pointer. Words are initialized by contiguous writes before they can be read: append or overwrite with nupp.codec.valuebuilder.setScratchWord, drain a SIMD mask into it with nupp.codec.valuebuilder.appendSetBits, and read it back with nupp.codec.valuebuilder.scratchWord. For transformed bytes rather than indexes, use nupp.codec.valuebuilder.newByteScratch. A zero-capacity scratch allocates no userdata and remains valid until an attempted read or write reports the ordinary bounds error.

Arguments

Returns

valuebuilder.appendSetBitsfunction#

function valuebuilder.appendSetBits(scratch: any, index: uint32, base: uint32, bits: any): uint32

Appends base + bit_index for every set bit in one 64-bit mask, low bit first, and returns the next unwritten index.

This is the structural-indexing step of a SIMD parser. A 64-byte block's interesting positions arrive as a nupp.simd.maskBits64, and this drains the whole mask into the buffer behind one capacity check rather than one check per bit. Feeding the result back as the next call's index walks the document a block at a time.

It has no ordinary implementation and raises if it is reached, because there is no ordinary way to hold the mask it takes. Call it only from an @aot function.

Arguments

Returns

Raises

• when called without AOT lowering or when the append exceeds scratch

valuebuilder.appendSetBitsEagerfunction#

function valuebuilder.appendSetBitsEager(scratch: any, index: uint32, base: uint32, bits: any): uint32

Eager-only form of nupp.codec.valuebuilder.appendSetBits, for a scratch whose back half is not shared with escape metadata.

Arguments

Returns

Raises

• when called without AOT lowering or when the append exceeds scratch

valuebuilder.appendStringBitsfunction#

function valuebuilder.appendStringBits(scratch: any, index: uint32, base: uint32, events: any, quotes: any, slashes: any, inString: boolean, stringEscaped: boolean): uint32

Appends structural and quote positions from one JSON cache line while marking closing quotes whose string contained a backslash.

The low 31 bits of the result are the next unwritten index. Bit 31 reports that the still-open final string contains a backslash, so the next block can carry that metadata without exposing a native pointer or a second return value. events, quotes, and slashes are masks over the same 64 bytes.

Arguments

Returns

Raises

• when called without AOT lowering or when the append exceeds scratch

valuebuilder.appendStringBitsSharedfunction#

function valuebuilder.appendStringBitsShared(scratch: any, index: uint32, base: uint32, events: any, quotes: any, slashes: any, inString: boolean, stringEscaped: boolean): uint32

Escape-scratch-sharing form of nupp.codec.valuebuilder.appendStringBits. It leaves room for positions growing backward from the same allocation.

Arguments

Returns

Raises

• when called without AOT lowering or when the append exceeds scratch

valuebuilder.appendStringEscapeBitsfunction#

function valuebuilder.appendStringEscapeBits(scratch: any, escapes: any, index: uint32, base: uint32, events: any, quotes: any, slashes: any, slashCarry: boolean, slashOdd: boolean, inString: boolean, stringEscaped: boolean): uint32

Appends events for a string that crosses this block boundary and retains the exact initiating backslash positions needed to unescape it later.

This is deliberately separate from nupp.codec.valuebuilder.appendStringBits: complete short strings stay on that smaller operation and pay no escape-tape cost. The parser calls this form only when a string begins before or ends after the block.

Arguments

Returns

Raises

• when called without AOT lowering or when either append exceeds its scratch

valuebuilder.booleanfunction#

function valuebuilder.boolean(builder: any, value: boolean): nil

Adds a boolean value.

Arguments

Returns

Raises

• when the value would be a second root, or an object value with no key

valuebuilder.bytefunction#

function valuebuilder.byte(borrows bytes: any, offset: uint32): uint32

Reads one byte at a zero-based offset.

The scanning counterpart of nupp.codec.valuebuilder.length; read whole words with nupp.codec.valuebuilder.word when the bytes are a packed side table rather than the document.

Arguments

Returns

Raises

• when offset is outside bytes

valuebuilder.byteAtfunction#

function valuebuilder.byteAt(borrows bytes: any, offset: uint32): uint32

Reads one byte under a dominating offset < length(bytes) proof.

Ordinary Lua performs the same checked read as nupp.codec.valuebuilder.byte. In an AOT entry the verifier requires this call to be inside the true arm or body of that exact bounds check, then emits a direct rooted-string load. Reach for it in a parser's established cursor loop; use byte everywhere else.

Arguments

Returns

Raises

• when offset is outside bytes outside AOT

valuebuilder.closefunction#

function valuebuilder.close(builder: any): nil

Closes the innermost open container and adds it to whatever encloses it.

Every nupp.codec.valuebuilder.openArray and nupp.codec.valuebuilder.openObject needs exactly one of these. Closing the outermost container publishes the root, which nupp.codec.valuebuilder.finish then returns.

Arguments

Returns

Raises

• when no container is open, or when an object key has no value

• when the closed container would be a second root

valuebuilder.countfunction#

function valuebuilder.count(builder: any): uint32

Returns the number of complete values in the current container.

Complete means a value has arrived, so an object with a key pending and no value yet still answers with the count before it.

Arguments

Returns

Raises

• when the stream has no current container

valuebuilder.decimal64function#

function valuebuilder.decimal64(builder: any, borrows source: any, start: uint32, length: uint32, magnitude: uint64, exponent: int32, negative: boolean, exact: boolean): nil

Publishes a decimal from the uint64 mantissa and base-ten exponent already accumulated by a codec. The native fast path handles Clinger's exact range; other inputs fall back to the authored source slice for correct rounding.

Arguments

Returns

Raises

• outside an AOT-compiled function or when the source range is invalid

valuebuilder.depthfunction#

function valuebuilder.depth(builder: any): uint32

Returns how many containers are currently open.

This and nupp.codec.valuebuilder.state let an iterative parser see the construction state; there is deliberately no way to read a value back out of a container it has already been added to.

Arguments

Returns

valuebuilder.finishfunction#

function valuebuilder.finish(builder: any): any

Ends the stream and returns its single root value.

valuebuilder.openArray(builder, 1)
valuebuilder.number(builder, 41)
valuebuilder.close(builder)
local result = valuebuilder.finish(builder)  -- {41}

Arguments

Returns

Raises

• when a container is still open, or when no root was published

valuebuilder.integer64function#

function valuebuilder.integer64(builder: any, magnitude: uint64, negative: boolean): nil

Publishes an integer magnitude accumulated in native uint64 arithmetic. Internal to AOT codecs that must avoid a decimal rescan.

Arguments

Returns

Raises

• outside an AOT-compiled function

valuebuilder.integerSlicefunction#

function valuebuilder.integerSlice(builder: any, borrows source: any, start: uint32, length: uint32): nil

Adds an integer token from a range of the rooted source.

The integer counterpart of nupp.codec.valuebuilder.numberSlice. AOT accumulates short integers directly without entering strtod, and falls back to the same checked binary64 conversion for longer tokens; the ordinary path retains tonumber as the behavioral oracle either way.

Arguments

Returns

Raises

• when the source range is not an integer token

• when the value would be a second root, or an object value with no key

valuebuilder.keyfunction#

function valuebuilder.key(builder: any, borrows source: any, start: uint32, length: uint32, escaped: boolean): nil

Sets the next object key from a range of the rooted source.

The range is read in place, so a key that holds no escape costs one copy and no substring. Pass escaped as true only when the range may hold backslash escapes; false takes the bytes exactly as they lie. Assemble a key that is not a contiguous source range with nupp.codec.valuebuilder.keyScratch instead.

Arguments

Returns

Raises

• when outside an object, or when a key is already pending

• when an escape in the range is malformed

valuebuilder.keyEscapesfunction#

function valuebuilder.keyEscapes(builder: any, borrows source: any, start: uint32, length: uint32, escapes: any, escapeStart: uint32, escapeCount: uint32): nil

Selects one escaped object key using retained backslash positions. Internal to AOT JSON stage two.

Arguments

Returns

Raises

• outside an AOT-compiled function

valuebuilder.keyScratchfunction#

function valuebuilder.keyScratch(builder: any, scratch: any, start: uint32, length: uint32): nil

Publishes initialized scratch bytes as the next object key.

The key-position counterpart of nupp.codec.valuebuilder.stringScratch. The very next value added becomes this key's value.

Arguments

Returns

Raises

• when outside an object, when a key is already pending, or when the range is not initialized

valuebuilder.kindfunction#

function valuebuilder.kind(builder: any): uint32

Returns the current container's tag: 5 for an array and 6 for an object.

Arguments

Returns

Raises

• when the stream has no current container

valuebuilder.lengthfunction#

function valuebuilder.length(borrows bytes: any): uint32

Returns the rooted string's byte length without making a substring.

This, nupp.codec.valuebuilder.byte, and nupp.codec.valuebuilder.word are the three readers that let an AOT parser read its input where it already lies. #bytes answers the same in ordinary Lua; going through here is what keeps the read admissible inside @aot.

Arguments

Returns

valuebuilder.nullfunction#

function valuebuilder.null(builder: any): nil

Adds the null replacement the stream was started with by nupp.codec.valuebuilder.new or nupp.codec.valuebuilder.newSized.

Arguments

Returns

Raises

• when the value would be a second root, or an object value with no key

valuebuilder.numberfunction#

function valuebuilder.number(builder: any, value: number): nil

Adds a number the parser has already converted.

Use nupp.codec.valuebuilder.numberSlice or nupp.codec.valuebuilder.integerSlice instead when the number is still a range of the source, so that the conversion happens without a substring.

Arguments

Returns

Raises

• when the value would be a second root, or an object value with no key

valuebuilder.numberSlicefunction#

function valuebuilder.numberSlice(builder: any, borrows source: any, start: uint32, length: uint32): nil

Adds a number token from a range of the rooted source, converting it in place.

This accepts any numeric token. When the token is known to be an integer, nupp.codec.valuebuilder.integerSlice says so and is cheaper.

Arguments

Returns

Raises

• when the source range is not a number token

• when the value would be a second root, or an object value with no key

valuebuilder.numberTokenfunction#

function valuebuilder.numberToken(builder: any, borrows source: any, start: uint32, limit: uint32): uint32

Parses and publishes one JSON number without returning to source-level byte loops. The low 31 bits are the stopping position; bit 31 says whether the token was valid. This is an AOT lowering boundary, not a portable parser API.

Arguments

Returns

Raises

• outside an @aot function

valuebuilder.openArrayfunction#

function valuebuilder.openArray(builder: any, capacity: uint32): nil

Opens an array. Every value added after it belongs to it until the matching nupp.codec.valuebuilder.close.

capacity is a presizing hint taken from what the parser already counted, not a limit: a wrong guess costs a rehash, never an error.

Arguments

Returns

valuebuilder.openObjectfunction#

function valuebuilder.openObject(builder: any, capacity: uint32): nil

Opens an object. Each value inside it must be preceded by a key from nupp.codec.valuebuilder.key or nupp.codec.valuebuilder.keyScratch, until the matching nupp.codec.valuebuilder.close.

capacity presizes the hash part and is a hint, as in nupp.codec.valuebuilder.openArray.

Arguments

Returns

valuebuilder.resetByteScratchfunction#

function valuebuilder.resetByteScratch(scratch: any): nil

Makes a byte scratch buffer empty without reallocating it.

Call this between values so that one bounded allocation serves a whole decode. Bytes below the old length remain in memory but are no longer readable, since nupp.codec.valuebuilder.scratchByte admits only initialized indexes.

Arguments

Returns

Raises

• when the buffer is a fixed one, which has no fill state to discard

valuebuilder.scratchBytefunction#

function valuebuilder.scratchByte(scratch: any, index: uint32): uint32

Reads one initialized byte from a local work buffer.

Arguments

Returns

Raises

• when the indexed byte has not been initialized

valuebuilder.scratchEscapeLengthfunction#

function valuebuilder.scratchEscapeLength(scratch: any): uint32

Answers how many escape positions the structural scan retained.

Arguments

Returns

Raises

• outside an AOT-compiled function

valuebuilder.scratchEscapeWordfunction#

function valuebuilder.scratchEscapeWord(scratch: any, index: uint32): uint32

Reads one retained escape position in source order.

Arguments

Returns

Raises

• outside an AOT-compiled function

valuebuilder.scratchLengthfunction#

function valuebuilder.scratchLength(scratch: any): uint32

Answers the initialized word count of a local work buffer.

Arguments

Returns

valuebuilder.scratchWordfunction#

function valuebuilder.scratchWord(scratch: any, index: uint32): uint32

Reads one word from a local work buffer.

Only initialized words can be read, so an index at or past the buffer's current length raises rather than answering whatever was there.

Arguments

Returns

Raises

• when the indexed word has not been initialized

valuebuilder.setScratchBytefunction#

function valuebuilder.setScratchByte(scratch: any, index: uint32, value: uint32): nil

Writes an existing byte or appends the next contiguous byte.

Appends and grows by one when index is the current length, overwrites below it, on the same terms as nupp.codec.valuebuilder.setScratchWord.

Arguments

Returns

Raises

• when the write leaves a gap, exceeds capacity, or is not a byte

valuebuilder.setScratchBytes4function#

function valuebuilder.setScratchBytes4(scratch: any, index: uint32, value: uint32): nil

Writes four bytes at once, least significant first.

The counterpart of nupp.codec.valuebuilder.word on the writing side, and the reason it exists: a codec that produces bytes in groups pays one bounds check and one store per group rather than four of each. bench/base64 measured the byte-at-a-time path at over half of its encode.

The order is little-endian and stated rather than native, unlike the read: a defined order is one a plain-Lua implementation can honour without knowing the machine, and on every target Nupp has it is what a native store already does. index is a byte index, so the four bytes land at index through index + 3.

Appends and grows by four when index is the current length, overwrites below it, and refuses a write that would straddle the two.

Arguments

Returns

Raises

• when the write leaves a gap, straddles the length, or exceeds capacity

valuebuilder.setScratchWordfunction#

function valuebuilder.setScratchWord(scratch: any, index: uint32, value: uint32): nil

Writes an existing word or appends the next contiguous word.

Writing at the buffer's current length appends and grows it by one; writing below that overwrites. There is no way to write past the end, which is what makes "initialized" mean the same thing as "below the length" for nupp.codec.valuebuilder.scratchWord.

Arguments

Returns

Raises

• when the index leaves a gap or exceeds the capacity

valuebuilder.statefunction#

function valuebuilder.state(builder: any): uint32

Returns the current container kind and whether it has a complete value in one query. Zero means no container is open. Otherwise the low byte is 5 for an array or 6 for an object, and bit 0x100 is set after its first value.

This is the delimiter-loop shape of nupp.codec.valuebuilder.depth, nupp.codec.valuebuilder.kind, and nupp.codec.valuebuilder.count. It avoids making a native parser cross the builder boundary three times when it only needs to distinguish the root, an empty container, and a nonempty container.

Arguments

Returns

valuebuilder.stringfunction#

function valuebuilder.string(builder: any, borrows source: any, start: uint32, length: uint32, escaped: boolean): nil

Adds a string value from a range of the rooted source.

The value counterpart of nupp.codec.valuebuilder.key, on the same terms: the range is read in place, and escaped says whether it needs decoding first.

Arguments

Returns

Raises

• when an escape in the range is malformed

• when the value would be a second root, or an object value with no key

valuebuilder.stringEscapesfunction#

function valuebuilder.stringEscapes(builder: any, borrows source: any, start: uint32, length: uint32, escapes: any, escapeStart: uint32, escapeCount: uint32): nil

Publishes one escaped source slice using backslash positions retained by the structural scan. Internal to AOT JSON stage two.

Arguments

Returns

Raises

• outside an AOT-compiled function

valuebuilder.stringScratchfunction#

function valuebuilder.stringScratch(builder: any, scratch: any, start: uint32, length: uint32): nil

Publishes initialized scratch bytes as a normal Lua string value.

The copy into Lua-owned storage happens here, exactly once. Use nupp.codec.valuebuilder.keyScratch for the object-key position instead, and nupp.codec.valuebuilder.string when the bytes are already a range of the source and need no assembling.

Arguments

Returns

Raises

• when the range is not initialized

• when the value would be a second root, or an object value with no key

valuebuilder.wordfunction#

function valuebuilder.word(bytes: string, index: uint32): uint32

Reads one native-endian uint32 at a zero-based word index.

This is how a parser reads a packed side table, a structural index or a tape, back out of a rooted string instead of a Lua array of numbers. The word at index begins at byte offset index * 4.

Arguments

Returns

Raises

• when the complete word is outside bytes