`nupp.gpu`

Resident compute buffers over the selected GPU provider.

The context owns every buffer and compiled kernel created from it. Buffers and kernels borrow that context, so checked code cannot close the device while one is live. Uploads, dispatches, and downloads only enqueue commands; synchronize() is the explicit CPU boundary.

local gpu = nupp.gpu
local kernels = require("kernels")
local ffi = require("ffi")

local context = gpu.open()
local input = context:buffer(ffi.typeof<float>(), 1024)
local output = context:buffer(ffi.typeof<float>(), 1024)
local scale = kernels.scale:compile(context):bind(output, input)
context:upload(input, values:read())
scale:dispatch(2.0)
context:enqueueDownload(output)
context:synchronize()
context:readDownloaded(output, result:write())

Here kernels.scale is generated from an @aot(target = "gpu") function by a target with aot = "require". Shader source, entrypoint identity, binding order and uniform packing remain compiler details. Ordinary spans are the CPU boundary; a chain of kernels keeps its intermediate buffers resident and pays upload/download only where the program asks for them.

context:tensor(element, shape) allocates dense row-major storage. A checked gpu.Layout can be sliced, transposed, broadcast, or explicitly strided, then applied with gpu.view(buffer, layout) without allocating. Transfers remain dense; layout-aware kernels consume the explicit metadata.

Submodules

Module	Description
`nupp.gpu.layout`

Module contents

Types

Type	Kind	Description
`Buffer`	interface	Typed resident device storage or a view borrowing its owning context.
`Context`	interface	Device operations shared by application and provider contexts.
`ContextToken`	interface	Canonical non-suspending teardown obligation for a device context.
`Layout`	record
`Phases`	interface
`Shared`	interface

Functions

Function	Kind	Description
`bufferIsDense`	function
`bufferIsInjective`	function
`bufferLayout`	function
`open`	function	Opens a compute device through the selected provider.
`view`	function
`workgroups`	function	Executes uniformly ordered workgroup phases on the CPU.

Values

Value	Kind	Description
`destroyContext`	variable
`PORTABLE_SCRATCH_BYTES`	variable
`PORTABLE_WORKGROUP_THREADS`	variable

Types#

`Buffer`interface#

interface Buffer<T>
    readonly count: integer
    readonly dimensions: function(borrows self: Buffer<T>): {integer}
    readonly strides: function(borrows self: Buffer<T>): {integer}
    readonly subview: function(
        borrows self: Buffer<T>,
        origin: {integer},
        shape: {integer}
    ): Buffer<T> borrows (self)
    readonly layout: function(borrows self: Buffer<T>): TensorLayout
    readonly view: function(borrows self: Buffer<T>, layout: TensorLayout): Buffer<T> borrows (self)
end

Typed resident device storage or a view borrowing its owning context.

Type parameters

Name	Description
`T`

Methods

`dimensions`#

dimensions: function(borrows self: gpu.Buffer<T>): {integer}

Returns the logical shape in elements.

Arguments

Name	Type	Description
`borrows self`	`gpu.Buffer<T>`

Returns

Type	Description
`{integer}`

`strides`#

strides: function(borrows self: gpu.Buffer<T>): {integer}

Returns the element strides for the logical view.

Arguments

Name	Type	Description
`borrows self`	`gpu.Buffer<T>`

Returns

Type	Description
`{integer}`

`subview`#

subview: function(
    borrows self: gpu.Buffer<T>,
    origin: {integer},
    shape: {integer}
): gpu.Buffer<T> borrows (self)

Creates a context-borrowed view at a zero-based origin with the requested shape.

Arguments

Name	Type	Description
`borrows self`	`gpu.Buffer<T>`
`origin`	`{integer}`
`shape`	`{integer}`

Returns

Type	Description
`gpu.Buffer<T> borrows (self)`

`layout`#

layout: function(borrows self: gpu.Buffer<T>): TensorLayout

Returns the canonical tensor layout for this view.

Arguments

Name	Type	Description
`borrows self`	`gpu.Buffer<T>`

Returns

Type	Description
`TensorLayout`

`view`#

view: function(borrows self: gpu.Buffer<T>, layout: TensorLayout): gpu.Buffer<T> borrows (self)

Creates a view using the supplied checked layout without copying storage.

Arguments

Name	Type	Description
`borrows self`	`gpu.Buffer<T>`
`layout`	`TensorLayout`

Returns

Type	Description
`gpu.Buffer<T> borrows (self)`

Fields

`count`#

count: integer

Number of elements, kept with the allocation.

`Context`interface#

interface Context is gpu.ContextToken
    readonly drop: nosuspend function(takes self: Context): nil
    readonly driver: function(borrows self: Context): string
    readonly buffer: function<T>(
        borrows self: Context,
        element: ctype<T>,
        count: integer
    ): gpu.Buffer<T> borrows (self)
    readonly tensor: function<T>(
        borrows self: Context,
        element: ctype<T>,
        shape: {integer}
    ): gpu.Buffer<T> borrows (self)
    readonly releaseBuffer: function<T>(borrows self: Context, borrows buffer: gpu.Buffer<T>): nil
    readonly upload: function<T>(borrows self: Context, borrows buffer: gpu.Buffer<T>, borrows source: Span<T>): nil
    readonly enqueueDownload: function<T>(borrows self: Context, borrows buffer: gpu.Buffer<T>): nil
    readonly synchronize: function(borrows self: Context): nil
    readonly readDownloaded: function<T>(
        borrows self: Context,
        borrows buffer: gpu.Buffer<T>,
        exclusive destination: WriteSpan<T>
    ): nil
    readonly download: function<T>(
        borrows self: Context,
        borrows buffer: gpu.Buffer<T>,
        exclusive destination: WriteSpan<T>
    ): nil
end

Device operations shared by application and provider contexts.

Methods

`drop`#

drop: nosuspend function(takes self: gpu.Context): nil

Consumes the context and releases all device resources without suspending.

Arguments

Name	Type	Description
`takes self`	`gpu.Context`

Returns

Type	Description
`nil`

`driver`#

driver: function(borrows self: gpu.Context): string

Returns a diagnostic name for the selected device implementation.

Arguments

Name	Type	Description
`borrows self`	`gpu.Context`

Returns

Type	Description
`string`

`buffer`#

buffer: function<T>(
    borrows self: gpu.Context,
    element: ctype<T>,
    count: integer
): gpu.Buffer<T> borrows (self)

Allocates a typed resident device buffer.

Arguments

Name	Type	Description
`borrows self`	`gpu.Context`
`element`	`ctype<T>`
`count`	`integer`

Returns

Type	Description
`gpu.Buffer<T> borrows (self)`

`tensor`#

tensor: function<T>(
    borrows self: gpu.Context,
    element: ctype<T>,
    shape: {integer}
): gpu.Buffer<T> borrows (self)

Allocates typed storage with the requested logical shape.

Arguments

Name	Type	Description
`borrows self`	`gpu.Context`
`element`	`ctype<T>`
`shape`	`{integer}`

Returns

Type	Description
`gpu.Buffer<T> borrows (self)`

`releaseBuffer`#

releaseBuffer: function<T>(borrows self: gpu.Context, borrows buffer: gpu.Buffer<T>): nil

Releases the device allocation represented by buffer. The caller must not use it or its dependent views afterward.

Arguments

Name	Type	Description
`borrows self`	`gpu.Context`
`borrows buffer`	`gpu.Buffer<T>`

Returns

Type	Description
`nil`

`upload`#

upload: function<T>(borrows self: gpu.Context, borrows buffer: gpu.Buffer<T>, borrows source: Span<T>): nil

Copies borrowed host-span data into the resident buffer.

Arguments

Name	Type	Description
`borrows self`	`gpu.Context`
`borrows buffer`	`gpu.Buffer<T>`
`borrows source`	`Span<T>`

Returns

Type	Description
`nil`

`enqueueDownload`#

enqueueDownload: function<T>(borrows self: gpu.Context, borrows buffer: gpu.Buffer<T>): nil

Queues a buffer readback for subsequent synchronization and reading.

Arguments

Name	Type	Description
`borrows self`	`gpu.Context`
`borrows buffer`	`gpu.Buffer<T>`

Returns

Type	Description
`nil`

`synchronize`#

synchronize: function(borrows self: gpu.Context): nil

Waits for queued device work and readbacks to complete.

Arguments

Name	Type	Description
`borrows self`	`gpu.Context`

Returns

Type	Description
`nil`

`readDownloaded`#

readDownloaded: function<T>(
    borrows self: gpu.Context,
    borrows buffer: gpu.Buffer<T>,
    exclusive destination: WriteSpan<T>
): nil

Copies a completed readback into an exclusively borrowed destination span.

Arguments

Name	Type	Description
`borrows self`	`gpu.Context`
`borrows buffer`	`gpu.Buffer<T>`
`exclusive destination`	`WriteSpan<T>`

Returns

Type	Description
`nil`

`download`#

download: function<T>(
    borrows self: gpu.Context,
    borrows buffer: gpu.Buffer<T>,
    exclusive destination: WriteSpan<T>
): nil

Performs readback, synchronization, and copying into the exclusive destination.

Arguments

Name	Type	Description
`borrows self`	`gpu.Context`
`borrows buffer`	`gpu.Buffer<T>`
`exclusive destination`	`WriteSpan<T>`

Returns

Type	Description
`nil`

`ContextToken`interface#

interface ContextToken
    readonly drop: nosuspend function(takes self: ContextToken): nil
end

Canonical non-suspending teardown obligation for a device context.

Methods

`drop`#

drop: nosuspend function(takes self: ContextToken): nil

Consumes and releases the context without suspending.

Arguments

Name	Type	Description
`takes self`	`ContextToken`

Returns

Type	Description
`nil`

`Layout`record#

record Layout

    readonly count: integer
end

Fields

`count`#

count: integer

`Phases`interface#

interface Phases
    readonly scratch: function<T>(borrows self: Phases, initial: T, count: integer): gpu.Shared<T>
    readonly run: function(borrows self: Phases, scoped stage: function(uint32): nil): nil
    readonly reduceSumF32: function(borrows self: Phases, exclusive values: gpu.Shared<float>): nil
    readonly inclusiveScanU32: function(
        borrows self: Phases,
        exclusive values: gpu.Shared<uint32>,
        exclusive temporary: gpu.Shared<uint32>
    ): nil
end

Methods

`scratch`#

scratch: function<T>(borrows self: gpu.Phases, initial: T, count: integer): gpu.Shared<T>

Allocates fresh zero-based scratch for this workgroup.

Arguments

Name	Type	Description
`borrows self`	`gpu.Phases`
`initial`	`T`
`count`	`integer`

Returns

Type	Description
`gpu.Shared<T>`

`run`#

run: function(borrows self: gpu.Phases, scoped stage: function(uint32): nil): nil

Runs one stage to completion in ascending local-index order.

Arguments

Name	Type	Description
`borrows self`	`gpu.Phases`
`scoped stage`	`function(uint32): nil`

Returns

Type	Description
`nil`

`reduceSumF32`#

reduceSumF32: function(borrows self: gpu.Phases, exclusive values: gpu.Shared<float>): nil

Reduces one power-of-two f32 workgroup in a fixed left-before-right tree. The sum is left in element zero.

Arguments

Name	Type	Description
`borrows self`	`gpu.Phases`
`exclusive values`	`gpu.Shared<float>`

Returns

Type	Description
`nil`

`inclusiveScanU32`#

inclusiveScanU32: function(
    borrows self: gpu.Phases,
    exclusive values: gpu.Shared<uint32>,
    exclusive temporary: gpu.Shared<uint32>
): nil

Computes one deterministic inclusive u32 prefix sum. temporary is a same-sized scratch array used for disjoint ping-pong stages; the result is left in values.

Arguments

Name	Type	Description
`borrows self`	`gpu.Phases`
`exclusive values`	`gpu.Shared<uint32>`
`exclusive temporary`	`gpu.Shared<uint32>`

Returns

Type	Description
`nil`

`Shared`interface#

interface Shared<T>
    readonly count: integer

    metamethod __len: function(borrows self: Shared<T>): integer
    metamethod __index: function(borrows self: Shared<T>, index: integer): T
    metamethod __newindex: function(exclusive self: Shared<T>, index: integer, value: T): nil
end

Type parameters

Name	Description
`T`

Methods

`__len`#

__len: function(borrows self: gpu.Shared<T>): integer

Arguments

Name	Type	Description
`borrows self`	`gpu.Shared<T>`

Returns

Type	Description
`integer`

`__index`#

__index: function(borrows self: gpu.Shared<T>, index: integer): T

Arguments

Name	Type	Description
`borrows self`	`gpu.Shared<T>`
`index`	`integer`

Returns

Type	Description
`T`

`__newindex`#

__newindex: function(exclusive self: gpu.Shared<T>, index: integer, value: T): nil

Arguments

Name	Type	Description
`exclusive self`	`gpu.Shared<T>`
`index`	`integer`
`value`	`T`

Returns

Type	Description
`nil`

Fields

`count`#

count: integer

Functions#

`bufferIsDense`function#

function bufferIsDense<T>(borrows buffer: api.Buffer<T>): boolean

Type parameters

Name	Description
`T`

Arguments

Name	Type	Description
`borrows buffer`	`api.Buffer<T>`

Returns

Type	Description
`boolean`

`bufferIsInjective`function#

function bufferIsInjective<T>(borrows buffer: api.Buffer<T>): boolean

Type parameters

Name	Description
`T`

Arguments

Name	Type	Description
`borrows buffer`	`api.Buffer<T>`

Returns

Type	Description
`boolean`

`bufferLayout`function#

function bufferLayout<T>(borrows buffer: api.Buffer<T>): tensorlayout.Layout

Type parameters

Name	Description
`T`

Arguments

Name	Type	Description
`borrows buffer`	`api.Buffer<T>`

Returns

Type	Description
`tensorlayout.Layout`

`open`function#

const open: function(): affine(api.Context, api.destroyContext)

Opens a compute device through the selected provider.

Returns

Type	Description
`affine(api.Context, api.destroyContext)`

`view`function#

function view<T>(borrows buffer: api.Buffer<T>, layout: tensorlayout.Layout): api.Buffer<T>

Type parameters

Name	Description
`T`

Arguments

Name	Type	Description
`borrows buffer`	`api.Buffer<T>`
`layout`	`tensorlayout.Layout`

Returns

Type	Description
`api.Buffer<T>`

`workgroups`function#

function workgroups(groups: uint32, size: uint32, scoped controller: function(uint32, gpu.Phases): nil): nil

Executes uniformly ordered workgroup phases on the CPU.

A GPU AOT body recognizes this call and its immediate callbacks as structure: the controller does not escape, each run is a device barrier, and each stage invocation becomes one local lane.

Arguments

Name	Type	Description
`groups`	`uint32`
`size`	`uint32`
`scoped controller`	`function(uint32, gpu.Phases): nil`

Returns

Type	Description
`nil`

Raises

when size is outside the portable workgroup limit

Values#

`destroyContext`variable#

const destroyContext

`PORTABLE_SCRATCH_BYTES`variable#

const PORTABLE_SCRATCH_BYTES

`PORTABLE_WORKGROUP_THREADS`variable#

const PORTABLE_WORKGROUP_THREADS

nupp.gpu

Submodules

Module contents

Types

Functions

Values

Types#

Bufferinterface#

Type parameters

Methods

dimensions#

Arguments

Returns

strides#

Arguments

Returns

subview#

Arguments

Returns

layout#

Arguments

Returns

view#

Arguments

Returns

Fields

count#

Contextinterface#

Methods

drop#

Arguments

Returns

driver#

Arguments

Returns

buffer#

Arguments

Returns

tensor#

Arguments

Returns

releaseBuffer#

Arguments

Returns

upload#

Arguments

Returns

enqueueDownload#

Arguments

Returns

synchronize#

Arguments

Returns

readDownloaded#

Arguments

Returns

download#

Arguments

Returns

ContextTokeninterface#

Methods

drop#

Arguments

Returns

Layoutrecord#

Fields

count#

Phasesinterface#

Methods

scratch#

Arguments

Returns

run#

Arguments

Returns

reduceSumF32#

Arguments

Returns

inclusiveScanU32#

Arguments

`nupp.gpu`

`Buffer`interface#

`dimensions`#

`strides`#

`subview`#

`layout`#

`view`#

`count`#

`Context`interface#

`drop`#

`driver`#

`buffer`#

`tensor`#

`releaseBuffer`#

`upload`#

`enqueueDownload`#

`synchronize`#

`readDownloaded`#

`download`#

`ContextToken`interface#

`drop`#

`Layout`record#

`count`#

`Phases`interface#

`scratch`#

`run`#

`reduceSumF32`#

`inclusiveScanU32`#

`Shared`interface#

`__len`#

`__index`#

`__newindex`#

`count`#

`bufferIsDense`function#

`bufferIsInjective`function#

`bufferLayout`function#

`open`function#

`view`function#

`workgroups`function#

`destroyContext`variable#

`PORTABLE_SCRATCH_BYTES`variable#

`PORTABLE_WORKGROUP_THREADS`variable#