# `nupp.gpu` Resident compute buffers over the selected GPU provider. The context owns every buffer and compiled kernel created from it. Buffers and kernels borrow that context, so checked code cannot close the device while one is live. Uploads, dispatches, and downloads only enqueue commands; `synchronize()` is the explicit CPU boundary. ```nupp local gpu = nupp.gpu local kernels = require("kernels") local ffi = require("ffi") local context = gpu.open() local input = context:buffer(ffi.typeof(), 1024) local output = context:buffer(ffi.typeof(), 1024) local scale = kernels.scale:compile(context):bind(output, input) context:upload(input, values:read()) scale:dispatch(2.0) context:enqueueDownload(output) context:synchronize() context:readDownloaded(output, result:write()) ``` Here `kernels.scale` is generated from an `@aot(target = "gpu")` function by a target with `aot = "require"`. Shader source, entrypoint identity, binding order and uniform packing remain compiler details. Ordinary spans are the CPU boundary; a chain of kernels keeps its intermediate buffers resident and pays upload/download only where the program asks for them. `context:tensor(element, shape)` allocates dense row-major storage. A checked `gpu.Layout` can be sliced, transposed, broadcast, or explicitly strided, then applied with `gpu.view(buffer, layout)` without allocating. Transfers remain dense; layout-aware kernels consume the explicit metadata. ## Submodules | Module | Description | | --- | --- | | `nupp.gpu.layout` | | ## Types ### `Buffer` _interface_ ```nupp interface Buffer readonly count: integer readonly dimensions: function(borrows self: Buffer): {integer} readonly strides: function(borrows self: Buffer): {integer} readonly subview: function( borrows self: Buffer, origin: {integer}, shape: {integer} ): Buffer borrows (self) readonly layout: function(borrows self: Buffer): TensorLayout readonly view: function(borrows self: Buffer, layout: TensorLayout): Buffer borrows (self) end ``` Typed resident device storage or a view borrowing its owning context. #### Type parameters | Name | Description | | --- | --- | | `T` | | #### Methods ##### `dimensions` ```nupp dimensions: function(borrows self: gpu.Buffer): {integer} ``` Returns the logical shape in elements. ###### Arguments | Name | Type | Description | | --- | --- | --- | | `borrows self` | `gpu.Buffer\` | | ###### Returns | Type | Description | | --- | --- | | `{integer}` | | ##### `strides` ```nupp strides: function(borrows self: gpu.Buffer): {integer} ``` Returns the element strides for the logical view. ###### Arguments | Name | Type | Description | | --- | --- | --- | | `borrows self` | `gpu.Buffer\` | | ###### Returns | Type | Description | | --- | --- | | `{integer}` | | ##### `subview` ```nupp subview: function( borrows self: gpu.Buffer, origin: {integer}, shape: {integer} ): gpu.Buffer borrows (self) ``` Creates a context-borrowed view at a zero-based origin with the requested shape. ###### Arguments | Name | Type | Description | | --- | --- | --- | | `borrows self` | `gpu.Buffer\` | | | `origin` | `{integer}` | | | `shape` | `{integer}` | | ###### Returns | Type | Description | | --- | --- | | `gpu.Buffer\ borrows (self)` | | ##### `layout` ```nupp layout: function(borrows self: gpu.Buffer): TensorLayout ``` Returns the canonical tensor layout for this view. ###### Arguments | Name | Type | Description | | --- | --- | --- | | `borrows self` | `gpu.Buffer\` | | ###### Returns | Type | Description | | --- | --- | | `TensorLayout` | | ##### `view` ```nupp view: function(borrows self: gpu.Buffer, layout: TensorLayout): gpu.Buffer borrows (self) ``` Creates a view using the supplied checked layout without copying storage. ###### Arguments | Name | Type | Description | | --- | --- | --- | | `borrows self` | `gpu.Buffer\` | | | `layout` | `TensorLayout` | | ###### Returns | Type | Description | | --- | --- | | `gpu.Buffer\ borrows (self)` | | #### Fields ##### `count` ```nupp count: integer ``` Number of elements, kept with the allocation. ### `Context` _interface_ ```nupp interface Context is gpu.ContextToken readonly drop: nosuspend function(takes self: Context): nil readonly driver: function(borrows self: Context): string readonly buffer: function( borrows self: Context, element: ctype, count: integer ): gpu.Buffer borrows (self) readonly tensor: function( borrows self: Context, element: ctype, shape: {integer} ): gpu.Buffer borrows (self) readonly releaseBuffer: function(borrows self: Context, borrows buffer: gpu.Buffer): nil readonly upload: function(borrows self: Context, borrows buffer: gpu.Buffer, borrows source: Span): nil readonly enqueueDownload: function(borrows self: Context, borrows buffer: gpu.Buffer): nil readonly synchronize: function(borrows self: Context): nil readonly readDownloaded: function( borrows self: Context, borrows buffer: gpu.Buffer, exclusive destination: WriteSpan ): nil readonly download: function( borrows self: Context, borrows buffer: gpu.Buffer, exclusive destination: WriteSpan ): nil end ``` Device operations shared by application and provider contexts. #### Methods ##### `drop` ```nupp drop: nosuspend function(takes self: gpu.Context): nil ``` Consumes the context and releases all device resources without suspending. ###### Arguments | Name | Type | Description | | --- | --- | --- | | `takes self` | `gpu.Context` | | ###### Returns | Type | Description | | --- | --- | | `nil` | | ##### `driver` ```nupp driver: function(borrows self: gpu.Context): string ``` Returns a diagnostic name for the selected device implementation. ###### Arguments | Name | Type | Description | | --- | --- | --- | | `borrows self` | `gpu.Context` | | ###### Returns | Type | Description | | --- | --- | | `string` | | ##### `buffer` ```nupp buffer: function( borrows self: gpu.Context, element: ctype, count: integer ): gpu.Buffer borrows (self) ``` Allocates a typed resident device buffer. ###### Arguments | Name | Type | Description | | --- | --- | --- | | `borrows self` | `gpu.Context` | | | `element` | `ctype\` | | | `count` | `integer` | | ###### Returns | Type | Description | | --- | --- | | `gpu.Buffer\ borrows (self)` | | ##### `tensor` ```nupp tensor: function( borrows self: gpu.Context, element: ctype, shape: {integer} ): gpu.Buffer borrows (self) ``` Allocates typed storage with the requested logical shape. ###### Arguments | Name | Type | Description | | --- | --- | --- | | `borrows self` | `gpu.Context` | | | `element` | `ctype\` | | | `shape` | `{integer}` | | ###### Returns | Type | Description | | --- | --- | | `gpu.Buffer\ borrows (self)` | | ##### `releaseBuffer` ```nupp releaseBuffer: function(borrows self: gpu.Context, borrows buffer: gpu.Buffer): nil ``` Releases the device allocation represented by buffer. The caller must not use it or its dependent views afterward. ###### Arguments | Name | Type | Description | | --- | --- | --- | | `borrows self` | `gpu.Context` | | | `borrows buffer` | `gpu.Buffer\` | | ###### Returns | Type | Description | | --- | --- | | `nil` | | ##### `upload` ```nupp upload: function(borrows self: gpu.Context, borrows buffer: gpu.Buffer, borrows source: Span): nil ``` Copies borrowed host-span data into the resident buffer. ###### Arguments | Name | Type | Description | | --- | --- | --- | | `borrows self` | `gpu.Context` | | | `borrows buffer` | `gpu.Buffer\` | | | `borrows source` | `Span\` | | ###### Returns | Type | Description | | --- | --- | | `nil` | | ##### `enqueueDownload` ```nupp enqueueDownload: function(borrows self: gpu.Context, borrows buffer: gpu.Buffer): nil ``` Queues a buffer readback for subsequent synchronization and reading. ###### Arguments | Name | Type | Description | | --- | --- | --- | | `borrows self` | `gpu.Context` | | | `borrows buffer` | `gpu.Buffer\` | | ###### Returns | Type | Description | | --- | --- | | `nil` | | ##### `synchronize` ```nupp synchronize: function(borrows self: gpu.Context): nil ``` Waits for queued device work and readbacks to complete. ###### Arguments | Name | Type | Description | | --- | --- | --- | | `borrows self` | `gpu.Context` | | ###### Returns | Type | Description | | --- | --- | | `nil` | | ##### `readDownloaded` ```nupp readDownloaded: function( borrows self: gpu.Context, borrows buffer: gpu.Buffer, exclusive destination: WriteSpan ): nil ``` Copies a completed readback into an exclusively borrowed destination span. ###### Arguments | Name | Type | Description | | --- | --- | --- | | `borrows self` | `gpu.Context` | | | `borrows buffer` | `gpu.Buffer\` | | | `exclusive destination` | `WriteSpan\` | | ###### Returns | Type | Description | | --- | --- | | `nil` | | ##### `download` ```nupp download: function( borrows self: gpu.Context, borrows buffer: gpu.Buffer, exclusive destination: WriteSpan ): nil ``` Performs readback, synchronization, and copying into the exclusive destination. ###### Arguments | Name | Type | Description | | --- | --- | --- | | `borrows self` | `gpu.Context` | | | `borrows buffer` | `gpu.Buffer\` | | | `exclusive destination` | `WriteSpan\` | | ###### Returns | Type | Description | | --- | --- | | `nil` | | ### `ContextToken` _interface_ ```nupp interface ContextToken readonly drop: nosuspend function(takes self: ContextToken): nil end ``` Canonical non-suspending teardown obligation for a device context. #### Methods ##### `drop` ```nupp drop: nosuspend function(takes self: ContextToken): nil ``` Consumes and releases the context without suspending. ###### Arguments | Name | Type | Description | | --- | --- | --- | | `takes self` | `ContextToken` | | ###### Returns | Type | Description | | --- | --- | | `nil` | | ### `Layout` _record_ ```nupp record Layout readonly count: integer end ``` #### Fields ##### `count` ```nupp count: integer ``` ### `Phases` _interface_ ```nupp interface Phases readonly scratch: function(borrows self: Phases, initial: T, count: integer): gpu.Shared readonly run: function(borrows self: Phases, scoped stage: function(uint32): nil): nil readonly reduceSumF32: function(borrows self: Phases, exclusive values: gpu.Shared): nil readonly inclusiveScanU32: function( borrows self: Phases, exclusive values: gpu.Shared, exclusive temporary: gpu.Shared ): nil end ``` #### Methods ##### `scratch` ```nupp scratch: function(borrows self: gpu.Phases, initial: T, count: integer): gpu.Shared ``` Allocates fresh zero-based scratch for this workgroup. ###### Arguments | Name | Type | Description | | --- | --- | --- | | `borrows self` | `gpu.Phases` | | | `initial` | `T` | | | `count` | `integer` | | ###### Returns | Type | Description | | --- | --- | | `gpu.Shared\` | | ##### `run` ```nupp run: function(borrows self: gpu.Phases, scoped stage: function(uint32): nil): nil ``` Runs one stage to completion in ascending local-index order. ###### Arguments | Name | Type | Description | | --- | --- | --- | | `borrows self` | `gpu.Phases` | | | `scoped stage` | `function(uint32): nil` | | ###### Returns | Type | Description | | --- | --- | | `nil` | | ##### `reduceSumF32` ```nupp reduceSumF32: function(borrows self: gpu.Phases, exclusive values: gpu.Shared): nil ``` Reduces one power-of-two f32 workgroup in a fixed left-before-right tree. The sum is left in element zero. ###### Arguments | Name | Type | Description | | --- | --- | --- | | `borrows self` | `gpu.Phases` | | | `exclusive values` | `gpu.Shared\` | | ###### Returns | Type | Description | | --- | --- | | `nil` | | ##### `inclusiveScanU32` ```nupp inclusiveScanU32: function( borrows self: gpu.Phases, exclusive values: gpu.Shared, exclusive temporary: gpu.Shared ): nil ``` Computes one deterministic inclusive u32 prefix sum. `temporary` is a same-sized scratch array used for disjoint ping-pong stages; the result is left in `values`. ###### Arguments | Name | Type | Description | | --- | --- | --- | | `borrows self` | `gpu.Phases` | | | `exclusive values` | `gpu.Shared\` | | | `exclusive temporary` | `gpu.Shared\` | | ###### Returns | Type | Description | | --- | --- | | `nil` | | ### `Shared` _interface_ ```nupp interface Shared readonly count: integer metamethod __len: function(borrows self: Shared): integer metamethod __index: function(borrows self: Shared, index: integer): T metamethod __newindex: function(exclusive self: Shared, index: integer, value: T): nil end ``` #### Type parameters | Name | Description | | --- | --- | | `T` | | #### Methods ##### `__len` ```nupp __len: function(borrows self: gpu.Shared): integer ``` ###### Arguments | Name | Type | Description | | --- | --- | --- | | `borrows self` | `gpu.Shared\` | | ###### Returns | Type | Description | | --- | --- | | `integer` | | ##### `__index` ```nupp __index: function(borrows self: gpu.Shared, index: integer): T ``` ###### Arguments | Name | Type | Description | | --- | --- | --- | | `borrows self` | `gpu.Shared\` | | | `index` | `integer` | | ###### Returns | Type | Description | | --- | --- | | `T` | | ##### `__newindex` ```nupp __newindex: function(exclusive self: gpu.Shared, index: integer, value: T): nil ``` ###### Arguments | Name | Type | Description | | --- | --- | --- | | `exclusive self` | `gpu.Shared\` | | | `index` | `integer` | | | `value` | `T` | | ###### Returns | Type | Description | | --- | --- | | `nil` | | #### Fields ##### `count` ```nupp count: integer ``` ## Functions ### `bufferIsDense` _function_ ```nupp function bufferIsDense(borrows buffer: api.Buffer): boolean ``` #### Type parameters | Name | Description | | --- | --- | | `T` | | #### Arguments | Name | Type | Description | | --- | --- | --- | | `borrows buffer` | `api.Buffer\` | | #### Returns | Type | Description | | --- | --- | | `boolean` | | ### `bufferIsInjective` _function_ ```nupp function bufferIsInjective(borrows buffer: api.Buffer): boolean ``` #### Type parameters | Name | Description | | --- | --- | | `T` | | #### Arguments | Name | Type | Description | | --- | --- | --- | | `borrows buffer` | `api.Buffer\` | | #### Returns | Type | Description | | --- | --- | | `boolean` | | ### `bufferLayout` _function_ ```nupp function bufferLayout(borrows buffer: api.Buffer): tensorlayout.Layout ``` #### Type parameters | Name | Description | | --- | --- | | `T` | | #### Arguments | Name | Type | Description | | --- | --- | --- | | `borrows buffer` | `api.Buffer\` | | #### Returns | Type | Description | | --- | --- | | `tensorlayout.Layout` | | ### `open` _function_ ```nupp const open: function(): affine(api.Context, api.destroyContext) ``` Opens a compute device through the selected provider. #### Returns | Type | Description | | --- | --- | | `affine(api.Context, api.destroyContext)` | | ### `view` _function_ ```nupp function view(borrows buffer: api.Buffer, layout: tensorlayout.Layout): api.Buffer ``` #### Type parameters | Name | Description | | --- | --- | | `T` | | #### Arguments | Name | Type | Description | | --- | --- | --- | | `borrows buffer` | `api.Buffer\` | | | `layout` | `tensorlayout.Layout` | | #### Returns | Type | Description | | --- | --- | | `api.Buffer\` | | ### `workgroups` _function_ ```nupp function workgroups(groups: uint32, size: uint32, scoped controller: function(uint32, gpu.Phases): nil): nil ``` Executes uniformly ordered workgroup phases on the CPU. A GPU AOT body recognizes this call and its immediate callbacks as structure: the controller does not escape, each `run` is a device barrier, and each stage invocation becomes one local lane. #### Arguments | Name | Type | Description | | --- | --- | --- | | `groups` | `uint32` | | | `size` | `uint32` | | | `scoped controller` | `function(uint32, gpu.Phases): nil` | | #### Returns | Type | Description | | --- | --- | | `nil` | | #### Raises - when size is outside the portable workgroup limit ## Values ### `destroyContext` _variable_ ```nupp const destroyContext ``` ### `PORTABLE_SCRATCH_BYTES` _variable_ ```nupp const PORTABLE_SCRATCH_BYTES ``` ### `PORTABLE_WORKGROUP_THREADS` _variable_ ```nupp const PORTABLE_WORKGROUP_THREADS ```