Tensors, Gradients and the GPU#

Status: partly built — std/gradient exists with forward mode (Dual, section 4); std/tensor, std/gpu and std/learning do not. std/linear and std/geometry do (LINEAR.md). This record is a stub that keeps the plan decided on 2026-09-21; its full design, with research into JAX, PyTorch, Burn, tinygrad, Mojo and Swift for TensorFlow, comes before the first of the remaining packages is built. std/tensor on the CPU has its design record: TENSOR.md (designed, not built).

Training and inference are possible with the standard library, and they stand on the same foundation as geometry and the game engine of ECS.md rather than on a tower of their own. Four layers, each of which is useful without the ones above it.

1. The layers#

Layer 0 - the language and the runtime, shared with the engine.

• Buffer<Item>, the heap storage primitive (COLLECTIONS.md, gap 6): contiguous memory, an in-place write where there is one owner, disjoint var windows.
• The trait Real of std/linear, and the small floating point types Float16 and BFloat16 beside Float32.
• Foreign functions, which the graphics interfaces need as well (CONCEPT, "Foreign Functions"; designed in FOREIGN.md, not built).
• Data-parallel loops over disjoint windows of a buffer, the same gap as parallel systems in an ECS (CONCURRENCY.md).
• SIMD in the C back end.

Layer 1 - pure values. std/linear and std/geometry (small fixed vectors and matrices, which exist), std/tensor (TENSOR.md; n-dimensional, broadcasting, slices as views, on Buffer<Item>; the shape is checked at run time, while Matrix<Scalar, Rows, Columns> stays in std/linear with its size in the type), std/random (seedable and splittable, for reproducible training) and std/statistics. Value semantics fit: a tensor is a value, and a change through a var path with a single owner happens without a copy, which is what JAX needs "donation" for.

Layer 2 - differentiation and computation.

• std/gradient: forward mode as Dual<Scalar> with Real, so that every function generic over Real - geometry, animation, physics included - can be differentiated without being written for it. Reverse mode through quoted expressions: gradient { x => ... } receives the expression tree, as the query provider of examples/query-provider does. Quoted expressions are the language's one mechanism of this kind, and they replace macros and tracing.
• std/gpu: an abstract layer for computation and drawing in the model of WebGPU, mapped onto Direct3D, Metal, Vulkan and WebGPU. The same path - a quoted expression becomes a kernel or a shader - serves std/render and std/tensor.

Layer 3 - applications. std/learning: layers, optimizers, losses and the training loop, with the data arriving as a Source<Batch, Failure>, so the stream protocol of STREAMS.md is the data pipeline. Model formats (safetensors, ONNX, GGUF) through the encoding layer (ENCODING.md); quantized inference over integer and Fixed scalars. Beside it, on the same layers: std/ecs, std/render, std/collision, std/animation.

2. The order#

std/linear and std/geometry (they bring Real; done), then Buffer<Item>, then std/tensor on the CPU and Dual (built), which are pure TorbScript without foreign functions; then foreign functions (FOREIGN.md, designed); then std/gpu; then reverse mode, std/learning, std/render and std/ecs.

3. What these packages will test in the language#

• Arithmetic on size parameters (Rows * 2): probably not, so the shape moves to run time where it has to.
• Operator traits with a foreign Other and Output (a tensor times a scalar).
• Buffers beyond 2^31 elements, and memory-mapped files.
• Quoted expressions that contain control flow.
• A guaranteed in-place change where there is one owner.

4. Forward mode: Dual#

std/gradient holds Dual<Scalar: Real = Float>, a value and the rate at which it changes, in pure TorbScript with no foreign function and no dependency on std/tensor. It is itself Numeric, Signed and Real, so a function written as fn f<Scalar: Real>(x: Scalar): Scalar - in geometry, animation or physics, or a Tensor<Dual<Scalar>> - is differentiated by calling it with a Dual.variable and reading rate, without being written for it. Reverse mode joins the package later.

Layout. std/gradient/src/dual.trb (the type and derivative(of:at:)), src/lib.trb (the exports), tests/dual.test.trb. The fields are primal (the word TENSOR.md section 9 already uses) and rate. They are not primal and tangent, because Real has a member tangent() and a type has one namespace of members. Dual.constant(value) has rate zero, Dual.variable(value) has rate one, and Dual(primal, rate) builds both parts. derivative(of: { x => ... }, at: value) is the one-call form.

The rules are the chain rule in the scalar's own arithmetic; a generic body writes Scalar.zero and Scalar.one.

== looks at the primal part. ==, < and compare ignore the rate, so a branch in a generic body (if x < Scalar.zero) takes the same path as the plain function, and compare == .Equal and == false cannot hold together. TENSOR.md section 9 needs compare on the primal part for minimum and maximum; == follows. The rate is read with the field. Show writes the primal part, From<Scalar> makes a constant of a scalar (what Dual.constant does) and TryFrom<String> reads one.

Pitfalls.

• A jump or a corner has the rate of the branch the primal part took: absolute at zero is zero, and floor, ceiling and round are zero everywhere, the derivative almost everywhere and not at the steps.
• Where the scalar's own function is undefined or infinite (squareRoot at zero, arcSine at 1, naturalLogarithm at zero) the rate is what the scalar's division gives: an infinity for Float, a panic for Fixed. arcSine and arcCosine panic outside [-1, 1] as the scalar does.
• A negative base with a whole constant exponent is differentiable, because the rule with g' == 0 has no logarithm in it.
• Real has no conversion from Int, so f ** n builds the scalar of n by doubling and adding one; n beyond the scalar's range overflows as the scalar does.
• A function of several variables is differentiated once per variable; nesting, Dual<Dual<Float>>, gives a second derivative.