Reference

std/gradient/dual

std/gradient/src/dual.trb

Dual, a value together with the rate at which it changes, and derivative, the one-call form of forward-mode differentiation.

type Dual

type Dual<Scalar: Real = Float> with Signed, Real, Power, Show, From<Scalar>, TryFrom<String, NumberParseError>

A number and its rate of change: the primal part is the value a plain Scalar would have, and the rate part is how fast that value changes with the variable being differentiated. The rate is the derivative, and the field is not called tangent because Real has a member of that name, the trigonometric function.

Every operation applies the chain rule in the scalar's own arithmetic, so the rate is exact up to the scalar's rounding and there is no step size to choose. Dual is a Real, which is what makes it useful: a function written as fn f<Scalar: Real>(x: Scalar): Scalar is differentiated by calling it with a Dual.variable and reading rate, and the function never learns about it. Dual over Dual gives the second derivative.

Examples

fn curve<Scalar: Real>(x: Scalar): Scalar {
  x * x * x + x.sine()
}

const at = Dual.variable 2.0
print curve(at).rate
print derivative(of: curve, at: 2.0)

Pitfalls

  • ==, < and compare look at the primal part alone, so a branch in a generic body (if x < Scalar.zero) takes the same path as the plain function. Two duals with different rates are therefore ==; read rate to tell them apart.
  • At a jump or a corner the rate is that of the branch the primal part took. absolute at zero has rate zero, and floor, ceiling and round always have rate zero: the derivative almost everywhere, not at the steps.
  • The rate of squareRoot at zero, of arcSine at -1 and 1, and of naturalLogarithm at zero is whatever the scalar's division by zero gives: an infinity for Float, a panic for Fixed.
  • A constant is Dual.constant or a conversion from the scalar, a variable is Dual.variable. A function of several variables is differentiated once per variable, each with its own variable and the others constant.

Related

  • derivative - the derivative of a one-variable function in one call.
  • Real - the bound that lets a Dual stand in for a Float or a Fixed.

field primal

primal: Scalar

The value the function has: what a plain Scalar computes.

field rate

rate: Scalar

The rate of the value with respect to the variable: how fast the primal part changes.

const zero

static zero: Self = Self Scalar.zero, Scalar.zero

The constant zero: its rate is zero too.

const one

static one: Self = Self Scalar.one, Scalar.zero

The constant one.

const pi

static pi: Self = Self Scalar.pi, Scalar.zero

The constant pi of the scalar.

const tau

static tau: Self = Self Scalar.tau, Scalar.zero

The constant tau, a whole turn in radians.

const e

static e: Self = Self Scalar.e, Scalar.zero

The constant e.

const epsilon

static epsilon: Self = Self Scalar.epsilon, Scalar.zero

The constant epsilon of the scalar.

fn constant

static fn constant(value: Scalar): Self

A value that does not change with the variable: the rate is zero.

print Dual.constant(3.0).rate

fn variable

static fn variable(value: Scalar): Self

The variable being differentiated, at this value: the rate is one, because the variable changes at the rate of itself.

print Dual.variable(3.0).rate

fn add

fn add(other: Self): Self

The sum rule: primal parts and rates added.

fn subtract

fn subtract(other: Self): Self

The difference rule: primal parts and rates subtracted.

fn multiply

fn multiply(other: Self): Self

The product rule: a' * b + a * b'.

fn divide

fn divide(other: Self): Self

The quotient rule: (a' - (a / b) * b') / b. Divides by the primal part of other, so a primal part of zero is the scalar's division by zero.

fn remainder

fn remainder(other: Self): Self

The remainder keeps the primal part's sign, and its rate is a' - b' * trunc(a / b): the quotient's whole part is constant between the jumps, so only the dividend and the divisor's rate move the result.

fn negate

fn negate(): Self

The sign flipped, rate and all.

fn absolute

fn absolute(): Self

The distance from zero. The rate is sign(primal) * rate, and zero where the primal part is zero.

fn equals

fn equals(other: Self): Bool

Whether the primal parts are equal. The rate does not take part, so == agrees with < and compare.

fn compare

fn compare(other: Self): Ordering

Which primal part comes first. The rate does not take part.

fn show

fn show(): String

The primal part as text: the text of the number the function computes.

fn from

static fn from(value: Scalar): Self

A scalar as a constant: the same as Dual.constant, so Dual<Float>.from(3.0) and 3.0.into() have rate zero.

const constant: Dual<Float> = 3.0.into()

fn tryFrom

static fn tryFrom(text: String): Result<Self, NumberParseError>

Reads a number the way the scalar does and makes a constant of it.

Errors

fn squareRoot

fn squareRoot(): Self

The square root: the rate is a' / (2 * sqrt(a)).

fn sine

fn sine(): Self

The sine: the rate is cos(a) * a'.

fn cosine

fn cosine(): Self

The cosine: the rate is -sin(a) * a'.

fn tangent

fn tangent(): Self

The tangent function: the rate is (1 + tan(a) ** 2) * a'.

fn arcSine

fn arcSine(): Self

The arc sine: the rate is a' / sqrt(1 - a ** 2).

Panics

Outside [-1, 1], as the scalar's arcSine does.

fn arcCosine

fn arcCosine(): Self

The arc cosine: the rate is -a' / sqrt(1 - a ** 2).

Panics

Outside [-1, 1], as the scalar's arcCosine does.

fn arcTangent

fn arcTangent(): Self

The arc tangent: the rate is a' / (1 + a ** 2).

fn arcTangentDivided

fn arcTangentDivided(by: Self): Self

The angle of the point (by, self): the rate is (x * y' - y * x') / (x ** 2 + y ** 2).

fn exponential

fn exponential(): Self

e to the power of the value: the rate is exp(a) * a'.

fn naturalLogarithm

fn naturalLogarithm(): Self

The natural logarithm: the rate is a' / a.

fn logarithm

fn logarithm(base: Self): Self

The logarithm to another base: the rate is a' / (a * ln b) - ln a * b' / (b * ln b ** 2), and the second part is left out while the base does not change.

fn floor

fn floor(): Self

Rounded towards negative infinity. The rate is zero: the rate almost everywhere.

fn ceiling

fn ceiling(): Self

Rounded towards positive infinity. The rate is zero: the rate almost everywhere.

fn round

fn round(): Self

Rounded to the nearest whole value. The rate is zero: the rate almost everywhere.

fn halved

fn halved(): Self

Half the value, rate and all.

fn doubled

fn doubled(): Self

Twice the value, rate and all.

fn radiansOfDegrees

fn radiansOfDegrees(): Self

The angle in radians that this many degrees is: the rate scales like the primal part.

fn degreesOfRadians

fn degreesOfRadians(): Self

The angle in degrees that this many radians is: the rate scales like the primal part.

fn power

fn power(exponent: Self): Self

f ** g, where both may change. The rate is f ** g * (g' * ln f + g * f' / f), but while the exponent does not change it is g * f ** (g - 1) * f', which has no logarithm of the base in it: a negative base with a constant exponent stays differentiable, and an exponent of zero has the rate zero.

Panics

Where the scalar's own power does, such as a negative base with an exponent that is not whole.

extend Dual<Scalar> with Power<Int64>

extend<Scalar: Real> Dual<Scalar> with Power<Int64>

A whole power: the rate is n * f ** (n - 1) * f', and zero for n == 0. The exponent is a constant, so the base may be negative.

fn power

fn power(exponent: Int64): Self

fn derivative

fn derivative<Scalar: Real>(of: Transform<Dual<Scalar>, Dual<Scalar>>, at: Scalar): Scalar

The derivative of a function of one variable, at a point: the function is called once with a Dual.variable and the rate of what it answers is the derivative.

const slope = derivative(of: { x => x * x }, at: 3.0)
print slope

A function written over a Real scalar is passed as a closure that fixes the scalar, such as { x => curve(x) }.