std/digest

Die generierte Referenz von std/digest zeigt jede öffentliche Deklaration mit Signatur, Doc-Kommentar und Beispielen.

std/digest computes message digests: SHA-256, which is what the package manager pins a package by (the tree hash of project.lock.trb) and what a download is checked against, and SHA-512, which is what Ed25519 hashes with (std/signature). SHA-1 and MD5 are here too, for the protocols that still ask for them - mysql_native_password, Postgres's md5 authentication, SMTP's CRAM-MD5 - and never for anything new. Hmac (RFC 2104) works over any of the four, and Hmac.pbkdf2 (RFC 8018) stretches a password through it: what SCRAM authenticates a password with. It is ordinary TorbScript with no native, so the same bytes give the same digest in the VM and in a native binary.

Import#

use Sha256, Sha512, Sha1, Md5, Digest from "std/digest"
use Hmac, HmacSha256, HmacSha1, HmacMd5 from "std/digest"
use constantTimeEqual from "std/digest"

use Sha256 from "std/digest"

print Sha256.of("abc".bytes()).hex()

Declarations#

Sha256#

type Sha256 with Hasher {
  static fn empty(): Self
  static fn of(bytes: Bytes): Digest
  static fn blockSize(): Int
  static fn digestLength(): Int
  var fn add(bytes: Bytes)
  fn finished(): Digest
}

Sha256.of(bytes) is the digest of everything at once. For input that arrives in pieces, Sha256() starts a hasher, add feeds the next bytes, and finished answers the digest of everything added so far without changing the hasher, so a hasher can be read, fed more, and read again. Pieces of any size give the same digest as the whole.

use Sha256 from "std/digest"

var hasher = Sha256()
hasher.add "ab".bytes()
hasher.add "c".bytes()
print(hasher.finished() == Sha256.of("abc".bytes()))

Sha512#

type Sha512 with Hasher {
  static fn empty(): Self
  static fn of(bytes: Bytes): Digest
  static fn blockSize(): Int
  static fn digestLength(): Int
  var fn add(bytes: Bytes)
  fn finished(): Digest
}

The same shape as Sha256, over 128-byte blocks, answering a digest of 64 bytes. Its words are UInt64s, summed with addedWrapping, the one arithmetic of the language that wraps.

use Sha512 from "std/digest"

print Sha512.of("abc".bytes()).hex()

Sha1#

type Sha1 with Hasher {
  static fn empty(): Self
  static fn of(bytes: Bytes): Digest
  static fn blockSize(): Int
  static fn digestLength(): Int
  var fn add(bytes: Bytes)
  fn finished(): Digest
}

SHA-1 (FIPS 180-4 / RFC 3174): the same shape as Sha256, over 64-byte blocks, answering a 20-byte digest. mysql_native_password, MongoDB's SCRAM-SHA-1 and the WebSocket handshake's accept key still ask for it, but nothing new should choose it over Sha256 - collisions are practical (SHAttered, 2017).

use Sha1 from "std/digest"

print Sha1.of("abc".bytes()).hex()

Md5#

type Md5 with Hasher {
  static fn empty(): Self
  static fn of(bytes: Bytes): Digest
  static fn blockSize(): Int
  static fn digestLength(): Int
  var fn add(bytes: Bytes)
  fn finished(): Digest
}

MD5 (RFC 1321), the same shape again, over 64-byte blocks little endian throughout, answering a 16-byte digest. Legacy: never for security. Collisions are cheap to produce and a chosen-prefix collision is practical - it survives only where a protocol demands its exact bytes: Postgres's md5 authentication, S3's Content-MD5 header.

use Md5 from "std/digest"

print Md5.of("abc".bytes()).hex()

Hasher#

trait Hasher {
  static fn empty(): Self
  static fn of(bytes: Bytes): Digest
  var fn add(bytes: Bytes)
  fn finished(): Digest
  static fn blockSize(): Int
  static fn digestLength(): Int
}

The shape Sha256, Sha512, Sha1 and Md5 all four share, named after the cryptographic sense of "hash" - a hash function's own equality-hashing is the unrelated Hash of std/core. Hmac is written once, over this trait, for any of them.

Hmac#

type Hmac<H: Hasher> {
  static fn keyed(key: Bytes): Self
  var fn add(bytes: Bytes)
  fn finished(): Digest
  static fn of(key: Bytes, message: Bytes): Digest
}
extend<H: Hasher> Hmac<H> {
  static fn pbkdf2(password: Bytes, salt: Bytes, iterations: Int, length: Int): Bytes
}
type HmacSha256 = Hmac<Sha256>
type HmacSha1 = Hmac<Sha1>
type HmacMd5 = Hmac<Md5>

HMAC (RFC 2104) over any Hasher: SCRAM (postgres, mongodb), S3's Signature Version 4, JWT's HS256, SMTP's CRAM-MD5. HmacSha256, HmacSha1 and HmacMd5 name the three combinations the packages after std need; Hmac is fed the same way as Sha256 and the rest.

use HmacSha256 from "std/digest"

print HmacSha256.of("key".bytes(), "message".bytes()).hex()

The key is folded into the hasher once, not once a round. keyed pads the key to a block, hashes the two pads into a fresh inner and outer hasher, and keeps only those two - never the key itself. A round of Hmac.pbkdf2 copies the keyed Hmac - copying a value costs its size, not its history - so many rounds hash the key's two blocks once between them, not once a round.

Hmac.pbkdf2#

PBKDF2 (RFC 8018): password stretched into length bytes, iterations rounds of HMAC salted forward from one round to the next. HmacSha256.pbkdf2 is what SCRAM-SHA-256 authenticates a password with.

use HmacSha256 from "std/digest"

const derived = HmacSha256.pbkdf2 "password".bytes(), "salt".bytes(), 4096, 32
print derived.length()

Digest#

type Digest with Show {
  bytes: Bytes
  fn hex(): String
  fn show(): String
  fn equalsConstantTime(other: Digest): Bool
}
fn constantTimeEqual(a: Bytes, b: Bytes): Bool

The bytes of a digest, most significant first: 32 of SHA-256, 64 of SHA-512. hex() and show() are the same text: two lowercase hexadecimal digits per byte, the spelling of sha256sum and of a lock file. Two digests are equal when their bytes are.

Digest.equalsConstantTime#

== on two digests is the derived list equality, which may stop at the first byte that differs and says nothing about timing. To check a MAC, a token or any digest that came from outside, call equalsConstantTime: it reads every byte pair, combines the differences by XOR and OR, and answers only at the end, so the running time does not depend on where the two first differ. Digests of different lengths are not equal; the length is not a secret and is compared first.

use HmacSha256 from "std/digest"

const expected = HmacSha256.of "key".bytes(), "message".bytes()
const received = HmacSha256.of "key".bytes(), "message".bytes()
print expected.equalsConstantTime(received)

Promised: the source has no data-dependent early exit and no branch on a byte's value. Not promised: nothing guards against a compiler or a CPU introducing a timing effect of its own, so this is a defence against the obvious leak, not a certified constant-time primitive.

constantTimeEqual(a, b) is a thin wrapper that does the same for two Bytes where no Digest is at hand; prefer the method where one is.

use constantTimeEqual from "std/digest"

print constantTimeEqual("abc".bytes(), "abd".bytes())

Pitfalls#

A digest is not a signature. SHA-256 says that two byte sequences are the same; it says nothing about who wrote them. A hash that travels next to the data it describes proves nothing an attacker who can change the one cannot also change in the other.

It is written for correctness, not speed. A few megabytes take a moment in a native binary and noticeably longer in the VM. Nothing in the standard library hashes on a hot path - except Hmac.pbkdf2's rounds, which is why it keys the hasher once and copies it, instead of re-hashing the key on every round.

MD5 and SHA-1 are for interoperability, not security. Neither resists a deliberate collision, so nothing signs with them and nothing new should ask for them; they exist because a handful of protocols still do.

A UInt64 literal above the largest Int64 is not a compile-time constant of a module yet, so SHA-512's constants are written as their upper and lower 32 bits and joined where they are used.

Related#

• std/signature - Ed25519, which hashes with Sha512: what a signature adds to a digest.
• std/archive and std/compression - the other two halves of a package archive.
• project.lock.trb - the tree hash, which is a SHA-256 over SHA-256s.
• std/encoding - Base64 and hexadecimal, for a digest or a MAC that travels as text.
• The standard library - the other packages.