ember.cpp 0.1.0
C++ inference engine for Llama-family and paperkiln models (CPU, CUDA, WebAssembly)
Loading...
Searching...
No Matches
ir_digest.h
Go to the documentation of this file.
1#pragma once
2// The digest of an IR document (paperkiln spec/IR_SPEC.md section 3.10, revision 1.1), written for ember
3// from the specification text (no code taken from paperkiln):
4// 1. the text is parsed as I-JSON (RFC 7493): no duplicate member names, every number a finite binary64,
5// every integer token within -2^53 .. 2^53, every string well-formed Unicode (no lone surrogates, valid
6// UTF-8);
7// 2. every member whose name starts with "_" is removed, at any depth (kept when prose is true: a test of
8// the serialiser alone);
9// 3. the rest is serialised by RFC 8785 (JCS): no whitespace, member names sorted by their UTF-16 code
10// units, strings as ECMAScript's JSON.stringify writes them (only '"', '\' and U+0000 .. U+001F escaped),
11// numbers as ECMAScript's Number.prototype.toString writes the binary64 value (shortest round-trip
12// digits, from std::to_chars);
13// 4. the digest is SHA-256 (FIPS 180-4) of that text's UTF-8 bytes, 64 lowercase hex digits.
14// A document that holds a negative zero has no digest (RFC 8785 writes -0 as 0). An integer token "-0" is the
15// integer zero, not a negative zero (the suite's vector minus_zero_integer_form). The I-JSON conditions and
16// the negative zero are checked on the whole parsed text, "_" members included (step 1 precedes step 2).
17#include <string>
18
19namespace ember_ir {
20
22 bool ok = false; // false: the text has no digest (refused says why)
23 std::string sha256; // 64 lowercase hex digits
24 std::string canonical; // the RFC 8785 text (UTF-8)
25 std::string refused;
26};
27
28DigestResult ir_digest(const std::string& text, bool prose = false);
29
30// SHA-256 of bytes, 64 lowercase hex digits.
31std::string sha256_hex_bytes(const std::string& bytes);
32
33// ECMAScript Number::toString of a finite binary64 (exposed for the self-test).
34std::string es_number_string(double v);
35
36} // namespace ember_ir
std::string text
std::vector< float > v
size_t j
Definition api.h:20
DigestResult ir_digest(const std::string &text, bool prose)
std::string sha256_hex_bytes(const std::string &bytes)
std::string es_number_string(double v)
std::string canonical
Definition ir_digest.h:24
std::string refused
Definition ir_digest.h:25