ember.cpp 0.1.0
C++ inference engine for Llama-family and paperkiln models (CPU, CUDA, WebAssembly)
Loading...
Searching...
No Matches
ember::SpmTokenizer::Symbol Struct Reference
Collaboration diagram for ember::SpmTokenizer::Symbol:
Collaboration graph

Public Attributes

int prev
 
int next
 
size_t off
 
size_t n
 

Detailed Description

Definition at line 139 of file spm_tokenizer.h.

Member Data Documentation

◆ n

size_t ember::SpmTokenizer::Symbol::n

Definition at line 141 of file spm_tokenizer.h.

Referenced by ember::SpmTokenizer::encode_fragment().

◆ next

int ember::SpmTokenizer::Symbol::next

Definition at line 140 of file spm_tokenizer.h.

◆ off

size_t ember::SpmTokenizer::Symbol::off

Definition at line 141 of file spm_tokenizer.h.

◆ prev

int ember::SpmTokenizer::Symbol::prev

Definition at line 140 of file spm_tokenizer.h.


The documentation for this struct was generated from the following file: