ember::ChatVocab Struct Reference

#include <chat_tokenizer.h>

Collaboration diagram for ember::ChatVocab:

Public Attributes
microtorch::bpe::GgufTokenizer	g
std::vector< float >	scores
bool	add_space_prefix = true
std::string	arch
std::string	chat_template
std::string	declared

Detailed Description

Definition at line 247 of file chat_tokenizer.h.

Member Data Documentation

add_space_prefix

bool ember::ChatVocab::add_space_prefix = true

Definition at line 250 of file chat_tokenizer.h.

arch

std::string ember::ChatVocab::arch

Definition at line 251 of file chat_tokenizer.h.

chat_template

std::string ember::ChatVocab::chat_template

Definition at line 252 of file chat_tokenizer.h.

declared

std::string ember::ChatVocab::declared

Definition at line 253 of file chat_tokenizer.h.

g

microtorch::bpe::GgufTokenizer ember::ChatVocab::g

Definition at line 248 of file chat_tokenizer.h.

scores

std::vector<float> ember::ChatVocab::scores

Definition at line 249 of file chat_tokenizer.h.

---

The documentation for this struct was generated from the following file:

• chat_tokenizer.h