ember.cpp 0.1.0
C++ inference engine for Llama-family and paperkiln models (CPU, CUDA, WebAssembly)
Loading...
Searching...
No Matches
ember::ReplyBuilder Class Reference

#include <chat_reply.h>

Collaboration diagram for ember::ReplyBuilder:
Collaboration graph

Public Member Functions

 ReplyBuilder (int call_id, ChatDecode decode)
 
std::string next (int id)
 
std::string finish (ChatReply &out)
 
const std::vector< int > & ids () const
 

Private Member Functions

std::ptrdiff_t first_call () const
 

Private Attributes

int call_id_
 
ChatDecode decode_
 
std::vector< int > ids_
 
bool in_call_ = false
 
ReplyStream stream_
 

Detailed Description

Definition at line 76 of file chat_reply.h.

Constructor & Destructor Documentation

◆ ReplyBuilder()

ember::ReplyBuilder::ReplyBuilder ( int  call_id,
ChatDecode  decode 
)
inline

Definition at line 78 of file chat_reply.h.

78: call_id_(call_id), decode_(std::move(decode)) {}
size_t j

Member Function Documentation

◆ finish()

std::string ember::ReplyBuilder::finish ( ChatReply &  out)
inline

Definition at line 87 of file chat_reply.h.

87 {
89 if (!in_call_) return stream_.next(decode_(ids_), true);
90 if (!out.tool_calls.empty()) { // parsed: the content before the first call
91 std::vector<int> before(ids_.begin(), ids_.begin() + first_call());
92 return stream_.next(decode_(before), true);
93 }
94 return stream_.next(decode_(ids_), true); // not parsed: the whole reply is content
95 }
ReplyStream stream_
Definition chat_reply.h:110
std::ptrdiff_t first_call() const
Definition chat_reply.h:100
std::vector< int > ids_
Definition chat_reply.h:108
std::string next(const std::string &full, bool end)
Definition utf8_text.h:116
node out
ChatReply reply_of(const std::vector< int > &ids, int call_id, const ChatDecode &decode)
Definition chat_reply.h:45

References j, out, and ember::reply_of().

◆ first_call()

std::ptrdiff_t ember::ReplyBuilder::first_call ( ) const
inlineprivate

Definition at line 100 of file chat_reply.h.

100 {
101 for (size_t i = 0; i < ids_.size(); ++i)
102 if (ids_[i] == call_id_) return static_cast<std::ptrdiff_t>(i);
103 return static_cast<std::ptrdiff_t>(ids_.size());
104 }

References j.

◆ ids()

const std::vector< int > & ember::ReplyBuilder::ids ( ) const
inline

Definition at line 97 of file chat_reply.h.

97{ return ids_; }

◆ next()

std::string ember::ReplyBuilder::next ( int  id)
inline

Definition at line 80 of file chat_reply.h.

80 {
81 ids_.push_back(id);
82 if (call_id_ >= 0 && id == call_id_) in_call_ = true;
83 if (in_call_) return std::string();
84 return stream_.next(decode_(ids_), false);
85 }

Member Data Documentation

◆ call_id_

int ember::ReplyBuilder::call_id_
private

Definition at line 106 of file chat_reply.h.

◆ decode_

ChatDecode ember::ReplyBuilder::decode_
private

Definition at line 107 of file chat_reply.h.

◆ ids_

std::vector<int> ember::ReplyBuilder::ids_
private

Definition at line 108 of file chat_reply.h.

◆ in_call_

bool ember::ReplyBuilder::in_call_ = false
private

Definition at line 109 of file chat_reply.h.

◆ stream_

ReplyStream ember::ReplyBuilder::stream_
private

Definition at line 110 of file chat_reply.h.


The documentation for this class was generated from the following file: