tinyllama::DistributedCoordinator::InferenceRequest Struct Reference

#include <distributed_coordinator.h>

Collaboration diagram for tinyllama::DistributedCoordinator::InferenceRequest:

Public Attributes
std::string	request_id
std::string	prompt
int	max_tokens
float	temperature
int	top_k
float	top_p
std::string	system_prompt
int	target_node = -1

Detailed Description

Definition at line 41 of file distributed_coordinator.h.

Member Data Documentation

max_tokens

int tinyllama::DistributedCoordinator::InferenceRequest::max_tokens

Definition at line 44 of file distributed_coordinator.h.

Referenced by tinyllama::DistributedCoordinator::deserialize_request().

prompt

std::string tinyllama::DistributedCoordinator::InferenceRequest::prompt

Definition at line 43 of file distributed_coordinator.h.

request_id

std::string tinyllama::DistributedCoordinator::InferenceRequest::request_id

Definition at line 42 of file distributed_coordinator.h.

Referenced by run_interactive_master().

system_prompt

std::string tinyllama::DistributedCoordinator::InferenceRequest::system_prompt

Definition at line 48 of file distributed_coordinator.h.

target_node

int tinyllama::DistributedCoordinator::InferenceRequest::target_node = -1

Definition at line 49 of file distributed_coordinator.h.

temperature

float tinyllama::DistributedCoordinator::InferenceRequest::temperature

Definition at line 45 of file distributed_coordinator.h.

top_k

int tinyllama::DistributedCoordinator::InferenceRequest::top_k

Definition at line 46 of file distributed_coordinator.h.

top_p

float tinyllama::DistributedCoordinator::InferenceRequest::top_p

Definition at line 47 of file distributed_coordinator.h.

---

The documentation for this struct was generated from the following file:

• distributed_coordinator.h