ember.cpp 0.1.0
C++ inference engine for Llama-family and paperkiln models (CPU, CUDA, WebAssembly)
Loading...
Searching...
No Matches
tinyllama::DistributedCoordinator::LoadStats Struct Reference

Get load balancing statistics. More...

#include <distributed_coordinator.h>

Collaboration diagram for tinyllama::DistributedCoordinator::LoadStats:
Collaboration graph

Public Attributes

int total_requests
 
int completed_requests
 
int failed_requests
 
float average_response_time_ms
 
std::map< int, int > requests_per_node
 

Detailed Description

Get load balancing statistics.

Definition at line 131 of file distributed_coordinator.h.

Member Data Documentation

◆ average_response_time_ms

float tinyllama::DistributedCoordinator::LoadStats::average_response_time_ms

Definition at line 135 of file distributed_coordinator.h.

◆ completed_requests

int tinyllama::DistributedCoordinator::LoadStats::completed_requests

Definition at line 133 of file distributed_coordinator.h.

◆ failed_requests

int tinyllama::DistributedCoordinator::LoadStats::failed_requests

Definition at line 134 of file distributed_coordinator.h.

◆ requests_per_node

std::map<int, int> tinyllama::DistributedCoordinator::LoadStats::requests_per_node

Definition at line 136 of file distributed_coordinator.h.

◆ total_requests

int tinyllama::DistributedCoordinator::LoadStats::total_requests

The documentation for this struct was generated from the following file: