|
ember.cpp 0.1.0
C++ inference engine for Llama-family and paperkiln models (CPU, CUDA, WebAssembly)
|
Get load balancing statistics. More...
#include <distributed_coordinator.h>

Public Attributes | |
| int | total_requests |
| int | completed_requests |
| int | failed_requests |
| float | average_response_time_ms |
| std::map< int, int > | requests_per_node |
Get load balancing statistics.
Definition at line 131 of file distributed_coordinator.h.
| float tinyllama::DistributedCoordinator::LoadStats::average_response_time_ms |
Definition at line 135 of file distributed_coordinator.h.
| int tinyllama::DistributedCoordinator::LoadStats::completed_requests |
Definition at line 133 of file distributed_coordinator.h.
| int tinyllama::DistributedCoordinator::LoadStats::failed_requests |
Definition at line 134 of file distributed_coordinator.h.
Definition at line 136 of file distributed_coordinator.h.
| int tinyllama::DistributedCoordinator::LoadStats::total_requests |
Definition at line 132 of file distributed_coordinator.h.
Referenced by tinyllama::DistributedCoordinator::get_load_stats().