KVCacheLayer Struct Reference

Key-Value cache for a single transformer layer. More...

#include <model.h>

Collaboration diagram for KVCacheLayer:

Public Attributes
std::vector< float >	k
std::vector< float >	v

Detailed Description

Key-Value cache for a single transformer layer.

Stores the key and value tensors for attention mechanism, with optional CUDA support for GPU acceleration.

Definition at line 136 of file model.h.

Member Data Documentation

k

std::vector<float> KVCacheLayer::k

Definition at line 137 of file model.h.

Referenced by ember_ir::Interpreter::run_nodes().

v

std::vector<float> KVCacheLayer::v

Definition at line 138 of file model.h.

---

The documentation for this struct was generated from the following file:

• model.h