block_q6_K Struct Reference

6-bit K-quantized block structure More...

#include <quantization.h>

Collaboration diagram for block_q6_K:

Public Attributes
uint8_t	ql [GGML_QK_K/2]
uint8_t	qh [GGML_QK_K/4]
int8_t	scales [GGML_QK_K/16]
uint16_t	d

Detailed Description

6-bit K-quantized block structure

Stores weights quantized to 6 bits with block-wise scaling. Provides better precision than Q4_K at the cost of more storage.

Definition at line 88 of file quantization.h.

Member Data Documentation

d

uint16_t block_q6_K::d

Block scale

Definition at line 92 of file quantization.h.

qh

uint8_t block_q6_K::qh[GGML_QK_K/4]

Upper 2 bits of quantized values

Definition at line 90 of file quantization.h.

ql

uint8_t block_q6_K::ql[GGML_QK_K/2]

Lower 4 bits of quantized values

Definition at line 89 of file quantization.h.

Referenced by quantize_q6_k().

scales

int8_t block_q6_K::scales[GGML_QK_K/16]

Sub-block scales

Definition at line 91 of file quantization.h.

---

The documentation for this struct was generated from the following file:

• quantization.h