ggml_types.h File Reference

Type definitions for GGML (Georgi Gerganov Machine Learning) library. More...

#include <cstdint>

Include dependency graph for ggml_types.h:

This graph shows which files directly or indirectly include this file:

Go to the source code of this file.

Enumerations
enum	GGMLType {   GGML_TYPE_F32 = 0 , GGML_TYPE_F16 = 1 , GGML_TYPE_Q4_0 = 2 , GGML_TYPE_Q4_1 = 3 ,   GGML_TYPE_Q5_0 = 6 , GGML_TYPE_Q5_1 = 7 , GGML_TYPE_Q8_0 = 8 , GGML_TYPE_Q8_1 = 9 ,   GGML_TYPE_Q2_K = 10 , GGML_TYPE_Q3_K = 11 , GGML_TYPE_Q4_K = 12 , GGML_TYPE_Q5_K = 13 ,   GGML_TYPE_Q6_K = 14 , GGML_TYPE_Q8_K = 15 , GGML_TYPE_I8 = 16 , GGML_TYPE_I16 = 17 ,   GGML_TYPE_I32 = 18 , GGML_TYPE_BF16 = 30 , GGML_TYPE_COUNT }
	Enumeration of GGML tensor data types. More...
enum class	GGUFValueType : uint32_t {   UINT8 = 0 , INT8 = 1 , UINT16 = 2 , INT16 = 3 ,   UINT32 = 4 , INT32 = 5 , FLOAT32 = 6 , BOOL = 7 ,   STRING = 8 , ARRAY = 9 , UINT64 = 10 , INT64 = 11 ,   FLOAT64 = 12 }
	Enumeration of value types used in GGUF metadata. More...

Detailed Description

Type definitions for GGML (Georgi Gerganov Machine Learning) library.

This file defines the various data types used in the GGML library for tensor operations and quantization. It includes both standard floating-point types and various quantized formats for efficient model storage and computation.

Definition in file ggml_types.h.

Enumeration Type Documentation

GGMLType

enum GGMLType

Enumeration of GGML tensor data types.

Defines the various data types that can be used for tensors in GGML, including standard floating point types (F32, F16, BF16) and various quantized formats (Q2_K through Q8_K) for model compression.

Enumerator
GGML_TYPE_F32	32-bit floating point
GGML_TYPE_F16	16-bit floating point
GGML_TYPE_Q4_0	4-bit quantization (version 0)
GGML_TYPE_Q4_1	4-bit quantization (version 1)
GGML_TYPE_Q5_0	5-bit quantization (version 0)
GGML_TYPE_Q5_1	5-bit quantization (version 1)
GGML_TYPE_Q8_0	8-bit quantization (version 0)
GGML_TYPE_Q8_1	8-bit quantization (version 1)
GGML_TYPE_Q2_K	2-bit quantization with K-means
GGML_TYPE_Q3_K	3-bit quantization with K-means
GGML_TYPE_Q4_K	4-bit quantization with K-means
GGML_TYPE_Q5_K	5-bit quantization with K-means
GGML_TYPE_Q6_K	6-bit quantization with K-means
GGML_TYPE_Q8_K	8-bit quantization with K-means
GGML_TYPE_I8	8-bit signed integer
GGML_TYPE_I16	16-bit signed integer
GGML_TYPE_I32	32-bit signed integer
GGML_TYPE_BF16	Brain floating point (16-bit)
GGML_TYPE_COUNT	Number of defined types

Definition at line 21 of file ggml_types.h.

              {
  GGML_TYPE_F32 = 0,    
  GGML_TYPE_F16 = 1,    
  GGML_TYPE_Q4_0 = 2,   
  GGML_TYPE_Q4_1 = 3,   
  GGML_TYPE_Q5_0 = 6,   
  GGML_TYPE_Q5_1 = 7,   
  GGML_TYPE_Q8_0 = 8,   
  GGML_TYPE_Q8_1 = 9,   
  GGML_TYPE_Q2_K = 10,  
  GGML_TYPE_Q3_K = 11,  
  GGML_TYPE_Q4_K = 12,  
  GGML_TYPE_Q5_K = 13,  
  GGML_TYPE_Q6_K = 14,  
  GGML_TYPE_Q8_K = 15,  
  GGML_TYPE_I8 = 16,    
  GGML_TYPE_I16 = 17,   
  GGML_TYPE_I32 = 18,   
  GGML_TYPE_BF16 = 30,  
  GGML_TYPE_COUNT,      
};

GGUFValueType

enum class GGUFValueType : uint32_t

strong

Enumeration of value types used in GGUF metadata.

Defines the possible data types that can be stored in GGUF metadata key-value pairs. This includes basic numeric types, strings, arrays, and boolean values.

Enumerator
UINT8	8-bit unsigned integer
INT8	8-bit signed integer
UINT16	16-bit unsigned integer
INT16	16-bit signed integer
UINT32	32-bit unsigned integer
INT32	32-bit signed integer
FLOAT32	32-bit floating point
BOOL	Boolean value
STRING	String value
ARRAY	Array of values
UINT64	64-bit unsigned integer
INT64	64-bit signed integer
FLOAT64	64-bit floating point

Definition at line 51 of file ggml_types.h.

                         : uint32_t {
  UINT8 = 0,     
  INT8 = 1,      
  UINT16 = 2,    
  INT16 = 3,     
  UINT32 = 4,    
  INT32 = 5,     
  FLOAT32 = 6,   
  BOOL = 7,      
  STRING = 8,    
  ARRAY = 9,     
  UINT64 = 10,   
  INT64 = 11,    
  FLOAT64 = 12,  
}