veloxquant_mlx.cache.options module

Per-method option dataclasses for KVCacheConfig.

KVCacheConfig grew one flat dataclass field per quantization/eviction method’s hyperparameter (207 fields across 37 method families as of this module’s introduction) rather than one option object per method (#420). This module groups each method’s already-adjacent fields into its own small dataclass so a caller configuring, say, H2O only sees H2O’s five fields instead of all 207.

Usage:

from veloxquant_mlx.cache.options import H2OOptions

config = KVCacheConfig(method="h2o", options=H2OOptions(budget=1024))

Passing options populates the matching flat fields on KVCacheConfig (here, h2o_budget) for any flat field still at its dataclass default, so every existing cache class – which reads getattr(config, "h2o_budget", 512) directly – keeps working unchanged; see KVCacheConfig.__post_init__. Setting a flat per-method field directly still works too (backwards compatible) but now emits a DeprecationWarning naming the options replacement.

These classes are pure data containers with no behavior of their own – field names, types, defaults, and comments are carried over verbatim from the original flat fields on KVCacheConfig.

class veloxquant_mlx.cache.options.VecInferOptions(key_sub_dim=4, value_sub_dim=8, key_codebook_bits=12, value_codebook_bits=8, residual_length=128)[source]

Bases: object

VecInfer-specific configuration.

Parameters:

  • key_sub_dim (int)

  • value_sub_dim (int)

  • key_codebook_bits (int)

  • value_codebook_bits (int)

  • residual_length (int)

key_sub_dim: int = 4
value_sub_dim: int = 8
key_codebook_bits: int = 12
value_codebook_bits: int = 8
residual_length: int = 128
class veloxquant_mlx.cache.options.KIVIOptions(kivi_group_size=32)[source]

Bases: object

KIVI configuration (asymmetric group quantization).

Parameters:

kivi_group_size (int)

kivi_group_size: int = 32
class veloxquant_mlx.cache.options.SVDqOptions(svdq_rank=None, svdq_energy_threshold=0.95, svdq_bit_schedule=(8, 4, 2, 1, 1, 0, 0, 0), svdq_group_size=32)[source]

Bases: object

SVDq configuration (sub-2-bit key compression via offline SVD).

Parameters:

  • svdq_rank (int | None)

  • svdq_energy_threshold (float)

  • svdq_bit_schedule (tuple[int, ...])

  • svdq_group_size (int)

svdq_rank: int | None = None
svdq_energy_threshold: float = 0.95
svdq_bit_schedule: tuple[int, ...] = (8, 4, 2, 1, 1, 0, 0, 0)
svdq_group_size: int = 32
class veloxquant_mlx.cache.options.KittyOptions(kitty_hi_fraction=0.25, kitty_hi_bit=4, kitty_lo_bit=2, kitty_group_size=32)[source]

Bases: object

Kitty configuration (dynamic channel-wise mixed-precision).

Parameters:

  • kitty_hi_fraction (float)

  • kitty_hi_bit (int)

  • kitty_lo_bit (int)

  • kitty_group_size (int)

kitty_hi_fraction: float = 0.25
kitty_hi_bit: int = 4
kitty_lo_bit: int = 2
kitty_group_size: int = 32
class veloxquant_mlx.cache.options.AdaKVOptions(adakv_target_avg_bits=2.5, adakv_lo_bit=2, adakv_mid_bit=3, adakv_hi_bit=4, adakv_group_size=32, adakv_update_interval=1, adakv_importance='norm_variance', adakv_obs_window=32)[source]

Bases: object

AdaKV-proxy configuration (per-head adaptive bit allocation).

Parameters:

  • adakv_target_avg_bits (float)

  • adakv_lo_bit (int)

  • adakv_mid_bit (int)

  • adakv_hi_bit (int)

  • adakv_group_size (int)

  • adakv_update_interval (int)

  • adakv_importance (str)

  • adakv_obs_window (int)

adakv_target_avg_bits: float = 2.5
adakv_lo_bit: int = 2
adakv_mid_bit: int = 3
adakv_hi_bit: int = 4
adakv_group_size: int = 32
adakv_update_interval: int = 1
adakv_importance: str = 'norm_variance'
adakv_obs_window: int = 32
class veloxquant_mlx.cache.options.XQuantOptions(xquant_group_size=2, xquant_base_bits=2, xquant_residual_bits=4, xquant_group_quant_size=32, xquant_max_ctx=8192)[source]

Bases: object

XQuant configuration (cross-layer KV cache reuse).

Parameters:

  • xquant_group_size (int)

  • xquant_base_bits (int)

  • xquant_residual_bits (int)

  • xquant_group_quant_size (int)

  • xquant_max_ctx (int)

xquant_group_size: int = 2
xquant_base_bits: int = 2
xquant_residual_bits: int = 4
xquant_group_quant_size: int = 32
xquant_max_ctx: int = 8192
class veloxquant_mlx.cache.options.KVQuantOptions(kvquant_bits=3, kvquant_outlier_fraction=0.01, kvquant_group_size=32, kvquant_lloyd_iters=8, kvquant_refit_interval=0, kvquant_n_sink=1)[source]

Bases: object

KVQuant-NUQ configuration (non-uniform datatype + outlier isolation).

Parameters:

  • kvquant_bits (int)

  • kvquant_outlier_fraction (float)

  • kvquant_group_size (int)

  • kvquant_lloyd_iters (int)

  • kvquant_refit_interval (int)

  • kvquant_n_sink (int)

kvquant_bits: int = 3
kvquant_outlier_fraction: float = 0.01
kvquant_group_size: int = 32
kvquant_lloyd_iters: int = 8
kvquant_refit_interval: int = 0
kvquant_n_sink: int = 1
class veloxquant_mlx.cache.options.PALUOptions(palu_rank=None, palu_energy_threshold=0.9, palu_n_head_groups=4, palu_hi_bit=4, palu_lo_bit=2, palu_hi_fraction=0.25, palu_group_size=32, palu_quantize_values=True)[source]

Bases: object

PALU configuration (true-latent low-rank K and V).

Parameters:

  • palu_rank (int | None)

  • palu_energy_threshold (float)

  • palu_n_head_groups (int)

  • palu_hi_bit (int)

  • palu_lo_bit (int)

  • palu_hi_fraction (float)

  • palu_group_size (int)

  • palu_quantize_values (bool)

palu_rank: int | None = None
palu_energy_threshold: float = 0.9
palu_n_head_groups: int = 4
palu_hi_bit: int = 4
palu_lo_bit: int = 2
palu_hi_fraction: float = 0.25
palu_group_size: int = 32
palu_quantize_values: bool = True
class veloxquant_mlx.cache.options.CacheGenOptions(cachegen_bits=4, cachegen_group_size=32, cachegen_use_delta=True, cachegen_per_channel=True, cachegen_layer_groups=3, cachegen_resolved_bits=None)[source]

Bases: object

CacheGen configuration (entropy-coded byte model over group quant).

Parameters:

  • cachegen_bits (int)

  • cachegen_group_size (int)

  • cachegen_use_delta (bool)

  • cachegen_per_channel (bool)

  • cachegen_layer_groups (int)

  • cachegen_resolved_bits (int | None)

cachegen_bits: int = 4
cachegen_group_size: int = 32
cachegen_use_delta: bool = True
cachegen_per_channel: bool = True
cachegen_layer_groups: int = 3
cachegen_resolved_bits: int | None = None
class veloxquant_mlx.cache.options.MiniCacheOptions(minicache_start_frac=0.5, minicache_group_size=2, minicache_retention_threshold=0.9, minicache_slerp_t=0.5, minicache_max_ctx=8192)[source]

Bases: object

MiniCache configuration (cross-layer depth-dimension SLERP merge).

Parameters:

  • minicache_start_frac (float)

  • minicache_group_size (int)

  • minicache_retention_threshold (float)

  • minicache_slerp_t (float)

  • minicache_max_ctx (int)

minicache_start_frac: float = 0.5
minicache_group_size: int = 2
minicache_retention_threshold: float = 0.9
minicache_slerp_t: float = 0.5
minicache_max_ctx: int = 8192
class veloxquant_mlx.cache.options.GEAROptions(gear_bits=2, gear_rank=None, gear_energy_threshold=0.9, gear_sparse_fraction=0.01, gear_group_size=32, gear_quantize_values=True)[source]

Bases: object

GEAR configuration (error-feedback: residual low-rank + sparse outliers).

Parameters:

  • gear_bits (int)

  • gear_rank (int | None)

  • gear_energy_threshold (float)

  • gear_sparse_fraction (float)

  • gear_group_size (int)

  • gear_quantize_values (bool)

gear_bits: int = 2
gear_rank: int | None = None
gear_energy_threshold: float = 0.9
gear_sparse_fraction: float = 0.01
gear_group_size: int = 32
gear_quantize_values: bool = True
class veloxquant_mlx.cache.options.ZipCacheOptions(zipcache_hi_bits=4, zipcache_lo_bits=2, zipcache_hi_fraction=0.2, zipcache_group_size=32, zipcache_quantize_values=True)[source]

Bases: object

ZipCache-adapted configuration (saliency-adaptive per-token mixed-precision).

Parameters:

  • zipcache_hi_bits (int)

  • zipcache_lo_bits (int)

  • zipcache_hi_fraction (float)

  • zipcache_group_size (int)

  • zipcache_quantize_values (bool)

zipcache_hi_bits: int = 4
zipcache_lo_bits: int = 2
zipcache_hi_fraction: float = 0.2
zipcache_group_size: int = 32
zipcache_quantize_values: bool = True
class veloxquant_mlx.cache.options.SnapKVOptions(snap_dtype='auto', snap_batched_scoring=True, snap_backend='auto', snap_budget=512, snap_obs_window=32, snap_n_sink=4)[source]

Bases: object

SnapKV-adapted configuration (prefill observation-window token eviction).

Parameters:

  • snap_dtype (str)

  • snap_batched_scoring (bool)

  • snap_backend (str)

  • snap_budget (int)

  • snap_obs_window (int)

  • snap_n_sink (int)

snap_dtype: str = 'auto'
snap_batched_scoring: bool = True
snap_backend: str = 'auto'
snap_budget: int = 512
snap_obs_window: int = 32
snap_n_sink: int = 4
class veloxquant_mlx.cache.options.StreamingLLMOptions(stream_n_sink=4, stream_window_size=512)[source]

Bases: object

StreamingLLM-adapted configuration (sink + recency-window structural eviction).

Parameters:

  • stream_n_sink (int)

  • stream_window_size (int)

stream_n_sink: int = 4
stream_window_size: int = 512
class veloxquant_mlx.cache.options.H2OOptions(h2o_budget=512, h2o_n_sink=4, h2o_rope_base=10000.0, h2o_grace=16, h2o_decay=0.98)[source]

Bases: object

H2O-adapted configuration (cumulative attention-mass heavy-hitter eviction).

Parameters:
h2o_budget: int = 512
h2o_n_sink: int = 4
h2o_rope_base: float = 10000.0
h2o_grace: int = 16
h2o_decay: float = 0.98
class veloxquant_mlx.cache.options.TOVAOptions(tova_budget=512, tova_n_sink=4, tova_backend='auto')[source]

Bases: object

TOVA-adapted configuration (current-step attention-weight eviction, memoryless).

Parameters:

  • tova_budget (int)

  • tova_n_sink (int)

  • tova_backend (str)

tova_budget: int = 512
tova_n_sink: int = 4
tova_backend: str = 'auto'
class veloxquant_mlx.cache.options.PyramidKVOptions(pyramid_budget=512, pyramid_backend='reference', pyramid_n_sink=4, pyramid_beta=2.0, pyramid_resolved_budget=None)[source]

Bases: object

PyramidKV-adapted configuration (layer-adaptive budget attention-mass eviction).

Parameters:

  • pyramid_budget (int)

  • pyramid_backend (str)

  • pyramid_n_sink (int)

  • pyramid_beta (float)

  • pyramid_resolved_budget (int | None)

pyramid_budget: int = 512
pyramid_backend: str = 'reference'
pyramid_n_sink: int = 4
pyramid_beta: float = 2.0
pyramid_resolved_budget: int | None = None
class veloxquant_mlx.cache.options.SqueezeAttentionOptions(squeeze_budget=512, squeeze_n_sink=4, squeeze_strength=1.0, squeeze_resolved_budget=None)[source]

Bases: object

SqueezeAttention-adapted configuration (2D layer×token data-driven budget eviction).

Parameters:

  • squeeze_budget (int)

  • squeeze_n_sink (int)

  • squeeze_strength (float)

  • squeeze_resolved_budget (int | None)

squeeze_budget: int = 512
squeeze_n_sink: int = 4
squeeze_strength: float = 1.0
squeeze_resolved_budget: int | None = None
class veloxquant_mlx.cache.options.ChunkKVOptions(chunkkv_budget=512, chunkkv_chunk_size=8, chunkkv_n_sink=4, chunkkv_score='attn_mass', chunkkv_reuse_layers=1)[source]

Bases: object

ChunkKV-adapted configuration (chunk-level / semantic-block eviction).

Parameters:

  • chunkkv_budget (int)

  • chunkkv_chunk_size (int)

  • chunkkv_n_sink (int)

  • chunkkv_score (str)

  • chunkkv_reuse_layers (int)

chunkkv_budget: int = 512
chunkkv_chunk_size: int = 8
chunkkv_n_sink: int = 4
chunkkv_score: str = 'attn_mass'
chunkkv_reuse_layers: int = 1
class veloxquant_mlx.cache.options.CaMOptions(cam_budget=512, cam_n_sink=4, cam_merge='sim_weighted', cam_merge_keys=False, cam_merge_gate=True)[source]

Bases: object

CaM-adapted configuration (cache merging — merge evicted tokens, not drop).

Parameters:

  • cam_budget (int)

  • cam_n_sink (int)

  • cam_merge (str)

  • cam_merge_keys (bool)

  • cam_merge_gate (bool)

cam_budget: int = 512
cam_n_sink: int = 4
cam_merge: str = 'sim_weighted'
cam_merge_keys: bool = False
cam_merge_gate: bool = True
class veloxquant_mlx.cache.options.XKVOptions(xkv_group_size=2, xkv_rank=None, xkv_energy_threshold=0.95, xkv_latent_bits=4, xkv_group_quant_size=32, xkv_max_ctx=8192)[source]

Bases: object

xKV configuration (cross-layer shared-subspace key compression).

Parameters:

  • xkv_group_size (int)

  • xkv_rank (int | None)

  • xkv_energy_threshold (float)

  • xkv_latent_bits (int)

  • xkv_group_quant_size (int)

  • xkv_max_ctx (int)

xkv_group_size: int = 2
xkv_rank: int | None = None
xkv_energy_threshold: float = 0.95
xkv_latent_bits: int = 4
xkv_group_quant_size: int = 32
xkv_max_ctx: int = 8192
class veloxquant_mlx.cache.options.NSNQuantOptions(nsn_bits=2, nsn_residual_length=64, nsn_codebook_size=256, nsn_subvector_dim=8, nsn_seed=1234, nsn_max_ctx=8192)[source]

Bases: object

NSNQuant configuration (calibration-free universal-codebook VQ).

Parameters:

  • nsn_bits (int)

  • nsn_residual_length (int)

  • nsn_codebook_size (int)

  • nsn_subvector_dim (int)

  • nsn_seed (int)

  • nsn_max_ctx (int)

nsn_bits: int = 2
nsn_residual_length: int = 64
nsn_codebook_size: int = 256
nsn_subvector_dim: int = 8
nsn_seed: int = 1234
nsn_max_ctx: int = 8192
class veloxquant_mlx.cache.options.L2NormOptions(knorm_budget=512, knorm_n_sink=4, knorm_recent=0, knorm_keep='low')[source]

Bases: object

L2Norm-adapted configuration (intrinsic key-norm eviction).

Parameters:

  • knorm_budget (int)

  • knorm_n_sink (int)

  • knorm_recent (int)

  • knorm_keep (str)

knorm_budget: int = 512
knorm_n_sink: int = 4
knorm_recent: int = 0
knorm_keep: str = 'low'
class veloxquant_mlx.cache.options.SKVQOptions(skvq_bits_key=2, skvq_bits_value=2, skvq_group_size=32, skvq_window=128, skvq_n_sink=5, skvq_reorder=True, skvq_clip_search=True, skvq_clip_alpha=1.0, skvq_max_ctx=None)[source]

Bases: object

SKVQ-adapted configuration (sliding-window reorder + clip quant).

Parameters:

  • skvq_bits_key (int)

  • skvq_bits_value (int)

  • skvq_group_size (int)

  • skvq_window (int)

  • skvq_n_sink (int)

  • skvq_reorder (bool)

  • skvq_clip_search (bool)

  • skvq_clip_alpha (float)

  • skvq_max_ctx (int | None)

skvq_bits_key: int = 2
skvq_bits_value: int = 2
skvq_group_size: int = 32
skvq_window: int = 128
skvq_n_sink: int = 5
skvq_reorder: bool = True
skvq_clip_alpha: float = 1.0
skvq_max_ctx: int | None = None
class veloxquant_mlx.cache.options.QFiltersOptions(qfilters_budget=512, qfilters_n_sink=4, qfilters_recent=0, qfilters_calib_tokens=128, qfilters_sign=1, qfilters_min_retention=0.9, qfilters_on_low_retention='warn')[source]

Bases: object

Q-Filters-adapted configuration (query-agnostic projection eviction).

Parameters:

  • qfilters_budget (int)

  • qfilters_n_sink (int)

  • qfilters_recent (int)

  • qfilters_calib_tokens (int)

  • qfilters_sign (int)

  • qfilters_min_retention (float | None)

  • qfilters_on_low_retention (Literal['warn', 'raise', 'ignore'])

qfilters_budget: int = 512
qfilters_n_sink: int = 4
qfilters_recent: int = 0
qfilters_calib_tokens: int = 128
qfilters_sign: int = 1
qfilters_min_retention: float | None = 0.9
qfilters_on_low_retention: Literal['warn', 'raise', 'ignore'] = 'warn'
class veloxquant_mlx.cache.options.KeyformerOptions(keyformer_budget=512, keyformer_n_sink=4, keyformer_recent=0, keyformer_tau=None, keyformer_tau_init=1.0, keyformer_tau_end=1.0, keyformer_anneal_steps=0, keyformer_rope_base=10000.0, keyformer_seed=0)[source]

Bases: object

Keyformer-adapted configuration (Gumbel-regularized eviction).

Parameters:

  • keyformer_budget (int)

  • keyformer_n_sink (int)

  • keyformer_recent (int)

  • keyformer_tau (float | None)

  • keyformer_tau_init (float)

  • keyformer_tau_end (float)

  • keyformer_anneal_steps (int)

  • keyformer_rope_base (float)

  • keyformer_seed (int)

keyformer_budget: int = 512
keyformer_n_sink: int = 4
keyformer_recent: int = 0
keyformer_tau: float | None = None
keyformer_tau_init: float = 1.0
keyformer_tau_end: float = 1.0
keyformer_anneal_steps: int = 0
keyformer_rope_base: float = 10000.0
keyformer_seed: int = 0
class veloxquant_mlx.cache.options.MorphKVOptions(morphkv_budget=512, morphkv_n_sink=4, morphkv_window=8)[source]

Bases: object

MorphKV-adapted configuration (recent-window correlation retention).

Parameters:

  • morphkv_budget (int)

  • morphkv_n_sink (int)

  • morphkv_window (int)

morphkv_budget: int = 512
morphkv_n_sink: int = 4
morphkv_window: int = 8
class veloxquant_mlx.cache.options.KVzipOptions(kvzip_budget=512, kvzip_n_sink=4, kvzip_probe='context')[source]

Bases: object

KVzip-adapted configuration (context-reconstruction reliance eviction).

Parameters:

  • kvzip_budget (int)

  • kvzip_n_sink (int)

  • kvzip_probe (str)

kvzip_budget: int = 512
kvzip_n_sink: int = 4
kvzip_probe: str = 'context'
class veloxquant_mlx.cache.options.KVTCOptions(kvtc_bit_budget=512, kvtc_bit_choices=(0, 1, 2, 3, 4, 6, 8), kvtc_beta=3.5)[source]

Bases: object

KVTC-adapted configuration (local PCA + DP-optimal bit allocation + entropy coding).

Parameters:

  • kvtc_bit_budget (int)

  • kvtc_bit_choices (tuple)

  • kvtc_beta (float)

kvtc_bit_budget: int = 512
kvtc_bit_choices: tuple = (0, 1, 2, 3, 4, 6, 8)
kvtc_beta: float = 3.5
class veloxquant_mlx.cache.options.CurDKVOptions(curdkv_budget=512, curdkv_n_sink=4, curdkv_rank_cap=16, curdkv_rope_base=10000.0)[source]

Bases: object

CurDKV-adapted configuration (value-aware leverage-score eviction).

Parameters:

  • curdkv_budget (int)

  • curdkv_n_sink (int)

  • curdkv_rank_cap (int)

  • curdkv_rope_base (float)

curdkv_budget: int = 512
curdkv_n_sink: int = 4
curdkv_rank_cap: int = 16
curdkv_rope_base: float = 10000.0
class veloxquant_mlx.cache.options.NestedKVOptions(nestedkv_budget=512, nestedkv_n_sink=4, nestedkv_window=64, nestedkv_beta=3.0, nestedkv_tau=0.6, nestedkv_kappa=10.0, nestedkv_safeguard_alpha=0.2)[source]

Bases: object

NestedKV-adapted configuration (multi-scale ensembled prefill eviction; no verified venue).

Parameters:

  • nestedkv_budget (int)

  • nestedkv_n_sink (int)

  • nestedkv_window (int)

  • nestedkv_beta (float)

  • nestedkv_tau (float)

  • nestedkv_kappa (float)

  • nestedkv_safeguard_alpha (float)

nestedkv_budget: int = 512
nestedkv_n_sink: int = 4
nestedkv_window: int = 64
nestedkv_beta: float = 3.0
nestedkv_tau: float = 0.6
nestedkv_kappa: float = 10.0
nestedkv_safeguard_alpha: float = 0.2
class veloxquant_mlx.cache.options.AMCOptions(amc_k_high=0.2, amc_k_mid=0.3, amc_use_query_saliency=False, amc_query_alpha=0.5, amc_adaptive_thresholds=False, amc_threshold_window=64, amc_gamma=0.1, amc_calib_variance=None, amc_group_size=32)[source]

Bases: object

AMC-adapted configuration (saliency-driven tiered rank+precision; no verified venue).

Parameters:

  • amc_k_high (float)

  • amc_k_mid (float)

  • amc_use_query_saliency (bool)

  • amc_query_alpha (float)

  • amc_adaptive_thresholds (bool)

  • amc_threshold_window (int)

  • amc_gamma (float)

  • amc_calib_variance (float | None)

  • amc_group_size (int)

amc_k_high: float = 0.2
amc_k_mid: float = 0.3
amc_use_query_saliency: bool = False
amc_query_alpha: float = 0.5
amc_adaptive_thresholds: bool = False
amc_threshold_window: int = 64
amc_gamma: float = 0.1
amc_calib_variance: float | None = None
amc_group_size: int = 32
class veloxquant_mlx.cache.options.A2ATSOptions(a2ats_codebook_bits=8, a2ats_sub_dim=8, a2ats_window=128, a2ats_b=2048, a2ats_use_query_aware=True, a2ats_beta=0.5, a2ats_retrieval_fraction=0.2, a2ats_rope_base=10000.0, a2ats_rope_scaling=None, a2ats_query_h=None, a2ats_codebook=None)[source]

Bases: object

A2ATS-adapted configuration (windowed RoPE + query-aware VQ).

Parameters:

  • a2ats_codebook_bits (int)

  • a2ats_sub_dim (int)

  • a2ats_window (int)

  • a2ats_b (int)

  • a2ats_use_query_aware (bool)

  • a2ats_beta (float)

  • a2ats_retrieval_fraction (float)

  • a2ats_rope_base (float)

  • a2ats_rope_scaling (Any)

  • a2ats_query_h (Any)

  • a2ats_codebook (Any)

a2ats_codebook_bits: int = 8
a2ats_sub_dim: int = 8
a2ats_window: int = 128
a2ats_b: int = 2048
a2ats_use_query_aware: bool = True
a2ats_beta: float = 0.5
a2ats_retrieval_fraction: float = 0.2
a2ats_rope_base: float = 10000.0
a2ats_rope_scaling: Any = None
a2ats_query_h: Any = None
a2ats_codebook: Any = None
class veloxquant_mlx.cache.options.AnchorKVOptions(anchorkv_theta=0.05, anchorkv_window=32, anchorkv_rho=0.7, anchorkv_anchor_frac=0.0078125, anchorkv_residual_bits=2, anchorkv_seed=42)[source]

Bases: object

AnchorKV-adapted configuration (anchor-residual compression, no eviction; no verified venue).

Parameters:

  • anchorkv_theta (float)

  • anchorkv_window (int)

  • anchorkv_rho (float)

  • anchorkv_anchor_frac (float)

  • anchorkv_residual_bits (int)

  • anchorkv_seed (int)

anchorkv_theta: float = 0.05
anchorkv_window: int = 32
anchorkv_rho: float = 0.7
anchorkv_anchor_frac: float = 0.0078125
anchorkv_residual_bits: int = 2
anchorkv_seed: int = 42
class veloxquant_mlx.cache.options.RocketKVOptions(rocketkv_compression_ratio=8.0, rocketkv_page_size=None, rocketkv_head_topk1=None, rocketkv_obs_window=32, rocketkv_n_sink=4)[source]

Bases: object

RocketKV-adapted configuration (two-stage: SnapKV eviction + hybrid sparse attention).

Parameters:

  • rocketkv_compression_ratio (float)

  • rocketkv_page_size (int | None)

  • rocketkv_head_topk1 (int | None)

  • rocketkv_obs_window (int)

  • rocketkv_n_sink (int)

rocketkv_compression_ratio: float = 8.0
rocketkv_page_size: int | None = None
rocketkv_head_topk1: int | None = None
rocketkv_obs_window: int = 32
rocketkv_n_sink: int = 4
class veloxquant_mlx.cache.options.AgeTieredKVOptions(age_recent_boundary=128, age_mid_boundary=1024, age_bits_recent=8, age_bits_mid=4, age_bits_old=2, age_group_size=32)[source]

Bases: object

AgeTieredKV configuration (position/age-gated 3-tier precision, issue #256).

Parameters:

  • age_recent_boundary (int)

  • age_mid_boundary (int)

  • age_bits_recent (int)

  • age_bits_mid (int)

  • age_bits_old (int)

  • age_group_size (int)

age_recent_boundary: int = 128
age_mid_boundary: int = 1024
age_bits_recent: int = 8
age_bits_mid: int = 4
age_bits_old: int = 2
age_group_size: int = 32
class veloxquant_mlx.cache.options.SpectralQuantOptions(spectral_key_d_eff=4, spectral_val_d_eff=50, spectral_apply_qjl=True, spectral_model_name='model')[source]

Bases: object

SpectralQuant configuration (data-aware, calibration-based).

Parameters:

  • spectral_key_d_eff (int)

  • spectral_val_d_eff (int)

  • spectral_apply_qjl (bool)

  • spectral_model_name (str)

spectral_key_d_eff: int = 4
spectral_val_d_eff: int = 50
spectral_apply_qjl: bool = True
spectral_model_name: str = 'model'