veloxquant_mlx.profiling.hardware_profiler module

Apple-Silicon hardware detection for the auto-selection pipeline.

Extends the minimal veloxquant_mlx.config.auto_config.HardwareInfo (total/active memory) with everything the strategy planner needs from the machine: chip identity and generation, MLX and macOS versions, Metal availability, and (optionally) measured bandwidth/latency characteristics.

The detector never raises: if MLX is missing or device introspection fails it returns a best-effort HardwareProfile built from stdlib calls alone (platform), so callers degrade to unknown-hardware heuristics instead of crashing.

veloxquant_mlx.profiling.hardware_profiler.chip_generation(chip)

Extract the numeric generation from a chip name.

"M1" -> 1, "M4 Pro" -> 4, "M2 Ultra" -> 2. Returns 0 when no generation can be parsed (unknown or CPU-only host).

Return type:

int

Parameters:

chip (str)

class veloxquant_mlx.profiling.hardware_profiler.HardwareProfile(chip='unknown', chip_generation=0, total_memory_bytes=None, available_memory_bytes=0, mlx_version=None, macos_version=None, metal_available=True, peak_memory_bandwidth_gbps=None, avg_quantize_latency_ms_per_token=None)

Bases: object

Comprehensive description of the Apple-Silicon environment.

Unlike the minimal HardwareInfo, this is the object the auto-selection pipeline passes around: every field is a fact the planner can cite in an explanation.

Parameters:

• chip (str)

• chip_generation (int)

• total_memory_bytes (int | None)

• available_memory_bytes (int)

• mlx_version (str | None)

• macos_version (str | None)

• metal_available (bool)

• peak_memory_bandwidth_gbps (float | None)

• avg_quantize_latency_ms_per_token (float | None)

chip: str = 'unknown'

chip_generation: int = 0

total_memory_bytes: int | None = None

available_memory_bytes: int = 0

mlx_version: str | None = None

macos_version: str | None = None

metal_available: bool = True

peak_memory_bandwidth_gbps: float | None = None

avg_quantize_latency_ms_per_token: float | None = None

classmethod detect()

Build a HardwareProfile for the current machine (see detect_hardware_profile).

Return type:

HardwareProfile

property bandwidth_gbps: float | None

measured, else nominal, else None.

Type:

Likely peak bandwidth

to_dict()

Serialize all fields to a plain dict.

Return type:

dict

veloxquant_mlx.profiling.hardware_profiler.detect_hardware_profile()

Detect a HardwareProfile from MLX + stdlib introspection.

Process: 1. Call _mlx_device_info() for chip, memory, active allocation 2. Call _stdlib_chip() as fallback if MLX unavailable 3. Extract chip generation from chip name (M1–M4) 4. Calculate available memory (min(total, Metal working-set limit) - active) 5. Probe MLX version, macOS version, Metal availability 6. Look up nominal peak bandwidth from chip generation table

Robustness: - Never raises (all failures result in “unknown” fallback values) - MLX absence only affects chip/memory detection; OS fields come from stdlib - Metal unavailability doesn’t fail; conservative assume True - Missing config fields default to None/0

Return type:

HardwareProfile

Returns:

HardwareProfile with: - chip: Detected Apple Silicon model (e.g., “Apple M4”) - chip_generation: Integer 1–4 (0 if unknown) - total_memory_bytes: GB from MLX (None if unavailable) - available_memory_bytes: min(total, Metal working-set limit) - active usage - mlx_version: MLX package version (None if not installed) - macos_version: macOS version string - metal_available: Whether Metal GPU support is available - peak_memory_bandwidth_gbps: Nominal peak (from table, generation-based)

Cost: ~5–10ms (includes sysctl call for chip brand)

Example:

>>> hw = detect_hardware_profile()
>>> print(f"{hw.chip} (gen {hw.chip_generation})")
Apple M4 (gen 4)
>>> print(f"Available: {hw.available_memory_bytes / 1e9:.1f} GiB")
Available: 18.2 GiB

veloxquant_mlx.profiling.hardware_profiler.measure_bandwidth_gbps(iterations=32, bytes_per_transfer=67108864)

Measure approximate peak GPU memory bandwidth via a copy benchmark.

Benchmark: Allocates a tensor, performs repeated additions (forcing GPU memory copy), measures wall time, and computes GB/s throughput.

Method: 3 untimed warmup iterations first (VeloxQuant-MLX#509 – a cold first call in a fresh process measured ~2x under this function’s own warm-process mean, since the timed loop otherwise pays JIT/allocator warmup itself), then for each of N timed iterations: 1. Allocate src tensor (fp16, bytes_per_transfer bytes) 2. Perform add operation (forces GPU read + write) 3. Evaluate result (force completion)

Return type:

float | None

Parameters:

• iterations (int)

• bytes_per_transfer (int)

Calculation: Bytes moved = (bytes_per_transfer × 2) × iterations

(factor 2 for read + write)

Time = wall clock Bandwidth = bytes_moved / time / 1e9 (convert to GB/s)

Arguments:

iterations: Number of copy operations (default 32, ~2 seconds on M4) bytes_per_transfer: Size per copy (default 64 MiB)

Returns:

Measured bandwidth in GB/s, or None if: - MLX not installed - Metal GPU unavailable - Operation times out - Any exception occurs

Cost: ~1–2 seconds (default parameters)

Accuracy: Rough proxy; affected by GPU clock throttling, thermal state, competing workloads. Used only to override nominal peak bandwidth table when user explicitly requests measurement.

Example:

>>> bw = measure_bandwidth_gbps()
>>> if bw:
...     print(f"Measured: {bw:.1f} GB/s")
... else:
...     print("Fallback to nominal peak")
Measured: 118.5 GB/s