Qualcomm Hardware Benchmarks

Measured on Snapdragon® X Elite (X1E80100) 12-Core Oryon CPU, Windows 11 ARM64, using python examples/benchmark.py.

1. Dense Layer Matrix Multiplication (GEMM + ReLU)

y = jax.nn.relu(jnp.matmul(x, w) + b) — 50-100 sustained iterations, averaged

Matrix Dimension	Host CPU (Measured)	QNN Backend (Measured)	Speedup
256 × 256	0.38 ms (2,639 FPS)	0.42 ms (2,353 FPS)	~1.0x
512 × 512	1.83 ms (546 FPS)	1.61 ms (623 FPS)	1.1x
1024 × 1024	6.07 ms (165 FPS)	6.12 ms (163 FPS)	~1.0x

ℹ️ Note: The QNN backend currently routes through the CPU reference runtime (QnnCpu.dll). When the full Hexagon NPU (HTP) hardware path is activated via QnnHtp.dll, significantly higher speedups are expected due to VTCM on-chip memory and hardware op-fusion.

2. Hardware Architecture Advantages (When NPU Path Is Enabled)

The Qualcomm Hexagon NPU is designed for high-throughput tensor workloads:

Feature	CPU (Oryon)	Hexagon NPU (HTP)
Memory	LPDDR5x (shared)	VTCM on-chip SRAM (TB/s bandwidth)
Op Fusion	None (sequential dispatch)	MatMul + Bias + ReLU fused in hardware
Peak Throughput	Reference baseline	45 TOPS (INT8) / 22.5 TFLOPS (FP16)

3. Reproducing Locally