WebGPU vs WASM: In-Browser LLM Inference Latency & Memory Benchmarks

Empirical Benchmark: WebGPU vs WASM SIMD (Batch Size = 1)

Testing hardware: Apple M3 Max (36GB Unified RAM) vs Intel Core i7-13700H (32GB DDR5). Chrome 128 execution.

1. Why Autoregressive LLMs Starve on WebAssembly

In large language model decoding, computing each output token requires streaming every model weight tensor through the processor once (Arithmetic Intensity ≈ 1 FLOP/byte). In a CPU environment executing WebAssembly, memory bandwidth is bottlenecked by the main system DRAM bus (typically 50-90 GB/s on desktop DDR5) and shared across all background OS processes.

In contrast, dedicated client GPUs provide 300 to 1,000+ GB/s of VRAM bandwidth, while Apple Silicon unified architectures deliver 150 to 400 GB/s with direct zero-copy GPU access. WebGPU allows WebGPU Shading Language (WGSL) workgroups to access this bandwidth directly with minimal context switching.

2. WGSL Compute Kernel Implementation: Quantized W4A16 GEMM

The code snippet below demonstrates how modern in-browser runtimes execute 4-bit dequantization and dot-product accumulation in a single WGSL compute shader pass:

@group(0) @binding(0) var<storage, read> weights_q4: array<u32>;
@group(0) @binding(1) var<storage, read> scales_f16: array<f32>;
@group(0) @binding(2) var<storage, read> input_act: array<f32>;
@group(0) @binding(3) var<storage, read_write> output_tensor: array<f32>;

var<workgroup> tile_input: array<f32, 64>;

@compute @workgroup_size(16, 4, 1)
fn matmul_w4a16(
  @builtin(workgroup_id) wg_id: vec3<u32>,
  @builtin(local_invocation_id) local_id: vec3<u32>,
  @builtin(global_invocation_id) global_id: vec3<u32>
) {
  let row = global_id.x;
  let col = global_id.y;
  var sum: f32 = 0.0;

  // Unpack 4-bit packed weights from uint32 (8 weights per word)
  let word_idx = (row * 64u + local_id.x) / 8u;
  let packed_val = weights_q4[word_idx];
  let shift = (local_id.x % 8u) * 4u;
  let raw_quant = f32((packed_val >> shift) & 0x0fu) - 8.0;

  let scale = scales_f16[row];
  let dequant_weight = raw_quant * scale;

  sum += dequant_weight * input_act[local_id.x];
  workgroupBarrier();

  if (local_id.x == 0u && local_id.y == 0u) {
    output_tensor[row] = sum;
  }
}

3. 2026 Browser Support & Extension Status

Browser deployment requires checking feature flags and extension support before initializing client pipelines: