An arXiv preprint posted October 20 introduces KernelClaw, a multi-agent system that achieved 100% success and speedups of up to 7.89× over PyTorch's default eager execution on GPU kernel benchmarks.
The results, if independently replicated, could lower the cost of training and running AI models by automating the labor-intensive optimization of GPU kernels—the small programs that execute on graphics processors.
Previous attempts to use large language models for GPU kernel optimization have relied on opaque heuristics learned by the model during training. The preprint argues this produces inefficient trial-and-error results. KernelClaw instead coordinates multiple agents with a dual‑memory architecture: a long‑term memory stores reusable expert knowledge about hardware bottlenecks and kernel structures, while a short‑term memory captures recent task‑specific performance data and error logs to prevent repetitive backtracking.
The accompanying GitHub repository details how the system works. Starting from a PyTorch forward-pass description, an LLM iteratively generates CUDA kernels. The short‑term memory is assembled from the most recent kernel versions, compilation errors, and profiling metrics collected by NVIDIA's Nsight Compute and Nsight Systems. The long‑term memory—stored under a memorybank/ directory—injects pre‑written knowledge about common optimization strategies, occupancy limits, and memory‑efficiency patterns into the prompt each round. The repository also implements automatic error repair: when a kernel fails to compile or produce correct numerical results, the system builds a repair prompt that includes historical failure logs and asks the LLM for a corrected version.
The preprint reports that KernelClaw was evaluated on KernelBench Levels 1 through 3. Across all tasks it attained a 100% success rate—meaning every task eventually produced a runnable and numerically correct kernel. Average speedups over Torch Eager were 6.98× on Level 1 (basic operators), 7.89× on Level 2 (small subnetworks), and 1.94× on Level 3 (complete models such as ResNet, VGG, LSTM and Transformer). In the Level 3 category, the paper states the framework outperformed all prior baselines.
No independent replication has been published, and the preprint has not undergone peer review. The release does not include hardware‑specific power‑consumption measurements or a breakdown of the LLM‑inference compute cost required to produce the optimized kernels. The source code, which the authors call KernelMem, is publicly available under the same GitHub repository, and the paper lists a nine‑author team from Chinese institutions.