Researchers posted a preprint on arXiv proposing a method to speed up sparse attention in large language models by removing a bottleneck in how tokens are selected for processing. The Multi-layer Indexer1 (MLI) targets the DeepSeek Sparse Attention mechanism used in DeepSeek-V3.2 and the GLM-5 model family, according to the paper.
The method matters for inference economics because the indexer that selects which tokens to attend to becomes the primary speed limit at long context lengths. The paper states that this scan, rather than the attention computation itself, consumes the decode time as context scales.
The authors observe that adjacent layers in these models select nearly the same tokens. They exploit this by having a layer attend to the token selection made by the previous layer's indexer, removing the sequential dependency. This allows each indexer to run concurrently with its layer's attention computation. The method requires no retraining or new weights, and its default implementation runs within SGLang's production decode graph.
When combined with IndexCache, a cross-layer reuse technique the paper says GLM-5.2 ships as IndexShare, MLI hides the indexer scan on the layers that still use one. At equal retention, the MLI and IndexCache composition decodes up to 14.5 percent faster than IndexCache alone on DeepSeek-V3.2 and 9.8 percent faster on GLM-5 at a 1/2 retention rate. At 1/4 retention, the gains are 7.9 percent and 5.1 percent, respectively. At a 1/4 retention rate and 128K context, the method reaches 1.32 times the throughput of the unmodified model, according to the paper.
The authors say accuracy on the LongBench, RULER and LongBench v2 benchmarks remains within a few percentage points of IndexCache's results. The paper is a preprint and has not been peer-reviewed. It has been accepted to the Long-Context Foundation Models Workshop at NeurIPS 2026, scheduled for December 12 in Atlanta.