A reference note — companion to the interactive trace lesson on the same mechanism.
Yes — a score is the raw material of attention, before it becomes a weight.
| Stage | What it is | Formula shape |
|---|---|---|
| Score / logit | Raw compatibility between a query and a key | q · k |
| Weight | Score turned into a probability | softmax(scores) |
| Attention output | Weighted sum of values | Σ weight · v |
Full attention: Attn(q,K,V) = softmax(qKᵗ/√d)V. Sparse attention reuses that shape twice, at two price points: a cheap indexer score that decides what's worth looking at, then the real, expensive score computed only for the winners.
Then a hard top-k cutoff decides who survives — DeepSeek-V3.2's real, published number is k = 2048 out of the whole context.
The catch: the indexer still has to look at every token to score it — so indexing itself stays O(L²) even though the real attention afterward is only O(L·k). LongCat's README names this directly as "the output discontinuity and quadratic scoring bottleneck of the Lightning Indexer in DSA" — the exact problem LSA exists to fix.
Three orthogonal fixes to the indexer's cost — not a change to the score → select → attend idea itself.
| Mechanism | What it does | Why |
|---|---|---|
| Streaming-aware Indexing (SI) | Combines hardware-aligned contiguous access with dynamic random selection | Random-access reads into a huge KV cache are slow (uncoalesced HBM access); SI turns fragmented reads into contiguous runs |
| Cross-Layer Indexing (CLI) | One indexing pass serves several consecutive layers — shared every 2 layers; all 3 MTP draft steps share a single pass | Attention saliency is empirically stable across adjacent layers, trained in via cross-layer distillation |
| Hierarchical Indexing (HI) | Coarse block-level score first, fine token-level selection only within surviving blocks | Shrinks the candidate pool before the expensive per-token scoring step |
How streaming vs. full-history tokens actually get separated — a general pattern across this whole family (DSA, LSA, StreamingLLM, NSA), not unique to one indexing trick:
Final attention = local_window ∪ top_k_selected_history. LSA's Streaming-aware Indexing is specifically about making the memory access pattern hardware-friendly for whichever tokens get selected — not a separate semantic rule for what counts as local.
Easy to lump in with LSA since both say "sparse" — they're unrelated mechanisms.
N-gram Embedding scales embedding parameters cheaply by giving every token an extra vector, looked up by hashing its last few token IDs. LongCat-2.0 inherits it from LongCat-Flash-Lite.
NE0(t_i) is the ordinary token embedding. H_n hashes the last n token IDs into a bucket (base V0 = vocab size, modulus V_n = table size); E_n looks that bucket up in a separate table and adds it on top. LongCat-2.0: n up to 5, 135B parameters in this table.
Why it matters for the "does it sparsify attention" question: it's sparsity in parameters (via hashing), fully orthogonal to LSA's sparsity in attention computation. Two different axes, bolted onto the same model.
Opposite bottlenecks, from the LongCat-Flash paper (§6.2.2).
| Phase | What happens | Bottleneck | Optimized via |
|---|---|---|---|
| Prefill | Process the whole input prompt at once, build the initial KV cache | Compute-bound | Chunked prefill |
| Decode | Generate one token at a time, autoregressively | Memory-bound (small/irregular batches) | KV-cache layout, custom kernels — where LSA/DSA pay off most |
Two latency metrics fall out of the split: TTFT (time-to-first-token, prefill) and TPOT (time-per-output-token, decode). Because the two phases have opposite resource profiles, LongCat runs them on separate hardware pools — a PD-disaggregated architecture — trading network cost (shipping the KV cache over) for independent tuning of each pool.
Training/serving infrastructure, not attention — grounded in what's published, with secondary reporting flagged.
Domestic (non-Nvidia) Chinese ASICs, reportedly wired in units of up to 48 machines — all-to-all high bandwidth inside the pod, RoCE fabric between pods. Same idea as Nvidia's "SuperPOD" branding, on Meituan's own chips.
TP (tensor) · CP (context) · EP (expert) · DP (data) · PP (pipeline) — five standard axes — plus a new sixth, EMBP, invented specifically to shard the 135B-parameter N-gram Embedding table.
An Adam alternative: orthogonalizes the momentum-accumulated gradient of a whole weight matrix (via Newton-Schulz iteration) instead of scaling each parameter independently. Reported "customized" large-scale deployment with TP-parallel and DP-state-redundancy optimizations.
It's not part of any standard parallelism taxonomy elsewhere — it exists specifically because N-gram Embedding (§3) is big enough to need its own splitting strategy, separate from how MoE experts are split.
q·k), different job: cheap filter vs. expensive compute.