MAI-Thinking-1
Microsoft AI · reasoning checkpoint
Facts describe MAI-Base-1, the pretrained base of MAI-Thinking-1; neither checkpoint is public (served via Microsoft Foundry).
Open in the comparison table Paper ↗
KV calculations use one sequence at 131,072 tokens and 16-bit cache elements. They include bounded sliding-window caches. This is retained cache payload, not peak prefill memory. The comparison table also offers sourced mixed-precision layouts, including quantization scales.
Release
- Released
2026-06-02
Source: blog · Microsoft Build 2026: Be yourself at work; byline
Jun 2, 2026 | Kyle Daigle
Post announces MAI-Thinking-1 in private preview on Foundry.
Date the weights (or, if earlier, the technical report) were first made public, YYYY-MM-DD, as dated by the publisher. A Hugging Face repository creation date is used only when no dated publisher page exists, with a note.
Parameters
- Total params
962B parameters
Source: paper · p.7 Table 1, MAI-Base-1 row
MAI-Base-1 34.7B 962B 78 6656 13312 3072 10240 8/512 8/80
Rounded to 1T in the report text.
Total parameter count as reported by the publisher. See params.note for whether embeddings, MTP heads or vision towers are included.
- Active params
34.7B parameters
Source: paper · p.7 Table 1, MAI-Base-1 row
MAI-Base-1 34.7B 962B 78 6656 13312 3072 10240 8/512 8/80
Rounded to 35B in the report text.
Parameters used per token in the forward pass, as reported by the publisher.
- Active non-embedding (stated)
Not disclosed
Active parameters excluding the input embedding and the output head (unembedding). Stated by the publisher, or computed from params.active by subtracting vocab × d_model for each of the embedding and output head that params.active is stated to include (basis inferred, note with the arithmetic). Not disclosed when the scope of params.active is not stated.
- Counting scope
Counts include the tied embedding (200,019 × 6656 ≈ 1.33B) once
Inferred from the source; see the explanation below.
Source: paper · p.7 Table 1, MAI-Base-1 row
MAI-Base-1 34.7B 962B 78 6656 13312 3072 10240 8/512 8/80
From Table 1 shapes: 39 MoE layers × 512 experts × 3·3072·5120 + 78 attention layers (80×128 Q/O, 8×128 K/V) + 39 dense FFNs (3·6656²) + latent projections + router + embedding = 962.1B total, 34.66B active.
What the publisher's counts include or exclude (MTP modules, embeddings, vision encoder, n-gram tables).
Shape
- Layers
78
Source: paper · p.7 §2.2.1
The final architecture of MAI-Base-1 has 78 layers.
Number of transformer blocks in the main backbone (attention or token mixer plus FFN), excluding MTP modules.
- d_model
6,656
Source: paper · p.7 Table 1, MAI-Base-1 row (Hidden)
MAI-Base-1 34.7B 962B 78 6656 13312 3072 10240 8/512 8/80
Residual stream width.
- Vocab
200,019
Source: paper · p.6 §2.1
we chose the o200k_base tokenizer with a vocabulary size of 200,019
Embedding table rows (vocab_size in config, may include padding).
- Tied embeddings
Yes
Source: paper · p.6 §2.1
the input and output embedding weights are tied
Input embedding and output head share weights.
- Tokenizer
o200k_base (OpenAI)
Source: paper · p.6 §2.1
we chose the o200k_base tokenizer with a vocabulary size of 200,019
Tokenizer type and lineage as described by the publisher (e.g. byte-level BPE shared with an earlier model).
- d_model / layers
85.3333
Calculated: d_model / layers
Width-to-depth ratio, d_model divided by the number of blocks.
Attention
- Token mixers
GQA, SWA
Source: paper · p.6 §2.1 Attention
pairing 5 local attention layers (Beltagy et al., 2020) with 1 global attention layer
Token-mixing layer types in the backbone, e.g. GQA, MLA, SWA, Gated DeltaNet, Mamba-2, DSA (DeepSeek sparse attention).
- Layer layout
5 SWA : 1 global
Inferred from the source; see the explanation below.
Source: paper · p.6 §2.1 Attention
pairing 5 local attention layers (Beltagy et al., 2020) with 1 global attention layer
The last layer of each group of 6 is global (p.7 §2.2.1). Ordered from the reported last-layer-of-each-group rule; no public checkpoint config.
Token-mixer order, summarized as a pattern or layer positions. The fact's optional sequence lists each mixer in order; a ratio alone does not establish the order. Auxiliary prediction modules and standalone FFNs are excluded.
- Global-attention layers
13
Inferred from the source; see the explanation below.
Source: paper · p.7 §2.2.1
For the 5:1 local to global attention ratio that we use, L must be a multiple of 6 so that the last attention layer applies global attention.
78 layers / 6 = 13 global.
Layers whose KV cache grows with full context (dense or sparse global attention, MLA). Excludes sliding-window and linear/recurrent layers.
- SWA : global
5
Calculated: attn.local_layers / attn.global_layers
attn.local_layers divided by attn.global_layers, the sliding-window to global interleave ratio (5 for '5 SWA : 1 global').
- Sliding-window layers
65
Inferred from the source; see the explanation below.
Source: paper · p.7 §2.2.1
For the 5:1 local to global attention ratio that we use, L must be a multiple of 6 so that the last attention layer applies global attention.
78 layers × 5/6 = 65 local.
Layers with a fixed attention window.
- Query heads
80
Source: paper · p.7 Table 1, MAI-Base-1 row (KV/Q)
MAI-Base-1 34.7B 962B 78 6656 13312 3072 10240 8/512 8/80
Query heads in the global-attention layers.
- KV heads
8
Source: paper · p.6 §2.1 Attention
We use group-query attention with 8 KV heads (Ainslie et al., 2023) and a per-head dimension of 128.
Key/value heads in the global-attention layers. Not applicable to MLA, whose cache is a shared latent.
- Heads × head dim
10,240
Calculated: heads × head_dim
attn.heads × attn.head_dim, the query projection width of global layers.
- Q heads / KV head
10
Calculated: heads / kv_heads
attn.heads divided by attn.kv_heads in global layers.
- QK head dim
128
Source: paper · p.6 §2.1 Attention
We use group-query attention with 8 KV heads (Ainslie et al., 2023) and a per-head dimension of 128.
Query/key dimension per head in global-attention layers (for MLA, nope + rope dims).
- Window
512 tokens
Source: paper · p.6 §2.1 Attention
with sliding window size of 512 and a base frequency of 10,000
Sliding-window size of the local layers.
- SWA KV heads
8
Source: paper · p.6 §2.1 Attention
We use group-query attention with 8 KV heads (Ainslie et al., 2023) and a per-head dimension of 128.
Stated for all attention layers.
KV heads in sliding-window layers (stated even when equal to attn.kv_heads).
- SWA query heads
80
Source: paper · p.7 Table 1, MAI-Base-1 row (KV/Q)
MAI-Base-1 34.7B 962B 78 6656 13312 3072 10240 8/512 8/80
The report gives one head configuration for local and global layers.
Query heads in sliding-window layers (stated even when equal to attn.heads).
- SWA QK head dim
128
Source: paper · p.6 §2.1 Attention
We use group-query attention with 8 KV heads (Ainslie et al., 2023) and a per-head dimension of 128.
Stated for all attention layers.
Query/key head dimension in sliding-window layers. Value head dimensions may differ; see attn.local_v_head_dim.
- Positional encoding
RoPE (SWA), NoPE (global)
Source: paper · p.6 §2.1 Attention
The global attention layers do not use any position encoding
Positional scheme in global layers: RoPE, partial RoPE, NoPE, or mixed per layer type.
- SWA RoPE base
10,000
Source: paper · p.6 §2.1 Attention
with sliding window size of 512 and a base frequency of 10,000
RoPE base frequency of the sliding-window layers (stated even when equal to attn.rope_theta).
- QK norm
Yes
Source: paper · p.6 §2.1 Attention
We apply RMSNorms to the queries and the keys
Queries and keys are normalized per head before the dot product.
- QKV bias
No
Source: paper · p.6 §2.1
We do not use any biases in the model
Query/key/value projections have bias terms.
Feed-forward
- Activation
SwiGLU
Source: paper · p.6 §2.1 Feed-forward and MoE
we use SwiGLU (Shazeer, 2020) for both dense and MoE feed-forward layers
FFN activation (SwiGLU, GeGLU, ...).
- Dense FFN dim
6,656
Inferred from the source; see the explanation below.
Source: paper · p.7 Table 1, MAI-Base-1 row (FFN) and caption
The (Expert-)FFN sizes refer to the output size of the first linear layer, which is 2x of the input size for the second linear layer due to SwiGLU.
Table 1 FFN 13312 is the fused gate+up output; intermediate width 13312 / 2 = 6656. Dense FFNs alternate with MoE layers (39 each).
Intermediate width of the dense FFN layers that are actually executed.
- FFN bias
No
Source: paper · p.6 §2.1
We do not use any biases in the model
FFN projections have bias terms.
- Dense FFN / d_model
1
Calculated: ffn.dim / d_model
ffn.dim divided by shape.d_model.
Mixture of experts
- Routed experts
512
Source: paper · p.6 §2.1 Feed-forward and MoE
each compressed representation is routed to 8 out of the 512 experts with softmax gating
Routed experts per MoE layer.
- Experts per token
8
Source: paper · p.6 §2.1 Feed-forward and MoE
each compressed representation is routed to 8 out of the 512 experts with softmax gating
Routed experts selected per token.
- Expert dim
5,120
Inferred from the source; see the explanation below.
Source: paper · p.7 Table 1, MAI-Base-1 row (Expert FFN) and caption
The (Expert-)FFN sizes refer to the output size of the first linear layer, which is 2x of the input size for the second linear layer due to SwiGLU.
Table 1 Expert FFN 10240 is the fused gate+up output; intermediate width 10240 / 2 = 5120, matching the 962B total.
Intermediate width of one routed expert.
- Leading dense layers
1
Source: paper · p.6 §2.1 Feed-forward and MoE
Our first feedforward layer is dense
Thereafter dense FFN and MoE alternate layer by layer.
Number of initial blocks that use a dense FFN instead of MoE.
- Router score
softmax
Source: paper · p.6 §2.1 Feed-forward and MoE
each compressed representation is routed to 8 out of the 512 experts with softmax gating
Router input is the uncompressed hidden state.
Gating function over expert logits (softmax, sigmoid).
- Load balancing
global-batch GShard-style load-balancing loss (expert frequencies aggregated across DP ranks and micro-batches); dropless
Source: paper · p.6 §2.1 Feed-forward and MoE
The GShard-style loss (Lepikhin et al., 2020) that we use performs similarly to the loss-free variant
Load-balancing method, with coefficients where disclosed.
- Renormalize top-k
Not disclosed
Selected expert weights are renormalized to sum to 1.
- Expert latent dim
3,072
Source: paper · p.7 Table 1, MAI-Base-1 row (Down Proj)
MAI-Base-1 34.7B 962B 78 6656 13312 3072 10240 8/512 8/80
Shared down-projection before all-to-all dispatch, up-projection after combine (LatentMoE); changed from L78's 3328 so the dimension is divisible by 512 for DeepEP.
Width of the projected space routed experts operate in (LatentMoE), when smaller than d_model.
- Latent compression α (d_model / latent)
2.16667
Calculated: d_model / moe.latent_dim
shape.d_model divided by moe.latent_dim, how much LatentMoE compresses the hidden state before routed experts.
- Expert dim / d_model
0.769231
Calculated: moe.expert_dim / d_model
moe.expert_dim divided by shape.d_model.
- Sparsity (experts / top-k)
64
Calculated: experts / top_k
Routed experts divided by experts selected per token.
- Active / total
0.0360707
Calculated: params.active / params.total
params.active divided by params.total.
Multi-token prediction
- MTP depth
Not disclosed
Additional future tokens predicted during pretraining (0 when the model was trained without MTP).
Normalization and stability
- Norm
RMSNorm
Source: paper · p.6 §2.1
we employ RMSNorm (Zhang and Sennrich, 2019) at both the input and output immediately before the residual addition
Normalization layer type in the blocks.
- Norm placement
sandwich
Source: paper · p.6 §2.1
we employ RMSNorm (Zhang and Sennrich, 2019) at both the input and output immediately before the residual addition
Pre-norm, post-norm, or sandwich (pre and post).
- Norm epsilon
Not disclosed
Epsilon of the block normalization layers.
- Residual stream
FP32 residual stream
Source: paper · p.22 §2.6.3
the entire residual stream all the way from the input embedding to the output
Residual design when it is not a plain sum, e.g. hyper-connections with n streams, gated residuals, depth-scaled residuals, FP32 residual accumulation.
Context length
- Pretraining context
16.38K tokens
Source: paper · p.21 §2.6.1 Table 6
Pre-training 30 T 16,384 8,192
Sequence length of the main pretraining stage.
- Trained context
262.1K tokens
Source: paper · p.21 §2.6.1 Table 6
Mid-training 2 150 B 262,144 4,096
Longest sequence length trained on after context extension.
- Advertised context
262.1K tokens
Source: paper · p.2 §1
After mid-training, MAI-Thinking-1 achieves a maximum context length of 256K.
Longest context the publisher says the model supports, possibly beyond trained length via RoPE scaling.
- Extension method
16K → 64K (3.4T tokens, mid-training 1) → 256K (150B tokens, mid-training 2); same mixture re-packed at the longer length; Ulysses context parallelism
Source: paper · p.20 §2.5.4; p.21 Table 6; p.85–87 App. C
the model is pre-trained at 16K, mid-trained at 64K (Stage 1), and then extended to 256K in a short dedicated phase (Stage 2).
App. C.5 gives 140B tokens for the 256K extension; Table 6 gives 150B, which matches the 3.55T mid-training total in §2.
How context was extended (stages, RoPE base change, YaRN factor).
Pretraining recipe
- Pretraining tokens
33.55T tokens
Inferred from the source; see the explanation below.
Source: paper · p.5 §2
we ran a main pre-training phase of 30 trillion tokens, followed by mid-training phases totaling 3.55 trillion tokens
30T pre-training + 3.55T mid-training (Table 6: 3.4T + 150B).
Tokens seen in pretraining including annealing and context extension, as reported.
- Optimizer
AdamW
Source: paper · p.21 §2.6.2
MAI-Base-1 is optimized with AdamW (Loshchilov and Hutter, 2017) (β1 = 0.95, β2 = 0.925, ε = 10−8 )
Optimizer used for the bulk of parameters.
- Peak LR
0.0002
Source: paper · p.21 §2.6.2
then decays along a cosine schedule from a peak of 2 × 10−4 to a minimum of 2 × 10−5
Peak learning rate of the main pretraining stage.
- Final LR
2e-05
Source: paper · p.21 §2.6.2
then decays along a cosine schedule from a peak of 2 × 10−4 to a minimum of 2 × 10−5
End of pre-training; mid-training decays further to 1e-5, then 1e-6 constant.
Learning rate at the end of the decay.
- LR schedule
linear warmup ~12B tokens, cosine 2e-4 → 2e-5 (pre-training); cosine 2e-5 → 1e-5 (mid-training 1); constant 1e-6 (mid-training 2)
Source: paper · p.21–22 §2.6.2
Mid-training phase 1 uses a cosine decay learning rate schedule with a peak learning rate of 2 × 10−5 and a minimum of 1 × 10−5 , without warmup.
Learning-rate schedule shape with phase boundaries.
- Warmup
~12B tokens (linear)
Source: paper · p.21 §2.6.2
The learning rate follows a linear warmup of approximately 12 B tokens
Warmup length in steps or tokens.
- Batch size
134M tokens, all phases
Source: paper · p.22 §2.6.2
All phases use a global batch size of 134M tokens.
Global batch size, including any batch-size ramp: sequences × sequence length (token total), or the token total alone when only that is stated.
- Batch (tokens)
134M tokens
Source: paper · p.22 §2.6.2
All phases use a global batch size of 134M tokens.
Global batch size in tokens during the main pretraining stage, after any ramp (sequences × sequence length when stated in sequences, marked inferred).
- Weight decay
0.1
Source: paper · p.21 §2.6.2
a constant weight decay of 0.1, with reduced weight decay on attention weights (0.01) and embedding weights (0.005)
0.01 on attention weights, 0.005 on embedding weights.
Decoupled weight decay coefficient.
- Adam betas
(0.95, 0.925)
Source: paper · p.21 §2.6.2
MAI-Base-1 is optimized with AdamW (Loshchilov and Hutter, 2017) (β1 = 0.95, β2 = 0.925, ε = 10−8 )
Adam ε 1e-8.
(beta1, beta2) of the Adam-family optimizer or its AdamW fallback.
- Grad clip
1
Source: paper · p.21 §2.6.2
The global gradient norm is clipped to 1.0.
Global gradient-norm clipping threshold.
- Dropout
0.15
Source: paper · p.21 §2.6.2
We apply dropout 0.15 at each layer's output before the residual add.
Applied to each sublayer output before the residual add.
Dropout rate applied during pretraining, with placement in the note. Recorded only when the publisher states it.
- Training precision
FP8 with delayed scaling (1024-step amax history): E4M3 forward GEMMs, E5M2 data-gradient GEMMs; BF16 weight-gradient GEMMs with FP32 accumulation; BF16 default weights and activations
Source: paper · p.22 §2.6.3
We use FP8 E4M3 for GEMMs in the forward pass, FP8 E5M2 for the data-gradient, and BF16 for the compute of the weight-gradient with FP32 gradient accumulation
FP32 for residual stream, pre-softmax activations, router, MoE combine, embedding/norm/router weights, LM-head GEMM, master weights and AdamW state; stochastic rounding on gradient downcasts.
Matmul precision of pretraining (BF16, FP8 recipe, NVFP4 recipe) with the scaling granularity where disclosed. Never inferred from the released checkpoint dtype.
- Initialization
normal std 0.02; residual output projections scaled by 1/sqrt(number of residual connections); attention output zero-init via zero output-RMSNorm gains
Source: paper · p.21–22 §2.6.2
we initialize the attention output to zero, achieved by setting the output RMSNorm gains to zero.
Weight initialization (std or scheme) as stated by the publisher; config initializer_range alone is not evidence.
- Data mix
30.0T tokens: code 54.6% (16.4T, 2.22 epochs), STEM 15.8%, web text 14.9%, math 5.4% (5.28 epochs), PDFs 4.7%, books and journals 3.1%, multilingual 1.6%; no LM-generated synthetic data
Source: paper · p.20 Table 5
Code 7.4 16.4 54.6 2.22×
Mid-training mix: STEM/math 35%, code 55%, other 10% (p.20 §2.5.4).
Pretraining data composition as disclosed (domain shares, synthetic fraction, repetition).
- Data cutoff
web HTML Sep 2025, web PDFs Dec 2025, GitHub code Jun 2025, books and journals Mar 2026
Source: paper · p.13 Table 4
Web HTML pages September 2025
Pretraining data cutoff date.
- Hardware
8,192 GB200 (pre-training, mid-training 1), 4,096 GB200 (mid-training 2); EP64 within NVLink domain, ZeRO-2
Source: paper · p.21 §2.6.1
We use expert parallelism 64 with ZeRO-2 for pre-training and mid-training 1, and we enable ZeRO-3 (FSDP) for mid-training phase 2.
Accelerators used for pretraining, count and duration where stated.
- Accelerators
8,192 GB200
Source: paper · p.21 §2.6.1 Table 6
Phase Tokens Context length GB200 GPUs Pre-training 30 T 16,384 8,192
Pre-training and mid-training 1; mid-training 2 used 4,096.
Accelerator count and type for the main pretraining run, as the count with thousands separators and the chip's short name ("2,048 H800", "2,048 GH200"). A stated range or bound stays as stated ("2,048–4,096 GH200").
- Parallelism
EP64 · ZeRO-2 DP
Source: paper · p.21 §2.6.1
We use expert parallelism 64 with ZeRO-2 for pre-training and mid-training 1
§2.8 states TP degree 1 with EP inside the NVL64 domain and DP across racks; mid-training 2 adds ZeRO-3 (FSDP) and context parallelism.
Parallelism layout of the main pretraining run as stated, as degree-annotated terms in the order TP, PP, VPP, EP, CP, DP ("PP16 · EP64 · ZeRO-1 DP"); a term without a stated degree has none ("FSDP").
- Stages
pre-training
- Tokens
- 30000000000000
- Seq len
- 16384
- Lr
- 0.0002
- Batch tokens
- 134000000
Source: paper · p.21–22 §2.6.1 Table 6, §2.6.2
Pre-training 30 T 16,384 8,192
Peak LR; ~12B-token warmup, cosine to 2e-5. 8,192 GB200.
mid-training 1
- Tokens
- 3400000000000
- Seq len
- 65536
- Lr
- 2e-05
- Batch tokens
- 134000000
Source: paper · p.21–22 §2.6.1 Table 6, §2.6.2
Mid-training 1 3.4 T 65,536 8,192
Cosine 2e-5 → 1e-5, no warmup; STEM/math 35%, code 55%. 8,192 GB200.
mid-training 2
- Tokens
- 150000000000
- Seq len
- 262144
- Lr
- 1e-06
- Batch tokens
- 134000000
Source: paper · p.21–22 §2.6.1 Table 6, §2.6.2
Mid-training 2 150 B 262,144 4,096
Constant LR, no warmup. 4,096 GB200.
Source: paper · p.21–22 §2.6.1 Table 6, §2.6.2
Ordered pretraining stages. Each item: name, tokens, seq_len, lr, batch_tokens, batch_sequences, steps, note, quote, and source when it differs from the fact's source.
Scaling & HP transfer
- Scaling ladder
L12–L78 (365M/3.9B to 35.6B/1015B active/total): L layers, D = L·256/3, query heads L rounded up to a multiple of 16, 8 KV heads, 8/512 experts; architecture ablations at 100–200 TPP
Source: paper · p.7 §2.2.1, Table 1
most architecture ablations are conducted near the Chinchilla optimal region (Hoffmann et al., 2022) at 100-200 TPP
Table 1 rungs L12, L18, L24, L30, L36, L42, L66, L78; MAI-Base-1 is L78 with latent 3072 instead of 3328 and Expert FFN 10240 instead of 9984.
Smaller models trained to choose the architecture or recipe (sizes, token budgets), as disclosed.
- Tokens / param
34.8753
Calculated: train.tokens / params.total
train.tokens divided by params.total.
- AdamW timescale τ
0.199702
Calculated: train.batch_tokens / (train.lr_peak × train.weight_decay × train.tokens)
B / (η λ D) with B = train.batch_tokens, η = train.lr_peak, λ = train.weight_decay and D = train.tokens: the AdamW EMA timescale as a fraction of training (Bergsma et al. 2025, Power Lines, arXiv 2505.13738). Computed only when the optimizer is AdamW; uses the peak LR.
KV cache
- KV bytes / token
54,288 bytes
Calculated: kv.bytes_at_context / 131072 tokens; uniform BF16, payload only; quantization overhead excluded. Global KV: 13 × (2048 × 16/8) B/entry; SWA KV: 65 × (2048 × 16/8) B/entry
Retained KV cache divided by context length, including bounded SWA caches and indexer keys. Exports use uniform BF16 at 131,072 tokens. The website can select uniform payload precision (no quantization overhead) or sourced component formats (including scales and documented padding). This is cache payload, not peak prefill memory or persistent prefix storage.
- Global KV bytes / token
53,248 bytes
Calculated: 53248 B / 1 tokens (Global KV); uniform BF16, payload only; quantization overhead excluded. Global KV: 13 × (2048 × 16/8) B/entry; SWA KV: 65 × (2048 × 16/8) B/entry
Long-context global KV bytes per token, including indexer keys; excludes bounded SWA caches. Compression rates and cross-layer cache reuse are included. Exports use uniform BF16; the website's storage selector also applies here. This is the field to compare with global-KV figures in reports, using the matching storage recipe.
- KV bytes / token / B active
1,564 bytes
Calculated: kv.bytes_per_token (131072 tokens) / (params.active / 1e9); uniform BF16, payload only; quantization overhead excluded. Global KV: 13 × (2048 × 16/8) B/entry; SWA KV: 65 × (2048 × 16/8) B/entry
kv.bytes_per_token divided by active parameters in billions (131,072 tokens in exports).
- KV at context / B active
205,061,577 bytes
Calculated: kv.bytes_at_context (131072 tokens) / (params.active / 1e9); uniform BF16, payload only; quantization overhead excluded. Global KV: 13 × (2048 × 16/8) B/entry; SWA KV: 65 × (2048 × 16/8) B/entry
kv.bytes_at_context divided by active parameters in billions (131,072 tokens in exports).
- KV at context
7,115,636,736 bytes
Calculated: 53248 B × 131072 (Global KV) + 266240 B × min(131072, 512) (SWA KV); uniform BF16, payload only; quantization overhead excluded. Global KV: 13 × (2048 × 16/8) B/entry; SWA KV: 65 × (2048 × 16/8) B/entry
Retained KV cache for one sequence at the selected context, including SWA windows and indexer keys. Exports use uniform BF16 at 131,072 tokens. Excludes recurrent state, compressor tail state, draft/MTP caches, temporary prefill tensors, allocator metadata and persistent prefix snapshots. Does not estimate peak prefill or total serving memory.