Inside MAI-Thinking-1: The Hill-Climbing Machine

Most model launches invite us to stare at the finish line: benchmark scores, context length, parameter count, and a few carefully chosen comparisons. Microsoft’s technical report for MAI-Thinking-1 is more interesting because it spends much of its time on the machinery behind the result.

The model is the first output of what Microsoft calls a Hill-Climbing Machine: an integrated system for improving data, architecture, training infrastructure, reinforcement learning, evaluation, and safety through repeated empirical loops. The central idea is simple but demanding: frontier-model development should look less like one heroic training run and more like an instrumented optimization process.

This article is therefore best read as a technical and architectural explanation, not a product buyer’s guide. The important question is not whether MAI-Thinking-1 tops every leaderboard. It does not. The important question is what its development process teaches us about scaling decisions that survive contact with larger models, longer runs, and real hardware.

What MAI-Thinking-1 actually is

MAI-Thinking-1 is a sparse Mixture-of-Experts reasoning model with approximately 1 trillion total parameters and 35 billion active parameters per token. Its base model, MAI-Base-1, was pretrained from scratch on 30 trillion tokens, followed by 3.55 trillion tokens of mid-training, using 8,000 GB200 GPUs on a Microsoft-operated Azure cluster. After mid-training, the model supports a context length of up to 256K tokens.

Microsoft reports 52.8% on SWE-Bench Pro, 97.0% on AIME 2025, and 87.7% on LiveCodeBench v6. Those results establish that the model is competitive in coding and STEM reasoning for its active size. But they are outputs of the system—not the system itself.

The report does not show a model that dominates every long-horizon benchmark. That is part of what makes the engineering story useful: Microsoft traded the shortcut of third-party distillation for greater control over data provenance and a cleaner account of where capabilities came from. Whether that trade-off ultimately closes the remaining gaps is an open question, not a settled result.

A useful mental model is a scientific climbing expedition. Each checkpoint is a camp, not the summit. The team changes one part of the route, measures whether the change produces a real gain, checks whether that gain survives at greater scale, and only then carries the winning recipe forward.

The philosophy rests on three principles:

1. Capabilities should be learned, not inherited. Microsoft avoided distillation from third-party models and did not intentionally use language-model-generated data during pretraining.
2. Simplicity is sustainable. The team preferred recipes that could remain stable across long runs rather than techniques that produced fragile short-term gains.
3. Scientific rigor avoids shortcuts. Decisions were tested with scaling ladders, ablations, held-out evaluations, and hardware measurements.

The wording matters. “No synthetic data” applies to pretraining, not the entire model-development lifecycle. Post-training explicitly uses rollouts generated by Microsoft’s own models for self-distillation and specialist consolidation.

The key lesson: small experiments can choose the wrong winner

Ablations are essential because full-scale experiments are extraordinarily expensive. The danger is assuming that a result observed on a small proxy model will preserve the same ranking at the final scale.

Microsoft’s data-mixture experiments exposed exactly this failure mode. At smaller scales, a STEM-heavy mixture outperformed a code-heavy mixture on STEM-focused NLL evaluations. After scaling to a 23-billion-active-parameter model trained for roughly 20 trillion tokens, the ordering reversed: the code-heavy mixture became better on those same STEM distributions.

That does not prove that “code is always better than STEM data.” It demonstrates a narrower and more valuable point: data mixtures have scale-dependent learning curves.

Why the ranking can flip

Imagine two reservoirs:

• One contains highly concentrated, high-quality material but relatively limited variety.
• The other is harder to learn from initially but contains far more structural diversity.

The first reservoir can produce faster early gains. Over a sufficiently long training run, however, repeated exposure yields diminishing information. The second reservoir may continue supplying novel structures and relationships after the first has begun to saturate.

The report connects the reversal partly to duplication and diversity. A high-quality dataset can still become less useful when it is repeated too aggressively. Code, meanwhile, can expose the model to compositional structure, dependencies, state changes, abstraction, and verification patterns that transfer beyond code generation.

The practical rule is not to abandon small ablations. It is to treat them as provisional evidence:

• Test candidates on a ladder of increasing model and token scales.
• Track the shape of each learning curve, not only its value at one checkpoint.
• Validate finalists close enough to the target scale to detect ranking reversals.
• Prefer mixtures whose gains remain stable as compute increases.

Microsoft reports that the compute used for one scaling-focused ablation program was approximately three times the compute used to train DeepSeek-V3. That comparison includes experimental ladders rather than one final run, but it shows how much compute can be spent reducing uncertainty before committing to a recipe.

Measuring the signal instead of generation noise

Public generative benchmarks are useful for measuring deployed behavior, but they can be noisy tools for pretraining decisions. A model may understand the underlying distribution and still fail because it sampled an unproductive reasoning path, used the wrong output format, or missed an evaluator-specific string.

Microsoft instead built 40 held-out negative log-likelihood (NLL) evaluations across coding, mathematics, STEM, knowledge, multilingual content, and other distributions.

For a sequence of correct tokens, NLL is:

NLL = −Σ log P(xᵢ | x₍<ᵢ₎)    for i = 1 to N

In plain English, NLL measures how surprised the model is by the next correct token. Lower is better. Because the metric scores probabilities against fixed text rather than asking the model to generate and format an answer, it provides a cleaner signal for comparing base models and data mixtures.

This does not make generative benchmarks unnecessary. It separates two questions:

A disciplined evaluation stack needs both. NLL helps isolate pretraining effects; downstream evaluations reveal whether those effects become useful behavior.

There is also a deeper methodological point: every optimization process climbs the hill defined by its measurements. Optimizing only MMLU, HumanEval, or whichever public benchmark is fashionable risks shaping the whole pipeline around a narrow target. Microsoft’s broader internal NLL suite does not eliminate measurement bias, but it gives the team more hills to inspect and makes pretraining decisions less dependent on one generative harness.

Building the 30-trillion-token corpus

The pretraining corpus combined publicly available and licensed human-generated material, including web pages, public GitHub code, books, academic papers, news, multilingual text, and domain-specific sources. Microsoft says every source family was processed through its own pipelines rather than imported as a ready-made open-source training dataset.

The broad pipeline can be summarized as:

This is more than housekeeping. At 30 trillion tokens, small filtering errors become large training signals. Broken HTML extraction, malformed tables, generated boilerplate, duplicated documents, and benchmark leakage can each be repeated billions of times.

Source-specific pipelines

Different sources require different definitions of quality:

• Web HTML requires structural extraction, language and quality classification, repetition filtering, and deduplication.
• PDFs and academic material require OCR and document-quality controls before subject and level classification become reliable.
• Code requires repository-level analysis, license and provenance controls, removal of generated files and boilerplate, deduplication, and decontamination against evaluation data.

For public GitHub data, the report describes a curated set of roughly 94,000 repositories after filtering. The final 30-trillion-token mixture included 16.4 trillion coding tokens. It also contained roughly 300 billion high-quality math tokens, sampled an average of 5.28 times because of their scarcity and value. Web text and PDFs were sampled less than once on average, meaning those pools were not exhausted during the main run.

These figures reveal the real optimization problem. Data mixture design is not simply “collect the highest-quality text.” It is a constrained allocation problem involving quality, diversity, scarcity, repetition, transfer, and behavior at scale.

The architecture: sparse capacity without surrendering the network

MAI-Thinking-1 uses a sparse MoE architecture. For each token, the router selects 8 of 512 experts, allowing a model with roughly one trillion total parameters to activate about 35 billion parameters per token.

Sparse models offer a compelling bargain: increase total capacity without paying dense-model compute for every parameter on every token. The hidden cost is communication. Tokens must be routed to experts distributed across accelerators, producing all-to-all network traffic. A design that looks efficient in FLOPs can become slow in wall-clock time.

Microsoft’s architecture addresses that tension through two related ideas.

Interleaved dense FFNs

Rather than making every feed-forward block sparse, the model alternates sparse MoE blocks with dense feed-forward networks:

The dense layers provide a predictable shared path for broadly useful transformations. That allowed Microsoft to remove the shared experts commonly placed inside MoE layers and reserve routed experts for specialization.

The design is not simply “dense is better” or “sparse is better.” It divides labor:

• Dense FFNs carry universal computation with regular execution.
• Sparse experts provide conditional capacity and specialization.
• Interleaving reduces the need to perform expert routing at every block.

Latent MoE

Latent MoE reduces communication volume by projecting hidden states into a smaller latent dimension before the all-to-all exchange. Experts operate in that compressed space, after which outputs are projected back to the model dimension.

Crucially, routing still uses the uncompressed hidden state. The gating decision therefore retains the full representation even though the expensive network payload is smaller.

Hybrid attention: local most of the time, global when needed

The model alternates local sliding-window attention and global attention at a 5:1 ratio. Local attention uses a 512-token window and Rotary Position Embeddings (RoPE); global attention layers use NoPE, meaning no explicit positional embedding.

The intuition is similar to reading a long technical document. Most token relationships are local: syntax, nearby references, short reasoning steps, and paragraph-level coherence. Periodically, the model needs a global pass to connect distant sections.

Local attention limits the cost of most layers. Global layers restore long-range information flow. The reported experiments found that omitting positional embeddings in global layers improved efficiency while retaining comparable evaluation quality in the tested configurations.

That is an empirical architecture choice, not a universal rule. Its significance lies in the co-design approach: attention pattern, positional encoding, network behavior, and measured model quality were optimized together.

FLOPs are not the same as speed

The report uses efficiency gain (EG) to compare configurations. Conceptually, an EG of 1.3 means the baseline needs 30% more compute to reach the optimized configuration’s performance. But Microsoft distinguishes between FLOPs-based EG and wall-clock EG.

That distinction is critical for MoE systems:

• FLOPs-based efficiency estimates the useful mathematical work required to reach a quality target.
• Wall-clock efficiency includes communication, kernel efficiency, synchronization, memory pressure, routing imbalance, and accelerator idle time.

The architecture ablations make the hardware penalty concrete. Relative to the selected interleaved design, an MoE-at-every-layer variant without a shared expert reached a weighted-average EG of 0.94 by FLOPs but only 0.73 by wall clock. A variant with seven routed experts plus one shared expert looked slightly better theoretically at 1.03 FLOPs EG, yet fell to 0.82 wall-clock EG. In other words, a configuration that appeared competitive—or even marginally better—under arithmetic accounting became substantially worse when communication and execution time were included.

Increasing sparsity from 8-of-256 to 8-of-512 and 8-of-1,024 improved FLOPs-based EG, but the team selected 8-of-512 for the final model. The highest theoretical sparsity was not automatically the best deployable system; the production choice had to balance model quality, training speed, and inference efficiency.

This is one of the report’s most transferable lessons: an architecture is not efficient until the cluster agrees.

From pretraining to reasoning: the RL climb

Pretraining teaches broad predictive competence. It does not fully determine how a model should reason through long tasks, use tools, follow instructions, or balance helpfulness and safety. MAI-Thinking-1 develops those behaviors in a prolonged reinforcement-learning phase.

Microsoft trained three specialist models using the same broad RL framework but different tasks and rewards:

1. STEM reasoning and competitive programming
2. Agentic coding and tool use
3. Helpfulness and safety

The specialists were later consolidated into one model through supervised fine-tuning followed by additional RL.

A simple recipe with important stability controls

The RL system is based on Group Relative Policy Optimization (GRPO), with adjustments intended to control entropy and prevent damaging policy-ratio outliers. The pipeline samples groups of responses, filters tasks that are either too easy or too hard to provide useful relative signal, and trains from verifiable or judged rewards.

For STEM tasks, answers can be checked against ground truth, symbolic tools, judges, or code test suites. Agentic coding environments are executable and test-driven: the model reads files, edits code, runs tests, observes failures, and tries again. Helpfulness and safety use reward signals designed to penalize both unsafe compliance and unnecessary refusal.

The team also used a length curriculum, increasing maximum rollout length from 8K through 16K, 32K, and 64K to 128K. This avoids paying for extremely long traces when the early policy is not yet capable of using them productively.

Reasoning was not expected to emerge without any scaffolding. Before the first self-distillation round, the training prompt instructed the assistant to place its reasoning in <thinking> tags and its final response in <answer> tags. That supplied an output structure, while RL supplied the task-specific pressure that shaped what happened inside it. The report’s learning curves show substantial early gains—especially during roughly the first thousand STEM-climb steps—followed by longer continued improvement rather than a single one-off “aha” event.

Self-distillation as a recovery and transfer mechanism

One of the most useful ideas in the report is self-distillation. During an RL climb, the model generates rollouts. Selected traces are then used to perform supervised fine-tuning on a stable mid-trained checkpoint. The resulting model becomes a new base from which RL can continue.

This mechanism serves several purposes:

• transferring learned behavior into a new chat format;
• recovering progress after numerical instability or a collapsed run;
• carrying capabilities into a newer base checkpoint;
• filtering traces that exhibit reward hacking;
• consolidating specialist teachers into one model.

The report’s experiments add nuance. Around one million reasoning traces were sufficient to match teacher performance in its setting, while much larger datasets risked constraining the policy too tightly. Diversity of prompts mattered more than accumulating many traces for the same prompt. Traces from a range of strong later checkpoints worked better than relying only on very early traces or only the final checkpoint.

Again, “learned, not inherited” needs careful interpretation. MAI-Thinking-1 avoided third-party distillation during pretraining. It absolutely used self-generated traces during post-training. The distinction is between importing another model’s behavior and recycling capabilities discovered within Microsoft’s own training system.

What changed inside the reasoning traces

The report includes qualitative examples of reasoning behavior evolving during the STEM climb. Early policies rely more heavily on shallow pattern matching or brute-force attempts. Stronger checkpoints increasingly test assumptions, inspect boundary conditions, search for counterexamples, and correct an approach before producing the final answer.

In one mathematics example, a weaker checkpoint proposes candidate values and commits too early. A stronger checkpoint derives the candidates, checks the domain constraints, rejects the invalid branch, and revises its path after testing a small case. The interesting shift is therefore not simply toward longer traces. It is toward traces containing more verification, invariant checking, skepticism, and recovery.

These examples are suggestive, not proof that a model “thinks” like a person. Chain-of-thought text is an observable artifact produced by the policy, not a transparent window into every internal computation. The defensible conclusion is narrower: the RL process changed the model’s generated problem-solving behavior in ways that correlated with improved task performance.

The report also shows a prompt format that separates reasoning and final answers with structural tags. That is a training and interaction convention, not evidence of a hidden cognitive module.

Training 8,000 GPUs: utilization is only half the story

MAI-Base-1 used Microsoft’s in-house distributed training stack, referred to in the report as YOLO. The system incorporated more than 20 optimizations across kernels, parallelism, memory management, sparse execution, checkpointing, orchestration, and fault recovery. Examples include grouped GEMM, custom low-precision kernels, context parallelism, DeepSpeed Ulysses, activation offloading, and dropless MoE execution.

The report discusses more than 20 infrastructure and kernel optimizations. These include custom low-precision GEMM and quantization kernels, grouped GEMM, DeepSpeed Ulysses, context parallelism, ZeRO-2 versus ZeRO-3 trade-offs, activation offloading, dropless MoE execution, and work toward deterministic kernels. Determinism matters because silent numerical differences between training and inference can accumulate during long RL runs; the report also acknowledges that deterministic execution can carry an efficiency cost.

The training run sustained model FLOPs utilization above 20%. For a sparse trillion-parameter system, that number must be interpreted in context: MoE communication and routing make peak theoretical compute harder to approach than in a regular dense workload. It is also a reminder that the theoretical capacity of an accelerator fleet is not the same as the compute a sparse training job can use productively.

More revealing is goodput—the share of elapsed cluster time that produced useful forward training progress. Microsoft reports 90.0% goodput at 8,000 GPUs for the final pretraining run. Total overhead was 51 hours, including failures, checkpoint-related pauses, recomputation, startup paths, scheduling delays, and periods of reduced utilization.

A simple distinction helps:

• Throughput: How much work can the system process while it is running normally?
• Goodput: How much of the expensive wall-clock allocation actually advances the training run?

At this scale, reliability becomes an optimization variable. A tiny recurring stall multiplied across thousands of accelerators becomes a major loss. The report’s operational approach was to give every failure mode an owner, a detection signal, a prevention path, and a quantified impact on usable FLOPs.

The published pretraining-loss curve is correspondingly smooth, with only occasional spikes. A smooth curve does not prove that every infrastructure decision was optimal, but alongside the 90% goodput figure it is evidence of a controlled run that recovered from failures without persistent training disruption.

What the report proves—and what it does not

The strongest contribution is not a claim that MAI-Thinking-1 is the best model overall. The evidence supports more specific conclusions:

• Small-scale data-mixture rankings can reverse at larger model and token scales.
• Held-out NLL suites can provide cleaner signals for pretraining decisions than generative benchmarks alone.
• Interleaving dense and sparse blocks can balance shared computation, expert specialization, and routing cost.
• Compressing expert communication through a latent space can reduce MoE network payloads while preserving full-state routing.
• Self-distillation can preserve and transfer gains across long RL climbs and unstable checkpoints.
• At frontier scale, wall-clock efficiency and goodput are first-class model-development metrics.

Several limitations remain:

• Most evidence comes from Microsoft’s own model family, infrastructure, datasets, and internal evaluations.
• Ablation results may not transfer unchanged to smaller clusters or different model families.
• The report describes a closed-weights model; independent reproduction is therefore limited.
• Benchmark scores remain sensitive to harnesses, prompting, contamination controls, and model versions.
• Qualitative reasoning traces should not be interpreted as direct evidence about internal cognition.

These limitations do not make the findings unhelpful. They define the boundary between an engineering case study and a universally established law.

The deeper lesson: optimize the learning system, not just the checkpoint

MAI-Thinking-1’s most important idea is organizational as much as technical. Data pipelines, model architecture, evaluation, RL, kernels, networking, and cluster operations cannot be optimized independently. A data mixture that wins at small scale may lose at large scale. An MoE configuration that wins on FLOPs may lose on the network. An RL run that reaches a strong checkpoint may still be unusable if progress cannot survive an instability.

The Hill-Climbing Machine turns those dependencies into a repeatable loop:

1. Form a measurable hypothesis.
2. Test it across enough scale to expose reversals.
3. Evaluate both model quality and hardware behavior.
4. Preserve the winning capability in a stable checkpoint.
5. Feed the result into the next climb.

That is a less glamorous story than a single architectural breakthrough. It is also more credible. Frontier AI progress increasingly depends on reducing the number of ways a promising idea can fail between a small experiment and a months-long run across thousands of accelerators.

MAI-Thinking-1 is one checkpoint. The Hill-Climbing Machine is the more consequential artifact: a blueprint for making improvement itself more observable, recoverable, and repeatable.

References

• Microsoft AI Team, MAI-Thinking-1: Building a Hill-Climbing Machine, June 2026.
• Microsoft AI, Introducing MAI-Thinking-1, June 2, 2026; updated June 8, 2026.