6 min read · 15 sources

Meta dropped open weights for Muse Spark Open, an 816-billion-parameter Mixture-of-Experts model that activates 42 billion parameters per token. The release carries a 1-million-token context window, native multimodal support, and targeting for autonomous coding workflows. Instead of locking the weights behind an enterprise cluster requirement, Meta tuned the memory layout so the model runs across two networked Apple M5 Ultra Mac Studios via MLX and custom Helion kernels.

For production teams, this means a frontier-class agentic model can run on workstation hardware without a cloud API dependency. If you run dedicated GPU clusters, the checkpoints plug directly into standard serving stacks including vLLM, SGLang, and TensorRT-LLM.

At the same time, enterprise pre-training infrastructure is setting new scale baselines. Microsoft published the architecture details behind MAI-Base-2, a 2.8-trillion-parameter MoE foundation model, while production telemetry from Vercel shows open-weight traffic crossing an unprecedented inflection point.

Open‑weight models now handle more than seven‑in‑ten gateway tokens yet capture less than a quarter of the spend.

Microsoft Trains 2.8T Parameters Across 16,000 GB200s

Source: microsoft.ai ↗

Microsoft outlined the pre-training run for MAI-Base-2, an enterprise MoE activating 80 billion parameters out of 2.8 trillion total. The run took 38 days across a cluster of 16,000 NVIDIA GB200 GPUs, maintaining an 89% cluster goodput and 21% Model FLOPs Utilization (MFU). Microsoft claims 4x to 12x better FLOPs efficiency compared to DeepSeek-V4-Pro-Base on held-out evals.

The stack avoids synthetic distillation in favor of organic data pipelines, backed by infra-model co-design techniques like Muon optimizers and aggressive activation gradient clipping. The resulting weights will be exposed downstream to enterprise clients via Azure Foundry, serving as the foundational layer for Microsoft’s proprietary business lines.

Vercel Gateway: Open Weights Reach 71.5% Token Share

Source: vercel.com ↗

Vercel published its October 2026 AI Gateway Production Index, revealing that open-weight models now drive 71.5% of total token volume. Open-weight compute spend has quadrupled since June, now capturing 23% of total gateway dollars. The report highlights massive churn in backend routing: nearly one in three engineering teams switched their primary model provider during September alone.

Cost distribution remains extremely lopsided across architectures. DeepSeek-v4.1-Flash handled almost half of all token volume routed through the gateway, yet captured only 2.6% of customer spend due to aggressive token pricing. Conversely, high-capacity models like Kimi K3 accounted for two-thirds of the open-weight spend increase, demonstrating that teams route commodity queries to ultra-cheap pipelines while preserving margin for complex agentic workloads.

Liquid AI Targets Classification With d1-ultra

Source: liquid.ai ↗

Liquid AI released d1-ultra, a multimodal decision model priced at $0.042 per million input tokens. Unlike generative text engines shoehorned into verification tasks, d1-ultra is built specifically for scoring, binary validation, and multi-choice classification. It accepts mixed image and text payloads and outputs calibrated probability scores without generating conversational overhead.

The model outperformed Jev 1.14 across 112 internal benchmarks and claimed the top spot on the Artificial Analysis Decision Index by 8 points. For production architectures running optical defect inspection, automated content moderation, or document intake, the API provides deterministic scoring at a fraction of the latency and cost of frontier LLMs. Liquid AI confirmed integrations for Vercel and OpenRouter will ship shortly.

Interpretability Research Argues Superposition Is Overblown

Source: already-clean.goodfire.com ↗

A research paper from Goodfire argues that neural networks are already clean, challenging the baseline assumption that polysemanticity is primarily caused by superposition. Historically, researchers assumed that models packed unrelated concepts into individual neurons because the parameter space lacked sufficient dimensions, requiring complex dictionary learning to untangle internal states.

Goodfire’s analysis shows that apparent arbitrary packing is largely recursive generalization. If networks organize concepts naturally into structured recursive abstractions rather than messy overlays, engineers can analyze internal representations directly without complex post-hoc decomposition layers. The finding simplifies the outlook for mechanistic interpretability, suggesting safety audits can inspect model activations without building massive projection dictionaries first.

Policy Gradients Act as Parameter Covariances

Source: richardli.xyz ↗

A mathematical deep dive into expected policy gradients demonstrates why reinforcement learning, supervised fine-tuning, and on-policy distillation alter only 5% to 30% of a model’s parameters during post-training. The paper formalizes that the expected policy gradient acts as a covariance, while the diagonal Fisher information matrix operates as the variance of logit sensitivities.

Because the score maintains a zero mean under the sampling distribution, parameter updates are strictly capped by the Fisher matrix regardless of how large the scalar reward is. If a parameter shift moves all candidate tokens equally, the update evaluates to zero. This framework gives ML engineers a way to predict parameter sparsity and calculate weight updates mathematically without running expensive on-policy rollouts.

Long-Horizon Agent RL Faces Trajectory Drift

Source: ianbarber.blog ↗

Post-training setups for multi-step agentic workflows are running into hardware and distribution limits, detailed in an analysis of multi-turn reinforcement learning. While single-turn RL relies on simple prompt-completion scoring, agent rollouts run for hours - such as Alibaba’s 4-hour sessions for its 2.4T model - involving arbitrary subagents, external bash commands, and dynamic context compaction.

These extended trajectories degrade GPU utilization and trigger policy drift when execution sandboxes preempt or delay rollouts. Infrastructure teams are responding with two patterns: running persistent sandbox proxies that shuffle KV caches across GPU clusters dynamically, or embedding the orchestration harness inside the container sandbox as implemented in DeepSeek v4.1.

NVIDIA Nemotron 3.5 Super VL Runs 12B Active Parameters on Single GPUs

Source: developer.nvidia.com ↗

NVIDIA launched Nemotron 3.5 Super VL, an open 120-billion-parameter MoE model that activates 12 billion parameters per token. Tuned for multi-step visual reasoning, document parsing, and code generation, the model scored 26.5 on the Artificial Analysis Intelligence Index while cutting output token counts compared to previous iterations.

The architecture runs on a single high-memory GPU and provides up to 5x higher inference throughput than comparable dense alternatives. Alongside Super VL, NVIDIA shipped a smaller Nemotron 3.5 Bolt variant (74B total, 9B active) and released native NVFP4 quantized weights for both models to simplify deployment via NVIDIA NeMo.

Anthropic Opens Internal Systems to Third-Party Auditors

Source: anthropic.com ↗

Anthropic granted independent safety evaluator METR internal, employee-grade access to inspect model internals, training runs, and alignment safeguards. Conducted alongside SecureBio and Accenture, the initiative complies with commitments made under the White House Accord on Super Intelligence.

The move mimics the inspection models used in central bank oversight. Rather than relying on black-box external API testing, third-party evaluators gain continuous, white-box auditing visibility to probe for alignment faking, dangerous capabilities, and autonomous replication risks prior to deployment.

Production Tooling and Quick Links

Source: threadreaderapp.com ↗

• OpenAI rolled out GPT-Live-1.1 across ChatGPT, Codex, and dots, adding lower-latency conversational voices for French, German, Hindi, Mexican Spanish, Korean, and Japanese with reduced hesitation pauses.
• Krea launched Krea 3, an image editing system that handles targeted modifications in under eight seconds at $0.06 per run, supporting composition across eight input image references.
• Black Forest Labs published FLUX 3 Image Klein, a 5.6-billion-parameter image generation and pixel-level editing model that runs inside 8GB of VRAM and ships in FP8, INT8, and NVFP4 formats.
• Perplexity open-sourced SPACE, releasing the isolation sandbox and container runtime used to execute dynamic tool calls inside its Computer agent platform.
• Cheng Lou introduced Handwired, an experimental compiler that converts plain JavaScript logic into differentiable neural computational graphs that execute client-side without GPUs or training steps.
• A retrospective from Every details how Codex evolved from an abandoned cloud platform into an autonomous editor agent, triggered by early competition from Anthropic’s Claude Code.