6 min read · 15 sources

OpenAI’s first inference chip, Jalapeño, is no longer a rumour. The company posted early benchmarks this week and said it plans to drop the accelerator into its own fleet before the end of the year. OpenAI’s design team built it around the same low-latency agent workloads that already dominate its production traffic, and the company claims AI tools helped design its circuits and program its kernels. If the numbers hold up, this is the first credible challenge to Nvidia from a frontier lab that controls its own model stack top to bottom.

Perplexity chose a different path to the same destination. Its new Portable Computer product is the company’s agentic Computer platform, repackaged to run entirely on hardware users already own. No token credits, no per-call billing, model and user data stay on device, and every task starts local by default. The system only asks for permission before handing a single step to a more capable cloud model. It is live today for Pro, Max, Enterprise Pro, and Enterprise Max subscribers on Linux, Windows ships in September, and the floor is an Nvidia RTX card with 24GB of VRAM.

On GPT-OSS 120B, Jalapeño delivered higher peak throughput per kilowatt and lower token latency than the commercial systems OpenAI tested it against

Claude's tokenizer just got a lot smaller

Source: ianbarber.blog ↗

Tokenizers are the unglamorous foundation of every LLM, and Anthropic appears to have made an unfashionable bet. Researcher Sander Land estimates the current Claude tokenizer holds only about 15,000 entries, down from roughly 50k in Claude 3 and orders of magnitude below Qwen 3.8’s 250k-vocab setup.

The proposed mechanism is the softmax layer at the language-model head. When vocabulary V dwarfs hidden dimension D, the gradient that updates logits has rank at most 2D, so each batch can only move a random D-sized slice of the vocabulary. The referenced paper, Lost in Backpropagation: The LM Head is a Gradient Bottleneck, argues that shrinking V can actually speed up training. Engineers tuning their own tokenizer and LM head should read it before assuming the bigger-is-better consensus is right.

The compute map is consolidating fast

Source: dwarkesh.com ↗

Dylan Patel sat down with Dwarkesh Patel to spell out the macro view. The SemiAnalysis founder argues OpenAI and Anthropic will soon control most usable global FLOPs because they monetize compute more efficiently than anyone else, including the hyperscalers. By 2030 the pair projects more than $10 trillion in cumulative AI capex, with non-trivial odds that the debt funding it lifts real interest rates, breaks non-AI-exposed sovereigns, and pulls non-AI equities down with it.

The interview is 48 minutes long and worth the time for anyone building a multi-year roadmap. The single biggest argument: compute is becoming a centralising force, and the window to enter at the frontier is narrower than the public market currently prices.

Application moats when the models flood in

Source: x.com ↗

Counterpoint to the doom side of that same trend: a long post titled Moats in the age of floods argues that abundant frontier intelligence will not erase the application layer. The author’s analogy is electricity, which was wired into factories in the 1880s but only showed up in productivity statistics in the 1920s. The lag between adoption and measurable transformation is, in this framing, the largest open arbitrage, and it is closing.

For engineers and founders, the operational read is that work which converts model output into reliable real-world outcomes is defensible. Coordination, workflow data, customer-side transformation, and higher-level abstractions are listed as the parts that hold their value.

Omnimodal world models, open weights

Source: github.com ↗

JD.com’s JoyAI-Echo repo bundles two projects under one roof. Echo-LongVideo generates up to five minutes of multi-shot audio-visual content with a paired memory bank for synchronisation. Echo-WM is an interactive omnimodal world model on LTX-2.3, available as a bidirectional Base variant for ~10s clips and a chunk-causal “Flash Preview” with KV-cache rollout and 4-step inference. A roadmap note promises LTX-2.5 migration and SageAttention-based cost cuts.

Both ship with separate Python environments and checkpoints and are licensed for non-commercial research only. The target hardware is consumer GPUs, which puts the bar somewhere an engineer with a workstation and a weekend can clear.

Short-lived credentials for agents

Source: vercel.com ↗

Vercel Connect hit general availability this week. It replaces long-lived provider tokens with runtime-issued credentials that agents request at call time via the @vercel/connect SDK. Each deployment carries an OIDC identity used to mint scoped, auto-refreshing tokens for 100-plus connectors (Slack, GitHub, Snowflake, Shopify, custom OAuth), so no provider secret ever has to live in the app or repo.

The pattern matters because it deletes a common leak class entirely. Every step in an agent run requests only the access it actually needs, and the token expires on its own. Anyone running multi-tenant agent systems should treat this as a baseline rather than a nice-to-have.

IBM ships Granite 4.2

Source: huggingface.co ↗

Granite 4.2 is IBM’s first family of dense, decoder-only reasoning LLMs trained from scratch, in 3B, 8B, and 30B sizes. All three were trained on ~15T tokens under a five-phase pipeline that extends context to 512K, then runs CoT, instruction, and agentic SFT, finishing with multi-stage RL. The 8B and 30B variants include agentic RL inside real sandboxed environments.

Architecturally they use GQA (40 attention heads, 8 KV heads) with rotary embeddings, expose a thinking/non-thinking switch plus a low-effort mode, and handle native OpenAI-format tool calling. They serve from an OpenAI-compatible endpoint, work with vLLM, and are supported in SGLang. Licence is Apache 2.0 across the family.

Anthropic merges memory by default

Source: thenextweb.com ↗

Claude and Claude Cowork now share one memory system, enabled by default for Free, Pro, and Max plans across web, desktop, and mobile. The technical shift is that memory capture no longer waits for end-of-conversation summarisation, topics are added in real time during the active session, and the user checkpoint at session end is gone.

Two toggles govern behaviour: one for memory generation and one for sensitive categories (health, race, religion, politics, and so on, off by default). Every stored item is surfaced as an editable file under “Topics,” so the data model and its consent surface are auditable. Certain hard-excluded items like social security numbers stay excluded regardless.

Local silicon from Apple, a departure at OpenAI

Source: apple.com ↗

Apple launched the M6 in Mac mini and the M5 Ultra in Mac Studio. The M6 is Apple’s first 2nm SoC: 12-core CPU, 12-core GPU with Neural Accelerators, dual 16-core Neural Engine, and up to 170GB/s of unified memory bandwidth. The M5 Ultra uses a quad-die UltraFusion design with up to 36 CPU cores, 80 GPU cores, and 1.2TB/s of unified memory bandwidth, which is 50% more than M3 Ultra. Both are explicitly positioned to run massive AI models locally.

OpenAI lost the executive running its data-center build-out. Chris Malone departed last week, the same week OpenAI was publishing Jalapeño results and planning a year-end install of its own silicon. Treat the timing as suggestive rather than conclusive; the company has not elaborated.

A new agent cloud, and a web index for agents

Source: appliedcompute.com ↗

Applied Compute’s AC2 is a platform for training, post-training, and serving open models on shared infrastructure. It ships with a research console for inspecting rollouts, an RL control plane that scales with training compute and context length, and “Ari,” an applied research agent that monitors runs, diagnoses failure modes, and feeds findings back into training data. Bring-your-own training harnesses are supported. Microsoft, NVIDIA, Cognition, Mercor, DoorDash, and Harvey are listed as current customers.

Keenable came out of stealth with a $26M seed led by Accel and Conviction Partners, claiming a 100B-plus document index and a query API built for AI agents rather than humans. The company says unnamed AI labs and inference providers already use it for training and runtime, and that voice-AI firm Gradium is signed up for live retrieval. The founders cite Google and Microsoft pulling back their public search APIs as the gap they are filling.