BriefTechNews

Anthropic Researcher Puts AI Extinction Odds at >10% - and Nobody Has a Plan

8 min read · 11 sources

TL;DR
  • Anthropic's Evan Hubinger said there is a >10% chance AI kills all humans within the next decade and that no alignment plan for superintelligence exists yet.
  • OpenAI non-profit board member Paul Christiano said the industry, including OpenAI, is not on track to reduce catastrophic loss-of-control risk to an acceptable level.
  • Oracle's cloud infrastructure revenue grew 121% year over year in Q1 2027 while it kept its capex forecast unchanged from June guidance.
  • Microsoft unveiled an AI-powered converter designed to migrate Salesforce and ERP users onto its own platforms.
  • Latham & Watkins bought Nvidia GPU servers and runs customized open-weight models on infrastructure it controls.

Anthropic’s alignment lead says there is a greater than 10% chance AI kills all humans within a decade - and admits no plan exists to stop it.

Anthropic's alignment lead says AI has a >10% chance of killing everyone

Evan Hubinger, Anthropic’s Alignment Science Lead, posted on X that he personally believes there is a greater than 10% chance AI “could kill all humans” within the next decade. He also admitted the company “does not yet have a plan to solve alignment for superintelligence” and is “not clearly on track to” developing one. The post came a day after fellow Anthropic researcher Jacob Coxon resigned over the pace of development.

This is not a fringe take from a random Twitter account. Hubinger is the person whose job is literally to make sure Anthropic’s models don’t do something catastrophic. If the alignment lead at one of the three frontier labs says the odds are double-digit and the plan doesn’t exist, that is a statement about the state of the industry, not a single company. For engineers running AI systems in production, the practical takeaway is grim: the people building the most capable models are publicly saying they don’t know how to keep them safe.

OpenAI board member: industry not on track to prevent catastrophe

Paul Christiano, a member of OpenAI’s non-profit board and a US government technology adviser, told The Guardian that “there is a meaningful risk that rapid acceleration in AI capabilities leads to catastrophic and irreversible loss of control in the very near term.” He added that he does not think the AI industry in general, including OpenAI, is “currently on track to reduce this risk to an acceptable level.”

Christiano is not a random commentator. He previously led the alignment team at OpenAI and has deep credibility on this exact question. When two senior people at two different frontier labs say the same thing in the same week, the “AI safety is a solved problem” position becomes hard to defend. The interesting operational detail is that both are pointing at the same failure mode: capability gains are outrunning the ability to steer the systems, and nobody has demonstrated a mechanism to close that gap.

Oracle's cloud infrastructure revenue jumps 121%

Oracle reported Q1 2027 earnings with cloud infrastructure revenue up 121% year over year. The company maintained its capex forecast, unchanged from the guidance it issued in June. That puts Oracle in a notable minority: most major cloud companies have been raising AI spending forecasts as they race to build data centers, with Microsoft being the other notable exception that kept its forecast flat in July and saw its stock jump after earnings.

The “keeping capex flat while revenue grows 121%” combo is the number that matters. It suggests Oracle is getting more utilization out of existing capacity rather than throwing new money at the problem. For anyone watching the AI infrastructure buildout, this is a useful counterpoint to the narrative that everyone must spend endlessly on new GPUs. The question is whether Oracle can sustain that efficiency as demand keeps climbing, or whether the flat forecast is just a one-quarter artifact.

Microsoft goes after Salesforce and ERP users with AI-powered converter

Microsoft launched an AI-powered converter designed to migrate Salesforce and ERP users onto its own platforms. The tool targets the massive installed base of legacy CRM and ERP deployments, using AI to handle the messy work of data mapping and schema conversion that has historically made these migrations slow and expensive. It is a direct attack on Salesforce’s core territory.

The engineering angle here is the conversion problem itself. CRM and ERP migrations are brutal because the data models are deeply customized per customer, and the business logic lives in workflows and triggers, not just tables. If Microsoft’s AI converter can actually handle that complexity at scale, it removes the biggest friction point that keeps enterprises locked into incumbent platforms. If it only handles the easy 80%, the remaining 20% will still eat the budget and the timeline. The Register’s coverage notes the tool is aimed squarely at poaching users, so expect Salesforce to respond with its own migration pitch shortly.

The "left" in shift-left moved

A Resilient Cyber essay argues that the economics of finding vulnerabilities have changed so dramatically that the traditional shift-left approach is now misaligned. AI agents are surfacing flaws in mature codebases at a rate the institutional ecosystem of disclosure, triage, and patching was never built to absorb. The author advocates eliminating entire vulnerability classes rather than grinding through tickets one at a time, and rethinking human-in-the-loop oversight when agents can both write and exploit code.

This is the AppSec version of the AI capacity problem: the discovery side has scaled with agents, but the remediation side still runs on human review cycles. The piece specifically calls out developer endpoints with broad privileges, where AppSec has limited visibility. The practical recommendation is to reduce attack surface by removing vulnerability classes at the framework level, rather than trying to patch faster. For teams that still do security reviews by hand, this is a warning that the backlog is about to get much worse.

Google adds five agentic capabilities to Workspace

Google is rolling out five new agentic Gemini capabilities across Workspace that complete cross-app tasks in the background. Users can now prompt Gemini from Gmail, Drive, Docs, Slides, or Chat to handle complex multi-app workflows. The features include generating Slides from Chat content and building Sheets from data in Docs. They are rolling out now to Business, Enterprise, and Google AI Pro tiers.

The shift here is from “chatbot that answers questions” to “agent that does the work.” For IT departments, the operational question is permissions: if Gemini can act across your Workspace estate, what can it do with the credentials it holds, and who audits those actions? Google’s blog post is light on the governance details, so expect the security team to be the one reading the fine print before this gets enabled broadly.

Accenture and Google Cloud launch Gemini Enterprise group

Accenture and Google Cloud formed the Accenture Gemini Enterprise Business Group, bringing together Gemini Enterprise-certified professionals and co-developed AI solutions. The group will establish a 1,000-person forward deployed engineer workforce to help enterprises move from AI pilots to production. Accenture cites results including 11% higher customer sentiment and 37% lower handle time for clients using Gemini.

The 1,000 FDEs number is the headline. That is a serious commitment of bodies to a single partnership, and it signals that Google Cloud is betting heavily on the services-led route to enterprise AI adoption. For engineers, the takeaway is that Google is not just selling models; it is selling the implementation layer. The risk is that a 1,000-person services arm creates a dependency on Accenture for anything Gemini-related, which could complicate multi-cloud strategies.

Big Law starts building its own AI infrastructure

Latham & Watkins has bought Nvidia GPU servers and is running customized open-weight models on infrastructure it controls. The firm is using the setup to keep sensitive client data, costs, and model choice under its own control rather than relying on public cloud AI APIs. This is one of the first major law firms to go down the private AI stack route.

The engineering detail that makes this credible is the migration work: Latham reportedly converted legacy Fortran 77 reservoir-simulator code to C++ as part of the effort, using a numerical parity harness to verify correctness and over a hundred agents to document the codebase. Structured workflows with human review gates beat full autonomy, and the first sprint covered core functionality of a 300,000-line system. That is a real-world data point for anyone arguing about whether agents can handle legacy modernization: they can, but only with human checkpoints.

Mistral's legacy code modernization playbook

Mistral published a detailed account of migrating a legacy Fortran 77 system to modern C++, covering the lessons learned. The key takeaways: prioritize numerical agreement between old and new code, document the codebase before starting the migration, and balance agent autonomy with human review. The post also notes that sticking with languages that have no active developer ecosystem means missing out on building on others’ work.

This is a useful counterweight to the “just let the AI rewrite it” crowd. Mistral’s team built a parity harness first, which is the boring but essential step that makes the migration verifiable. The finding that structured workflows with human review gates beat full autonomy matches what Latham & Watkins reported. If you are planning a legacy migration with AI agents, this is the playbook to crib from.

Keycloak vs Ory: two different bets on identity

A Cerbos comparison of Keycloak and Ory argues the two are not the same kind of thing, despite appearing on every self-hosted identity shortlist. Keycloak is a single Java server that does everything, now running on Quarkus and a CNCF project. Ory is a set of separate services you assemble into the shape you need. Both are Apache 2.0 and both avoid per-seat licensing, but the choice comes down to how much of your identity stack you want to own.

The comparison spends its second half on the authorization gap that most writeups skip. Keycloak bundles authorization into the server; Ory leaves it to you to assemble. For teams that want tight coupling between authn and authz, Keycloak is simpler. For teams that want modular, headless services and are willing to own the integration work, Ory gives more control. The answer depends on whether you want a product or a toolkit.

Nvidia and Palantir bring sovereign AI to supply chains

Nvidia and Palantir announced a collaboration to bring sovereign AI to critical supply chains, starting with Nvidia’s own operations. The stack combines Palantir’s Foundry and AIP with Nvidia Nemotron open models, grounded in the Palantir Ontology. It runs on cloud or on-premises infrastructure, and Nvidia is deploying it first to codify operational intelligence and accelerate the path from wafer to first token.

The “starting with NVIDIA’s own supply chain” detail is the one that gives this credibility. Nvidia is eating its own dog food on the most complex supply chain in the semiconductor industry. The sovereign angle matters for defense and critical infrastructure customers who cannot send data to public clouds. The open-model choice (Nemotron) is notable because it keeps model weights under customer control, which is the whole point of sovereign AI.

Source: nvidianews.nvidia.com ↗

Get the brief

Liked this one? The rest of today's stack — AI, crypto, fintech, infra — lands in your inbox tomorrow morning. Five minutes, no hype.

About Me Author

My name is

BriefTechNews

A daily digest of what actually moved in AI, tech, crypto and fintech, assembled and written with AI, and reviewed before it publishes. Read More
Tags

You May Also Like