GitHub's 7-Hour Outage Exposed a Capacity Crisis; Plus AWS Turns 20
8 min read · 12 sources
- GitHub's August 17 outage lasted 7 hours 47 minutes after record traffic overwhelmed a Central US data center component that failed to scale, with a Copilot retry loop complicating recovery.
- GitHub has since added over 3 million CPU cores and 120 petabytes of high-speed storage, while Azure now serves roughly 58% of platform load, up from 12% in May.
- AWS EC2 turned 20 on August 25, 2026, growing from a single m1.small instance type to over 1,200 types across 39 regions, with its own custom silicon (Graviton, Inferentia, Trainium).
- DigitalOcean patched two guest-to-host hypervisor vulnerabilities — Januscape (CVE-2026-53359) and AMD Safe RET — across its entire fleet within eight and five days respectively, with zero confirmed customer impact.
- Atlassian published details on a multi-signal RCA system that correlates metrics, traces, and logs on a shared timeline and traverses OpenTelemetry-derived dependency graphs to rank fault hypotheses.
The single biggest engineering story this week is GitHub’s postmortem. On August 17, github.com went dark for 7 hours and 47 minutes. Authentication failed. GitHub Actions seized. APIs returned errors. Copilot choked on its own retry logic. The root cause, now fully documented: traffic hit a new peak, and a critical infrastructure component in the Central US data center simply did not scale with it. The recovery was made worse by a client-side retry loop in Copilot that hammered the platform as it tried to come back up.
The response has been substantial. GitHub has added over 3 million CPU cores and 120 petabytes of high-speed storage. More structurally, Azure now serves roughly 58% of platform load — up from 12% in May. That is a migration that accelerated under outage pressure, not one that was leisurely planned. GitHub is also tightening retry budgets and timeouts across its services and reducing shared dependencies that turned single failures into cascading ones. If you run any GitHub Actions workflows in automation pipelines, this postmortem is worth reading specifically for the retry behavior section.
Azure now carries 58% of GitHub’s platform load, up from 12% in May — a migration that accelerated under outage pressure.
Happy Birthday, Amazon EC2
Twenty years ago, Jeff Barr posted a blog entry introducing a single instance type — m1.small — in one region, billed by the hour. That was Amazon EC2 on August 25, 2006. Today it offers over 1,200 instance types across 39 Regions.
The trajectory matters for how infrastructure thinking has evolved. EBS launched in 2008, VPC in 2009, Nitro System in 2017, Graviton processors in 2018, Mac instances in 2020. AWS has since built its own AI silicon — Inferentia for inference and Trainium for training. For engineers running cost-sensitive workloads, Graviton’s performance-per-dollar improvement over comparable x86 instances has been material. For AI workloads, Trainium and Inferentia represent a bet that AWS can undercut its own GPU offerings. Watch whether those chips gain adoption in 2027.
How DigitalOcean Patched Two Hypervisor Flaws in One Month
Two guest-to-host escape vulnerabilities landed in a cloud near you this summer, and DigitalOcean’s Kernel Engineering team documented exactly how they handled both. The first, Januscape (CVE-2026-53359), was a KVM nested virtualization flaw publicly disclosed on July 6. DigitalOcean built a livepatch within 45 minutes of the disclosure page going up and achieved full fleet coverage within 48 hours. Zero confirmed customer impact.
The second, an AMD Safe RET vulnerability, affected roughly 1,600 hypervisors across twelve regions and could not be livepatched — it required kernel updates and a reboot on every affected machine. Harder than the first, with a tighter deadline: the entire fleet was patched by August 5, one day before AMD’s public bulletin. Guest-to-host escape is the most serious class of vulnerability for any cloud provider; the delta between those two responses — 45 minutes to build a patch, five days to push it everywhere — is a window into what “operational discipline” actually looks like at fleet scale.
Solving the 1+N Query Problem at the Language Level
The N+1 query problem is a class of database performance bug so common it has its own Wikipedia page. You fetch a list of entities, then your ORM fires one query per entity to hydrate a related field. Multiply that by a large page load and your database falls over.
Acadia takes a structural approach: its query language has no general-purpose loops or recursion, so developers literally cannot write the code that causes N+1. Instead, relationships are expressed through set operations like intersect that compile into SQL. Queries are also guaranteed to terminate in polynomial time relative to data size — a correctness property most ORMs do not offer. The language is inspired by Datalog. If you have been burned by an ORM quietly generating hundreds of queries behind a clean-looking interface, this is worth understanding even if you never use Acadia.
On-Call Is Broken. Nobody Has Fixed It.
One long read this week argues that on-call as practiced is fundamentally an admission that software cannot run without human oversight, and that every alert threshold, escalation policy, and runbook is calibrated around the scarcity of human attention — not the reliability of the system. AI has transformed how software is built. It has not transformed how it is operated. Engineers still rotate, still get paged, still manually triage incidents the way they did before LLMs existed.
The proposed alternative is not new: agents that continuously detect, investigate, fix, and prevent incidents, escalating only genuinely novel problems. What is interesting is the framing — alert fatigue is not a configuration problem but a rationing mechanism, and fixing it requires rethinking who a human engineer actually is in the loop. Worth reading for the diagnosis even if the prescription is aspirational.
Root Cause Analysis at Atlassian Scale
When you have hundreds of microservices, a single degraded user experience can be caused by any one of dozens of components in any order. Atlassian has published its system for automating root cause analysis across that sprawl.
The approach pulls anomaly signals from metrics, distributed traces, and logs, aligns them on a shared timeline, and traverses a service dependency graph built from OpenTelemetry span data to generate ranked hypotheses about where a fault originated. This is not a chatbot that reads your logs — it is a correlation engine that weights multiple independent signal sources against a topology map. The dependency graph is the key piece: without it, you get a list of symptoms. With it, you get propagation chains and root candidates.
Apache Maka and Hister: Local-First Tools Worth Watching
Two new open-source tools landed this week for different sides of the local-first AI stack.
Apache Maka is an incubating project at the Apache Software Foundation providing a local-first agent workspace. Sessions, settings, and run records stay on your machine by default. The system records model messages, tool calls, and execution facts as durable evidence rather than discarding omitted context — useful for crash recovery and for trimming context windows without losing traceability. Entry points include an Electron desktop app with streaming sessions, a TUI/CLI, and an Eval interface for reproducible benchmark experiments. Built-in tools are intentionally minimal: Read, Write, Edit, Bash, Glob, Grep.
Hister is a private search engine that indexes visited web pages and local files. It runs locally with no telemetry by default, supports semantic search through configurable embeddings endpoints, and exposes itself via a web interface, TUI, CLI, and MCP client for AI assistants. If you have ever wanted personal search without sending your browsing history to a third party, this is one to evaluate — though the semantic search feature requires you to bring your own embeddings endpoint, which adds setup cost.
Building an Incident Bot That Uses Your Existing Approval Flows
A proof-of-concept called Octopus Healer demonstrates a pattern for AI-assisted remediation that does not bypass your existing infrastructure. The bot listens for Kubernetes Prometheus alerts, passes pod context to an AI model, and maps the model’s output to predefined Octopus Deploy runbooks. An operator reviews and approves the suggested remediation in Slack before anything executes.
The key design decision is routing execution through Octopus Deploy rather than running raw kubectl commands. This preserves existing approval flows, permission boundaries, and audit trails — the things that make compliance and security teams willing to let automated remediation run in production at all.
AI Coding Tools and the Expertise They Prevent
A contrarian take this week argues that AI coding tools create a paradox: they require expertise to use effectively while simultaneously bypassing the friction that builds that expertise. Developers entering the field during the LLM era lack the longevity of experience that veterans have, yet are pressured to use agents that demand expert-level judgment to evaluate. The “skilled orchestrator paradox” — the skills required to manage AI agents are the same ones that get atrophied through their use.
The counterproposal is to treat AI as a tutor or Socratic partner rather than a code generator: preserve the planning, debugging, and trial-and-error that turns experience into expertise. Whether you agree or not, the framing of the problem is more precise than most discourse on this topic.
Databricks Now Speaks SSH to Your Local IDE
Databricks shipped a workflow improvement that connects VS Code, Cursor, or a terminal directly to Databricks compute via SSH tunnel. Engineers can interactively run and debug Python and SQL on real Databricks clusters while keeping IDE ergonomics, browse Unity Catalog, and edit workspace files without switching to the web interface.
The connection is a single Databricks CLI command or IDE extension action. The practical impact: local AI coding agents like Cursor, Copilot, and Claude Code can now work against Databricks compute without you manually copying code back and forth. If you have been waiting to integrate AI-assisted development into a Databricks workflow, the plumbing is now simpler.
Jenkins Calls Contributors to a Virtual Hackathon
Jenkins announced a Contributor Summit for September 11, 2026, running 08:30 to 19:30 CEST entirely online. Two structured blocks cover APAC/EMEA and EMEA/Americas time zones. Squads of four pick GitHub Issues to own, with opening plenary, live coding sessions, and a showcase demo. If you have been meaning to contribute to Jenkins or want to steer its roadmap, the event is a structured on-ramp.
You May Also Like
Cache-Aware Routing, Glue 6.0, and the AI On-Call Risk Nobody Is Testing
DigitalOcean's Inference Router now factors prompt-cache value into model selection, citing 90%+ cache hit rates and Anthropic's use of prompt caching in Claude …
Kubeflow graduates, Bun 1.4 lands, and a Cloudflare Spectre bug that bit 12 bits a second
CNCF has graduated Kubeflow, the Kubernetes-native stack for AI training, fine-tuning, and inference, after it crossed nearly 260 million PyPI downloads. Bun …
Nvidia's Groq 3 LPX enters production, OpenAI claws back share from Anthropic
Nvidia's Groq 3 LPX inference accelerator hit full production at Hot Chips, with Nebius signed up as the first customer. Ramp data from 70,000+ US organisations …




