FOMO Buster #030: "The watchers get watched: METR red-teams OpenAI's agent monitor"

A newsletter that makes FOMO obsolete.

if i fear missing out on reading this newsletter, did i win or lose

Curated by Maurice.

01 · The One Thing

METR red-teamed OpenAI’s internal agent monitor and found the blind spots

For three weeks this summer (July 21 to August 11), one researcher from METR and one from Redwood Research worked inside OpenAI with deep access: the internal codebase, the monitor’s prompts and configuration, internal documentation, and technical interviews with about a dozen people across the monitoring, safety-systems, and security teams. They worked on OpenAI-issued devices that never left the premises, accompanied on site. The output was a 61-page internal report; OpenAI could cut anything it judged sensitive before the public post.

The monitor watches OpenAI’s own coding agents (mainly Codex), reading what each agent did and flagging scheming, sabotage, deception, or attempts to get around restrictions for human review. What the red team found, describing the system as it existed on or before July 21:

• Coverage math: 75% to 94% of internal agent traffic from models at least as strong as GPT-5.5 was eligible for monitoring; 99.2% for the internal-Codex subset. The estimates carry “substantial uncertainty,” and eligible does not mean watched: traffic can still slip through on overload, errors, or bugs.
• The compaction gap: roughly 10% of eligible internal Codex traffic went unmonitored from May through July because of how long sessions were handled. OpenAI had spotted this before the exercise and fixed it during it.
• The sharp findings: an exploit that could keep hundreds of recent agent actions unseen; a route by which a coding agent could in principle bypass Codex sandboxing and Auto-Review; inputs crafted to make the monitor fail so a session was never escalated for human review; and two client-controllable settings that could switch monitoring off for newly started agents.

OpenAI told the researchers that vulnerabilities affecting production safeguards have since been patched. METR says it has not evaluated that claim, and its review describes July, not today.

Signal: Quote the coverage number correctly: eligible is not watched, and the structural gap is the one METR names openly, they agreed upfront to publish only a high-level post and they cannot verify the fixes. The finding to hold is not that the monitor is broken, it is that every monitor needs an auditor, and every auditor needs independence it has not yet been given. Watch whether the next exercise gets the transparency METR is now asking for in its loss-of-control-risk proposal. Sources: METR’s report · Reuters

02 · Lab Watch

Anthropic proves 123 open-source libraries crash-free, with 180 million lines of proof

Anthropic’s Frontier Red Team and the formal-verification startup Theorem published Thursday: Claude turned Google’s OSS-Fuzz fuzz tests into specifications, then proved 239 entry points across 123 open-source libraries will never crash on any input under 64 MB, working at the compiled RISC-V machine-code level. The proofs total about 180 million lines of Rocq (Anthropic says Claude wrote roughly a billion lines in all, iterating), with a typical proof running ~600,000 lines for a binary with a median of 18,000 RISC-V instructions. Along the way Claude patched the bugs it found, including a rediscovery of CVE-2026-56132 in libexpat.

The mechanism is the elegant part: fuzzing can never try every input, so Claude converts each fuzz test into a mathematical spec and proves it against the binary using Theorem’s formal models of the RISC-V chip design and of a Linux process, with the Rocq proof checker confirming each proof mechanically. The trust anchor shrinks to three things: the machine models, the theorem statements, and the checker itself. “Virtually none” of the proof needs human review.

The honest caveat, which Anthropic leads with: the proofs assume single-threaded static binaries with a fixed memory budget, running on a RISC-V CPU under Linux, linked against musl libc. Most of the world runs these libraries on x86 or ARM, dynamically linked against glibc. “Formally verified” is not “bug-free,” and the gap between the proved world and the deployed world is named, not hidden.

Signal: Fuzzing proves the presence of bugs; proof proves their absence, and Anthropic just showed an LLM collapses the cost of the second from years of expert effort (the seL4 kernel took two person-years to build and twenty to verify) to a few weeks of agent time. The next question is who pays to re-run this against x86 and ARM, where the software actually ships. Sources: Anthropic’s research post · AI Weekly breakdown

Anthropic donates Agent Skills to the Linux Foundation

Anthropic is giving its Agent Skills standard, the SKILL.md format that teaches agents specific jobs, to the Agentic AI Foundation (AAIF) under the Linux Foundation. The transfer is total: the name and logo, the GitHub organization, and the agentskills.io domain. It is the second standard Anthropic has donated after MCP; the AAIF, which Anthropic co-founded last year with Block and OpenAI, simultaneously announced three more projects: Open Responses, OpenEnv, and OpenSandbox.

The adoption numbers are the story: 70+ agent platforms have integrated the standard (ChatGPT, Cursor, GitHub Copilot, VS Code among them), with roughly 1 million skills published and nearly 280 million installed. Maintenance passes to engineers from Anthropic, Google, OpenAI, and others acting as individuals, with David Soria Parra and Mahesh Murag as lead maintainers. Use and contribution stay open without joining the foundation.

Signal: The agent instruction layer is getting neutral governance, from a foundation the donor co-founded. “Neutral” here means multi-vendor, not vendor-free: watch whether the proposed roadmap (versioned releases, a formal proposal process, security guidance for skill authors) survives the gravity of the companies that both compete on agents and maintain the standard. Sources: Anthropic’s announcement · Linux Foundation

Claude Security now scans continuously and tracks fixes

Since entering public beta in April, thousands of organizations have used Claude Security to find over a million vulnerabilities. The new version stops waiting for someone to press scan: it runs a deep baseline scan, then re-scans changes based on their risk and complexity, so the findings list stays close to the current state of the code. Three details worth knowing: findings are marked fixed when the fix ships (checked against the changed code, reopened automatically if the issue returns), each project gets a knowledge base where teams describe deployment and policies (Claude generates a threat model from it to decide what is worth flagging), and every finding carries a severity rating plus an exploit-path diagram showing how an attacker could reach the vulnerable code. There is a new API and webhooks, an MCP connector in Claude Code, and role-based access controls for larger rollouts. Available to Claude Enterprise customers; $25,000 in free usage credits per month for two months (through December 31, 2026).

Claude’s full platform lands on Google Cloud

The Anthropic-managed deployment option for Claude on Google Cloud is now generally available. Same models on day zero, same pricing, same Google Cloud bill, but Anthropic operates the stack as the data processor: customers get same-day access to new Claude platform features (Claude Managed Agents in beta, skills, code execution, computer use, Files API, MCP connector) while signing in with Google Cloud IAM, keeping Cloud Audit Logging, and drawing spend down against existing Google Cloud commitments. The Google-managed option stays for strict governance (VPC Service Controls, customer-managed keys, data residency in US/EU/APAC), where new features arrive only after Google integrates them.

Also in Lab Watch: OpenAI and Poland’s NASK announced an AI safety and security partnership (Oct 23). Uber is rolling out ChatGPT Work to all employees. OpenAI posted a developer follow-up on the Decisions API (Oct 22), after its Oct 7 public-beta launch. OpenAI’s models-on-Google-Cloud deal was covered earlier this month.

03 · Open Weights

Recursive’s Flint: a 40B coding MoE that runs on a laptop, built in under four months

Recursive (Richard Socher’s lab, out of stealth in May with $650M at a $4.65B valuation) released Flint on October 21: a mixture-of-experts coding model with 39.8B total parameters and ~2.5B active per token (256 routed experts, 6 selected per token plus a shared expert), Apache 2.0, weights on Hugging Face. The numbers: 54.9% on Terminal-Bench 2.1, 89.7% on SWE-bench Pro v2, 74.2% on SWE-bench Verified (vendor-reported, agentic pass@1); 125+ tokens/sec on an M4 Pro MacBook Pro and 150+ on a 64GB RTX Spark at short contexts (company figures, prefill excluded); the BF16 checkpoint is ~80GB and serves from a single B300 or H200; local Apple-Silicon runs want 48GB+ of unified memory.

The build story is the deeper one: a small team of people and AI agents built Flint from scratch in under four months using Recursive’s automated research system, the Eureka Machine. “Nearly every aspect of the entire model pipeline was actually done and improved upon by AI,” Socher said. The company itself adds the caveats: Flint may need more detailed instructions than large closed models, and complex work should be split into subtasks. Flint is #3 on today’s HF trending list.

Signal: The MoE arithmetic is familiar; the manufacturing story is not. Flint is the first shippable artifact of the recursive-self-improvement agenda, and its thesis is that coding agents belong on the developer’s own machine, offline, with the code never leaving it. Watch whether the 125-tokens-on-a-laptop number holds up in independent hands, and whether “built by AI in four months” becomes a release cadence. Sources: Weights on Hugging Face · Testing Catalog

Trending vs. gravity wells: DeepSeek-V4.1-Flash-1017 still leads trending (2,505 likes, 57,715 downloads). Black Forest Labs’ FLUX 3 Klein keeps climbing (the Comfy repack at #2 trending with 1,363 likes and 534,837 downloads): image generation’s next gravity well is forming. OpenMOSS’s MOSS-Image-SFT (625 likes) and Base (511) are both trending while the Comfy-Org repack sits at 1.6M downloads: the runnable build still wins distribution. Google’s embeddinggemma-2 is trending (2,860 likes, 192,741 downloads). The gravity wells hold: Qwen3.8-27B (18,269 likes), FLUX.1-dev (15,712), DeepSeek-R1 (14,406), Kimi-K3 (11,805); LTX-2.5 (8,977) is the rare overlap, top-liked and trending at once.

04 · Money Flow

Google Cloud’s Vera Rubin A5X instances go GA in November, aimed at 960,000 GPUs

Announced at GTC Berlin: Google Cloud’s A5X instances, built on NVIDIA’s Vera Rubin NVL72 racks with ConnectX-9 SuperNICs, Google’s Virgo Network fabric, and fourth-generation liquid cooling, go generally available in November (the announcement said “next month”), with European expansion planned. The target workload is agentic inference and large-scale reinforcement learning, where a model generates experience, scores it, and updates itself in a loop. First customer: Ineffable Intelligence, David Silver’s London lab, building “superlearners” that learn continuously from experience. WPP tested creative-agent pipelines on it and found workloads that used to need a multi-node cluster ran on a single A5X.

The number to hold at arm’s length: Google plans to scale Vera Rubin infrastructure to as many as 960,000 GPUs across multiple sites. That is a stated scaling target, not installed capacity. The engineering pitch is the rack as the unit of deployment: Managed Lustre with GPUDirect Storage, a high-speed KV cache extending context beyond HBM, and non-blocking fabrics that let tens of thousands of Rubin GPUs act as one cluster.

Signal: The hyperscaler SKU is now the rack, not the VM. And Google’s agentic-inference framing, routing each step to the most efficient model, is the hardware version of the harness-orchestrator thesis: the scarce layer is the router, and everyone is building one. Sources: WebProNews · Vector Wire

05 · Leaderboard

OpenRouter: Hermes still closing on Claude Code

Command Code is the unmoved monopoly at ~4.16T tokens, flat. Below it the tie persists: Claude Code ~1.93T vs Hermes Agent ~1.88T, both slightly down on the rolling window, the wrapper still within striking distance of the official harness. Nous Research API holds ~1.33T. Kilo Code ~0.93T. Then pi ~0.63T, omp ~0.48T, DeepSeek Harness ~0.46T, Strix ~0.43T, CodeGPT ~0.38T, Codex ~0.35T, Freebuff ~0.27T, OpenClaw ~0.23T. Data notes: this feed’s rank numbers look reshuffled again, so trust token counts over rank positions; and there is a suspicious-looking entry (“Scammer Get Scammed,” ~0.50T, on a non-HTTPS domain) riding the mid-table, read it as feed noise until proven otherwise.

Benchmark notes (Artificial Analysis): quiet day on the boards; the Oct 22 HealthBench evals remain the newest graded work.

06 · The Argument

Trust is conserved, never eliminated

Today’s issue is one theme wearing three costumes. METR and Redwood showed that OpenAI’s agent monitor had blind spots, but their report cannot verify that the fixes landed: trust moved from the monitor to the red-team summary you cannot fully inspect. Anthropic proved 123 libraries cannot crash, but the proof rests on Theorem’s machine models, the theorem statements, and the Rocq checker: trust moved from the fuzz tests to the trusted components you did not write. Anthropic donated Agent Skills to a “neutral” foundation it co-founded with Block and OpenAI: trust moved from the vendor to the foundation the vendor helped create.

The argument: there is no trust-removal machine, only trust relocation. Every assurance mechanism in the agent era conserves the total amount of trust and shifts where it sits. The practical consequence is that the question “can I trust this?” is always the wrong question. The right one is “what exactly am I being asked to trust instead?” The vendors who name the anchor honestly (Anthropic’s RISC-V-and-musl caveat, METR’s published limitations) are the ones whose claims survive contact with the next incident. Audit the anchor, not the claim.

Signal: The scarce skill of the agent era is reading the “trusted components” section of any assurance claim. Everything else is marketing with footnotes. Sources: METR’s report · Anthropic’s research post

07 · Papers

• A Fix for Softmax and Cross-Entropy — a proposed fix at the very bottom of the training stack. If it holds, it touches every training run; read the paper before the takes.
• Reliable AI-Assisted Decisions via Selective Auditing — on theme with the monitor wars: do not audit everything, audit selectively, and prove the selection is sound.
• SpectralCanary: Embarrassingly Simple Low-Frequency Traces for T2I Dataset-Use Attribution — dataset attribution for text-to-image models via low-frequency traces. Provenance is becoming measurable.
• Questioning Instrumental Convergence in RLVR: Checkpoint Evidence from OLMo — checkpoint-level evidence pushing back on instrumental convergence in RLVR. The safety discourse’s favorite axiom, tested against weights.
• Causal Chain of Thought: Mitigating Attention Leakage via Context Isolation — attention leakage between reasoning steps, attacked with context isolation.
• EpiSEM-Bench: Evaluating Safety and Evidence Recovery in LLM Agents under Missing… — safety and evidence recovery when agents operate with missing information. Pairs with today’s monitor theme.

08 · GitHub Trending

Security week continues in repo form. cloudflare/security-audit-skill still leads daily (+2,706 today, 41,188 total): the coding-agent security-audit skill, landing the same week as Claude Security’s launch and Anthropic’s formal-verification push. NVIDIA/OpenShell added 1,240 (18,422 total): the agent runtime from yesterday’s One Thing, still accumulating. anthropics/knowledge-work-plugins (+643, 32,010) keeps its run. The PHP interpreter itself (php/php-src, +722) is still getting sympathy stars after the PHP2Shell disclosure. And the red-team counterpart to the week’s blue-team launches: SnailSploit/Claude-Red (+209, 8,145), a curated library of offensive security skills for Claude, from SQLi to EDR evasion. The rest of the board:

• Renhuai123/ziwei-doushu (+547) — an open-source Chinese astrology engine
• OpenBMB/VoxCPM (+144) — tokenizer-free multilingual TTS and voice cloning
• Weekly: morluto/rea (+106,341 this week, 371,051 total) — “reverse engineer anything with agents, from app behavior down to native binaries”; debpalash/VoiceStudio (+13,581, 82,527) the fully-local ElevenLabs alternative; stablyai/orca (+9,401) the agent-fleet UI; K-Dense-AI/scientific-agent-skills (+3,967, 54,118) the science agent-skills library; heygen-com/hyperframes (+5,669) HTML-to-video built for agents

09 · Product Hunt

Yesterday’s top 10 (October 22 board):

1. DeepCell — your deadline survival kit for office work (330 votes)

2. Claude Duet — proactive agent that understands what needs your attention (303 votes)

3. Doneas — a daily accountability call with other entrepreneurs (249 votes)

4. heal.dev — find bugs and vulnerabilities code review misses (238 votes)

5. Mistral Voxtral 3 — Mistral’s speech models for transcription, TTS and agents (207 votes)

6. Fastlane 2.0 — create, launch and scale winning ads with AI (177 votes)

7. Loom Record for agents — hit record and hand off to AI, now in the new Loom app (138 votes)

8. CutBay — AI video agent: turn anything into videos via chat (134 votes)

9. Dadan — one AI video workspace for recordings, meetings and guides (129 votes)

10. Resend Inboxes — create shared inboxes, connect your agents, build workflows (118 votes)

Claude Duet’s Product Hunt debut (303 votes) landed the same week Anthropic opened the research preview; heal.dev rides the security week; Loom and Resend both shipped agent-facing versions of existing products.

10 · Hacker News

Quad9 is leaving France

The day’s top thread (1,290 points, 444 comments): Quad9 will stop serving France on November 24 after a French court ruled for beIN, forcing the nonprofit DNS resolver to block domains for French users. The penalty was cut from €580,000 to €1,000 per day, but Quad9 says it was never about the money. The technical argument is the story: Quad9 keeps no user database and has no reliable way to map users to jurisdictions; complying globally would export French law to the whole world, and geoIP blocking already burned them once (Sony used a VPN to defeat it in the 2021–2023 German case, and Quad9 was fined for contempt). About 20 more lawsuits are pending in France against Quad9 and other resolver operators. The post’s closing line is the one the thread is quoting: fighting piracy through DNS resolvers is “disproportionate and misguided, ill-fitted and futile,” and the content stays up regardless.

The rest of the front page:

• Docker is weird — 655 points, 272 comments; the container runtime under the microscope
• Tell HN: I unknowingly broke the PHP RCE embargo. I am sorry — 654 points, 239 comments; the PHP2Shell disclosure’s human epilogue
• WorldTUI: I trained a 360M model to pretend to be a terminal — 578 points, 177 comments; a tiny world model hallucinating terminal sessions
• Every AI model has the same “controversial” take on humans — 536 points, 530 comments; the thread is arguing about the methodology, not the take
• Run full iOS 16.5 firmware on Android devices — 478 points, 177 comments
• OpenShell: a safe, private runtime for fleets of autonomous agents — 451 points, 203 comments; covered in yesterday’s One Thing
• Phone is about to stop being yours: 70 days until lockdown — 352 points, 205 comments; Android openness countdown
• Xbox wants you to pay 24 to 40 cents per hour to stream games — 349 points, 313 comments; per-hour cloud gaming pricing
• Getting ready to operate GNU Taler in Germany — 337 points, 152 comments
• Swedish government offices to ban ChatGPT and Claude — 221 points, 84 comments; the sovereignty wave reaches Stockholm

11 · OpenRouter Apps

Command Code is the unmoved monopoly at ~4.16T tokens, flat. Below it the tie persists: Claude Code ~1.93T vs Hermes Agent ~1.88T, both slightly down on the rolling window, the wrapper still within striking distance of the official harness. Nous Research API holds ~1.33T. Kilo Code ~0.93T. Then pi ~0.63T, omp ~0.48T, DeepSeek Harness ~0.46T, Strix ~0.43T, CodeGPT ~0.38T, Codex ~0.35T, Freebuff ~0.27T, OpenClaw ~0.23T. Data notes: this feed’s rank numbers look reshuffled again, so trust token counts over rank positions; and there is a suspicious-looking entry (“Scammer Get Scammed,” ~0.50T, on a non-HTTPS domain) riding the mid-table, read it as feed noise until proven otherwise.

12 · Watch

Short bench today: the clickbait titles teach nothing, so two picks.

• Fireship: “The last real programmers are dying out…” (Oct 22) — the companion piece to the Flint card: what happens to programming when the models get this fast and this local.
• Dwarkesh Patel: “AI’s bottleneck isn’t compute. It’s geometry.” (Oct 22) — pairs with the A5X card: the physical shape of AI infrastructure.

13 · Listen

• Latent Space: “Why AI Compute Comes in Rectangles” (Erik Bernhardsson, Modal CEO) (Oct 22) — the geometry of GPU clusters, from the company that rents them by the rectangle. Pairs with the A5X rack-scale card.

14 · Read

• METR: Red-Teaming OpenAI’s Internal Agent Monitoring Systems — the full post behind The One Thing, including the methodology appendix on the coverage estimates.
• Anthropic: LLMs are making formal software verification feasible at a much larger scale — the research post behind the Lab Watch card; the limitations section is the most honest part.
• Linux Foundation: Agentic AI Foundation announces four new projects — Agent Skills, Open Responses, OpenEnv, and OpenSandbox under one roof.

15 · Safe to Ignore

• Wan2.2 Animate re-surfacing — launched September 2025 (arXiv:2509.14055); the RSS packaging makes it look new, it is not.
• Z-Image Turbo “tops open-weight rankings” — released November 2025; no new announcement, RSS-only framing.
• Mistral Timestral — released October 15; the RSS story is a re-surface, not fresh.
• Voxtral 3 and Mistral–NVIDIA — covered October 21.
• Meta Muse Wonder — covered October 13–16.
• GLM 5.3 “cheapest on the market” — a partner sign-up link, not editorial signal.
• The AWS serverless marketplace item — a sponsored newsletter slot.
• “8 GitHub repos to build videos with Claude” — a listicle; in the appendix.
• The Nous robot-duck X post — a social clip with unverifiable specs.

16 · Build on This

SkillScan: the antivirus for agent skills

One million skills published, nearly 280 million installed, 70+ platforms, and every skill is a folder of markdown plus executable scripts that agents run with your permissions. Anthropic’s own security guidance says to audit unfamiliar scripts, dependencies, and network connections before installing a skill, because malicious code or instructions can expose data. Nobody does that by hand at 280 million installs.

SkillScan audits skills the way a package scanner audits dependencies. Point it at a skill repo or your local skills directory and it reports every script and dependency the skill bundles, every network call and file-system path it can touch, prompt-injection patterns in the SKILL.md itself, and a risk score with the receipts attached. The wedge is the install moment: a Claude Code and VS Code plugin that scans a skill when you add it, before the agent ever runs it. Pricing is trust-moment pricing: $49 one-time for a full repo audit, $29/month for a team plan that watches the skills your org installs and flags updates that add new network calls. The moat is the behavior corpus: every scanned skill makes the next scan smarter, and the corpus compounds exactly as fast as the skill ecosystem grows, which right now is very fast. Yesterday’s card audited what your infrastructure trusts; this one audits what your agent is allowed to run.