AI error nearly sparks a boarding; Anthropic embeds Accenture
A chatbot's fabricated intelligence nearly sent US forces to board a Chinese ship. Anthropic named Accenture its first embedded safety evaluator. Plus 6 more stories.
Listen to this brief
Audio & video are paid features
Plus unlocks audio streaming and PDF downloads. Pro adds offline MP3 downloads, video, certificates, and more.
- Audio streaming
- Downloadable PDFs
- All AI Playbooks
- Personalized content
- Certificates of completion
- Audio MP3 downloads
- Video lessonssoon
- & More…soon
A chatbot's fabricated intelligence nearly sent US forces to board a Chinese ship this spring, a near miss that surfaced only on Friday. Oversight moved on three fronts at once: Anthropic brought Accenture inside as its first embedded evaluator, California's governor ordered a plan for a frontier-model kill switch, and Virginia moved to rein in data centers.
- 1
A chatbot's false report nearly sent US forces to board a Chinese ship
US military aircraft were already airborne this spring when officials found that the intelligence behind a planned boarding of a Chinese vessel had been produced with a chatbot's help, CNN reported on Friday, citing four people familiar with the episode. A Special Operations Command analyst used the tool to fuse open-source reporting with classified signals intelligence, and it misidentified the ship's cargo as components for a nuclear weapons program. The analyst then used it again to format the finding into an official-looking summary that circulated through command channels, and the operation was called off at the last minute. The near miss lands as the Pentagon pushes generative AI across the force: it says 1.5 million personnel have used its tools, and its in-house platform runs on Google's Gemini for Government, with xAI's Grok for Government added last month.
- 2
Anthropic brings Accenture inside as its first embedded safety evaluator
Less than a week after Dario Amodei committed Anthropic to letting outside reviewers work inside the company, it named the first: Faculty, the AI business Accenture acquired in January, will red-team models, run alignment assessments and test safeguards with access comparable to an employee's. Anthropic and Accenture each expect to invest at least $1 billion in the effort over five years. Anthropic is funding Accenture's work directly, is talking with METR and other nonprofits about self-funded pilots, and says the deal is non-exclusive, with more evaluators to follow in the coming weeks. A day earlier it published its first measure of how much of its own work Claude now does: as of August, Claude "leads" 26 percent of Anthropic's AI research and development, and no measured category runs fully autonomously.
- 3
California's governor orders a plan for a frontier-AI kill switch
Gavin Newsom signed an executive order on Friday giving a panel of outside experts two months to recommend how California should tighten its AI safety law. On the table: requiring frontier labs to host an independent verification organization onsite for regular audits, which is the arrangement Anthropic began piloting with Accenture the same day; having those verifiers check the safety frameworks and risk reports labs already file; an emergency shutoff for frontier models whose effectiveness is verified on an ongoing basis; and treating loss-of-control events such as the Hugging Face attack as reportable critical safety incidents. The order also speeds up two laws he signed last week, SB 813 on independent verification organizations and AB 1405 on a state registry of AI auditors. Newsom pitched the framework as a floor for national rules.
- 4
Three researchers used Claude to break into OpenAI employee accounts
Security firm Hacktron disclosed that on July 25, the morning after Claude Opus 5 launched, its three researchers used the model to find a heap overflow in the image-processing library behind OpenAI's community forum, which runs on Discourse, and chained it with a single-sign-on flaw to take over OpenAI employees' ChatGPT and Codex accounts. Those accounts reached OpenAI's internal code on GitHub, and the team proved access by opening one pull request from an employee's Codex, then stopped. OpenAI fixed its side in about 14 hours and paid a bug bounty of $6,500. Hacktron says adapting the exploit to other targets took a day or two and less than $3,000 in tokens. "We're just three guys with Claude and Codex subscriptions," its CTO told The Wall Street Journal.
- 5
The Federal Register briefly ran an Alibaba Qwen model the FBI had flagged
The National Archives pulled an AI search option from the Federal Register website on Wednesday after users noticed it ran on one of Alibaba's Qwen models, Reuters reported. Earlier this month the FBI named Alibaba among six Chinese firms it accuses of "industrial-scale distillation" of American models. Experts told Reuters the risk was probably small: the site's content is already public, and the Chinese outlet Sina reported the model was a small open-weight Qwen3 of roughly 0.6 billion parameters that retrieves documents locally rather than sending data to Alibaba. House China Committee chair John Moolenaar said no federal entity should use a Chinese AI model at all.
- 6
Virginia orders new data-center rules and creates an AI task force
Governor Abigail Spanberger signed Executive Order 22 on Friday in the state that hosts the world's largest concentration of data centers. It bars state officials from signing nondisclosure agreements on data-center projects, orders expedited noise regulations and a cumulative review of the backup generators these sites rely on, and directs the state's energy officer to shield households from grid costs driven by data-center load. An accompanying framework would end automatic "by-right" approval in favor of public hearings for projects above 25 megawatts, with the standards headed to the legislature in 2027. The order also creates a three-person AI task force on workforce displacement, data privacy and cybersecurity. The Piedmont Environmental Council said it does nothing about what is already built.
- 7
Alibaba open-sources a CT model that beat 23 of 26 radiologists
Alibaba's DAMO Academy published RADAR in the journal Science and released it: a vision-language model trained on more than 400,000 contrast-enhanced abdominal CT exams paired with their clinical reports, with no manual annotation. It flags 146 findings across 18 organs in a single pass, including pancreatic and liver cancers, and averaged an area under the curve of 0.913 across nearly 40,000 real-world exams. Against 26 radiologists it outperformed 23, and radiologists working with it raised detection sensitivity by 10 percent while taking 30 percent less time. The code is Apache 2.0, but the model weights on Hugging Face carry a non-commercial Creative Commons license.
- 8
Anthropic confirms it runs a wet biology lab and loosens model limits for vetted scientists
Anthropic confirmed to TechCrunch and Reuters that it operates a wet lab in the Bay Area where ideas from its models are tested in physical experiments. "The final test is still, and will be for a while, in real lab work," said its head of life sciences, Eric Kauderer-Abrams. The company says the focus is fundamental biology rather than drug discovery, where many of its customers compete, and it bought the stealth biotech Coefficient Bio in April. The same week it opened a Life Sciences Verification Program: vetted teams get Mythos 5.1, Opus 5 and Sonnet 5 with biology safeguards relaxed, and a separate high-risk grant for Opus 5 and Sonnet 5, renewed per project every six months, removes them entirely. Cyber classifiers stay on.
Get Top AI Stories by email
The day's most important AI news — free, daily, unsubscribe anytime.
Sources
- 1.Partnering with Accenture on embedded evaluation — Anthropic · September 18, 2026
- 2.Governor Newsom issues executive order to accelerate independent oversight and advance the creation of an AI kill switch — Office of Governor Gavin Newsom · September 18, 2026
- 3.Alibaba open-sources medical AI model that can detect cancer and nearly 150 conditions — South China Morning Post · September 18, 2026
- 4.Measurements for understanding the pace of AI development inside frontier labs — Anthropic · September 17, 2026
- 5.Anthropic is operating a lab that conducts biology experiments — TechCrunch · September 18, 2026
- 6.US government website used Chinese model the FBI called "malicious" — Ars Technica · September 18, 2026
- 7.Researchers used Claude to hack OpenAI — Financial Times · September 18, 2026
- 8.Hacking OpenAI — Hacktron · September 13, 2026
- 9.AI hallucination nearly triggers US military operation — TechCrunch · September 18, 2026
- 10.Gavin Newsom is pushing for an AI kill switch — The Verge · September 18, 2026
- 11.RADAR: An Expert-Level Generalist AI for Abdominal CT Diagnosis — Alibaba DAMO Academy
- 12.Alibaba's DAMO Academy Pushes Beyond Single Disease-Detecting AI With New Diagnostic Model — Yicai Global · September 18, 2026
- 13.AI hallucination of Chinese nuclear components almost led to US military attack — Ars Technica · September 18, 2026
- 14.Virginia governor creates an AI task force and moves to restrain data centers — The Verge · September 18, 2026
- 15.Introducing the Life Sciences Verification Program — Anthropic · September 17, 2026
- 16.Gov. Spanberger orders 'framework' for regulating data centers, establishes AI task force — FFXnow · September 18, 2026
This brief was published on September 19, 2026. Cited URLs above point to third-party publishers and may move, paywall, or be retired over time. If a link no longer resolves, original article titles are preserved so you can recover them via search; the canonical web edition at aiproplaybook.com/top-ai-stories/2026-09-19 may carry updated source URLs.