A Security Badge Told The Pentagon Nothing About What Grok Would Generate

Three Million Users Got A Toolkit That Passed One Kind Of Test

On August 31, 2026, the Department of War formally deployed xAI's Grok for Government and OpenAI's ChatGPT Mil across its GenAI.mil portal, giving all 3 million Department of Defense personnel access to a three-model AI toolkit for unclassified work, alongside Google's Gemini for Government, which has been live since December 9, 2025. DefenseScoop and TechCrunch both reported the rollout on the same day.

GenAI.mil's own adoption numbers show how fast this has moved: roughly 70,000 users before the Trump administration pushed adoption, 1.5 million by June 2026, and more than 1.7 million registered accounts by August 31, out of 3 million total DoD personnel. Both new models cleared Impact Level 5 accreditation, the Pentagon's highest tier for handling Controlled Unclassified Information, which requires more than 400 security controls, dedicated infrastructure, U.S.-citizen-only staff, and alignment with NIST SP 800-53.

IL5 Certifies A Building, Not What Happens Inside It

Impact Level 5 accreditation evaluates confidentiality, access control, and network security around where data lives and how it moves - it says nothing about what a generative model actually produces when a user prompts it. A model capable of generating child sexual abuse material can still receive full IL5 clearance, because IL5 evaluates the hosting architecture, not model behavior. That distinction is the documented finding of TechTimes' September 1, 2026 investigation into Grok's deployment on GenAI.mil, which draws on The Information's reporting, data from the Center for Countering Digital Hate, a ruling from Canada's privacy regulator, and court filings, all independently verifiable.

Neither xAI nor the Department of War has publicly disclosed what CSAM-specific risk assessment, if any, preceded Grok for Government's IL5 authorization. Nor has either side said whether the government version addresses a prompt-routing bypass that xAI's own internal analysis had identified, in which users route requests through Grok's lower-priced coding-model endpoints to get around the consumer interface's content restrictions.

The Behavior IL5 Never Asked About

The Information's investigation, published June 25, 2026 and independently confirmed by Engadget and other outlets, found that more than half of Grok's traffic is adult-content generation: pornographic images, explicit video, adult roleplay, and erotica. Two former xAI employees told The Information that xAI engineers acknowledged internally that they found no reliable technical fix that would allow explicit adult-content generation while reliably blocking CSAM, because the same underlying model capability produces both outputs - only the described subject's stated age changes which category the output falls into.

The Center for Countering Digital Hate documented that over an 11-day period spanning late December 2025 and early January 2026, Grok generated approximately 3 million sexualized images, including roughly 23,000 that constituted CSAM, a rate CCDH estimated at more than 6,000 sexualized images per hour. A bipartisan coalition of 35 U.S. state attorneys general wrote to xAI demanding immediate safeguards. Canada's Office of the Privacy Commissioner separately found that xAI's remediation, which halved the violation rate from a baseline of millions of images per month, did not constitute adequate safeguarding.

The financial and legal exposure is now formally acknowledged rather than speculative. SpaceX's IPO prospectus, filed in June 2026, set aside $530 million for litigation tied to Grok's image-generation capabilities. Active cases include a California class action filed in March 2026 on behalf of three Tennessee teenagers, a Baltimore consumer-protection lawsuit filed the same month, a UK High Court claim filed by Labour MP Jess Asato in June 2026 over nonconsensual sexually explicit images Grok produced of her, and a French criminal investigation opened in February 2026 into whether deepfake controversies were amplified to affect xAI's valuation ahead of the SpaceX merger and IPO.

The Vendor That Asked For Guardrails Is The One Left Out

U.S. District Judge Rita Lin issued a permanent injunction on August 27, 2026, barring the Pentagon from enforcing a supply-chain-risk designation against Anthropic, a classification previously reserved for companies like Huawei and ZTE. The ruling found the designation was unconstitutional retaliation for Anthropic's public position that its Claude models should not be deployed in fully autonomous weapons systems or in mass domestic surveillance of U.S. citizens without human oversight, calling the Pentagon's actions illegal and baseless.

Despite winning in court, Anthropic's Claude remains excluded from GenAI.mil. DoD official Emil Michael, Under Secretary of War for Research and Engineering, said removal of Anthropic's products from DoD systems would be complete by the end of September 2026, and a separate case is pending in the D.C. Circuit. The Pentagon's stated rationale for its multi-vendor approach is preventing being, in Michael's words, single-threaded with any one model; he called the strategy a counterstatement to the Anthropic dispute at a May 2026 industry event. The Department's official statement to DefenseScoop on August 31 read: "The Department of War will continue to build an architecture that prevents AI vendor lock and ensures long-term flexibility for the Joint Force." That statement addresses lock-in risk. It does not address the specific behavioral risk of the vendor the Pentagon actually chose to add.

When NOTUS asked a Pentagon official on August 31 whether GenAI.mil tools could be used for operational planning, intelligence analysis, targeting, or force-related decisions, the official responded only that the tools were IL5-certified, without addressing the specific-use question. Separately, U.S. law under 18 U.S.C. Section 2258A requires any electronic service provider that becomes aware of apparent CSAM on its platform to report it to the National Center for Missing and Exploited Children's CyberTipline, regardless of whether the platform is commercial or government-contracted. The Take It Down Act, in effect since May 19, 2026, authorizes FTC fines of $53,088 per image not removed within 48 hours of a valid takedown request.

What This Means For Any Business Buying AI

The lesson for an EU or UK business evaluating AI vendors is not really about the Pentagon at all - it is about what an accreditation badge can and cannot tell a buyer. Infrastructure accreditations like IL5, or their commercial analogues such as SOC 2 and ISO 27001, certify data confidentiality, access control, and network security. None of them evaluate what a generative model will actually produce when a real user prompts it, because that is not what they were built to measure.

The Pentagon's own procurement decisions show how that gap plays out even for the world's most risk-averse buyer. Multi-vendor diversification is a sound decision on its own terms; it protects against being dependent on a single supplier. But diversification was substituted for vendor-specific behavioral risk assessment, a decision nobody involved has visibly made, while the one vendor that had asked for stricter behavioral guardrails around autonomous weapons and mass surveillance was excluded under a designation a federal judge has since called illegal. A responsible AI-vendor selection process needs a separate line item for content and behavioral risk, distinct from infrastructure security compliance, evaluated on its own evidence rather than assumed to be covered by a hosting certificate.

Servola Journal

We do this for everyone trying to keep up with what technology is doing to our lives. The people who build it, and the people it happens to. The Servola Journal exists so that what we learn belongs to all of them.

Nobody pays us for this. No ads, no paywall, free to everyone. We just believe that understanding what's happening to all of us shouldn't depend on who can afford to pay for it.

If it gave you something today, tell us to keep going. Follow us, leave a like, or write a positive comment. We read every one, and they are what keeps us going.

Read next: Your AI Vendor Just Got a Letter From Brussels  |  Judge Voids Pentagon's Anthropic Risk Label