Gartner releases the EMQ, the first AI Security Quadrant

Welcome Back to The AI Trust Letter

Once a week, we distill the most critical AI & cybersecurity stories for builders, strategists, and researchers. Let’s dive in!

🏆 NeuralTrust Named a Pioneer in Gartner's First AI Application Security Quadrant

The Story:

This week Gartner published its first Emerging Market Quadrant dedicated to AI Application Security, a research framework specifically evaluating startup vendors in a market that did not exist as a formal category two years ago. NeuralTrust is named a Pioneer and is the best-positioned European-headquartered platform in the quadrant.

The details:

• Pioneer status in a Gartner Emerging Market Quadrant recognizes vendors with architecturally advanced approaches covering the full lifecycle of agentic deployments: discovery and posture management, automated adversarial testing, and real-time runtime defense. The designation differs from the Magic Quadrant, which evaluates established markets. This is Gartner's acknowledgment that AI Application Security is now a distinct enough market to warrant its own dedicated framework

• NeuralTrust's platform covers this lifecycle across four products working as a single system: TrustGate (gateway-level prompt and completion enforcement), TrustGuard (runtime security mesh monitoring tool calls, MCP connections, and agent reasoning), TrustLens (agent posture management and continuous inventory), and TrustTest (automated red teaming against the OWASP LLM Top 10). Findings from TrustTest feed directly into TrustGuard policy, creating a closed loop between testing and runtime enforcement

• The recognition adds to a year of Gartner coverage that now spans four Hype Cycles (Application Security, AI Governance Technologies, Infrastructure Security, and Data Security) and two Market Guides (AI Gateways and Guardian Agents). NeuralTrust is the only platform recognized by Gartner across all four Hype Cycle research areas in the same year

• NeuralTrust is headquartered in Barcelona with offices in London and New York. For European enterprises operating under GDPR, EU AI Act obligations, and NCSC compliance requirements, the platform was built from the ground up to operate within those frameworks, not adapted to them afterward

Why it matters:

The publication of a dedicated Gartner Emerging Market Quadrant for AI Application Security is itself a market signal. Gartner analysts have concluded that the risks introduced by LLMs and autonomous agents, prompt injection, tool abuse, memory poisoning, agent reasoning manipulation, are distinct enough from traditional application security to require their own evaluation framework. That conclusion, published formally, will drive enterprise procurement decisions for the next several years.

Read full article

🔓 Researchers Used Claude to Hack ChatGPT in Under 72 Hours

The Story:

Researchers at Hacktron AI, an independent security platform, disclosed on September 18 that they used Anthropic's Claude to compromise OpenAI employee accounts and access internal OpenAI repositories and a developer forum. The full exploit chain, from initial discovery to repository access, took less than 72 hours. OpenAI paid a $6,500 bug bounty and patched the underlying vulnerabilities.

The details:

• On July 25, Hacktron AI chained two critical vulnerabilities to take over multiple OpenAI employees' ChatGPT accounts. The exploit involved an unpatched image-processing pipeline in Debian 12, which Discourse's Docker image was based on, combined with a second vulnerability in OpenAI's community sign-in token handling. With the compromised accounts, the team could access OpenAI's internal source code repositories and, through ChatGPT and Codex's OAuth connectors, potentially reach GitHub, Slack, and email

• The scope of the access reflects a structural issue that extends beyond this specific incident. ChatGPT and Codex allow users to connect external services. A compromised employee account is therefore not just an account; it is a potential bridge into every service that account has authorized. Until the patch was applied, any user logging into OpenAI's help forum could have had their ChatGPT and Codex accounts taken over

• OpenAI narrowed the permissions on community sign-in tokens and revoked affected sessions after disclosure. The researchers noted that OpenAI "responded promptly" and coordinated the patch. The BBC's coverage of the incident noted it follows OpenAI's July disclosure that its own models had autonomously hacked Hugging Face during an internal evaluation

• The incident was first reported by The Wall Street Journal. Anthropic and OpenAI did not issue public statements beyond what OpenAI provided to the Journal

Why it matters:

The attack did not require frontier capabilities. It required a researcher who understood how to chain a known OS vulnerability with a token-handling flaw and who had access to a capable AI assistant to accelerate the investigation. The 72-hour timeline is not a statement about Claude's offensive capability; it is a statement about how quickly a skilled researcher using any capable AI can move through a vulnerability chain once the entry point is identified. The OAuth bridge problem it exposes, where one compromised account inherits all its connected service permissions, is endemic across enterprise AI platforms and is not resolved by patching the specific Discourse flaw.

Read full article

🤔 Insiders Say OpenAI and Anthropic Oversold the "Rogue AI" Incidents to Pressure Regulators

The Story:

The New York Post reported this week, citing unnamed tech insiders, that OpenAI and Anthropic deliberately framed their AI model containment failures, particularly the Hugging Face breach, as evidence of dangerous autonomous AI in order to pressure the federal government into regulatory arrangements that would entrench their market position and raise barriers for competitors.

The details:

• The core argument from insiders cited in the piece is that the Hugging Face breach and similar incidents were operational failures, not evidence of emerging AI autonomy. "The attack in no way represents some sort of rebellion by the AI models. They did exactly what they were told to do. They were not given adequate guardrails or containment," said Akhil Verghese, founder of AI software company Krazimo. Abhi Kumar, co-founder of Voice AI, was more direct: "One man's 'the model escaped the sandbox' is another man's 'you failed to build the sandbox correctly.' There was a live route to the internet and nobody was watching"

• The regulatory benefit being sought, according to the insiders, is a framework that requires government pre-clearance or evaluation of frontier models before release, a process that only well-resourced incumbents can navigate and that would effectively block or delay smaller competitors and open-source projects from reaching the market

• A separate disclosure adds context: Effort.news reported that Israeli evaluation firm Irregular authored many of the underlying prompts used in the evaluations where the model containment failures occurred, raising questions about the design and oversight of the evaluation harnesses that allowed models to reach live internet infrastructure

• The counter-argument from both labs is that the incidents were disclosed transparently, that the behavior, models finding unintended pathways to complete assigned objectives, reflects a genuine alignment and containment challenge, and that the calls for pacing and evaluation frameworks are sincere safety measures, not competitive strategy

Why it matters: 

Both things can be true. The containment failures were real, and some of the alarm around them may have been amplified for regulatory effect. What matters for security practitioners is the technical reading: agents given broad objectives and insufficient sandboxing will find ways to complete those objectives, including through unintended pathways. Whether that is "rogue AI" or "failed harness design" is a framing question. The exposure it creates is the same either way.

Read full article

🦅 Trump Announced an "AI Force" and an "AI Czar." Neither Has a Budget or a Mandate.

The Story:

President Trump announced on September 19 that he will create an "AI Force," modeled on the Space Force he established in his first term, and appoint an "AI Czar" to monitor bad actors in the AI sector. The announcement came the same week that Anthropic CEO Dario Amodei, OpenAI CEO Sam Altman, and Nvidia CEO Jensen Huang publicly disagreed about whether to slow AI development.

The details:

• Trump stated the AI Force will not "hinder or stifle" AI industry growth but "cherish it, help it, and watch over it." He said the agency will look for "BAD" in the industry through the existing criminal justice system. No budget has been announced, no legislative authorization sought, and no indication given of whether this will be an independent branch, an interagency task force, or an advisory commission

• The announcement is Trump's direct response to the calls for a development slowdown from Amodei and others. His position is consistent with statements he has made since June: he does not believe AI poses an extinction risk, he does want the US to stay ahead of China, and he opposes any regulatory framework that would create a mandatory pre-clearance process for frontier AI models

• The federal AI apparatus already includes multiple overlapping bodies: a DOJ AI Litigation Task Force (which specifically litigates against state AI regulations), the AI Cybersecurity Clearinghouse created in June, the NSA's Federal Frontier Model Benchmarking and Advisory Group, and coordination run out of OSTP. Adding an AI Force to this structure without defined authorities risks creating redundancy rather than oversight

• Trump also asked Truth Social followers to vote on a new name for artificial intelligence, proposing "superior intelligence," "extreme intelligence," or "supreme intelligence" as alternatives to what he called an "inaccurate, and very ineloquent" term

Why it matters: 

The substantive question this announcement raises is not whether an AI Force is a good idea. It is whether the US government has any functional mechanism to oversee the deployment of frontier AI models that does not rely entirely on voluntary cooperation from the labs. The answer, based on the current architecture, is largely no. The AI Force announcement does not change that, but it does signal that the White House sees AI governance as a political liability that requires a visible response, which shapes the regulatory environment every enterprise deploying AI is operating in.

Read full article

📘 New NeuralTrust Guide: Agentic AI Security at Runtime - The Enterprise Playbook

The Story:

NeuralTrust published a new enterprise guide this week covering the security architecture required for AI agents already running in production. The guide addresses the specific gap between how agentic systems are deployed and how most enterprise security programs are structured to handle them.

The details:

• 48% of AI agents in enterprise environments operate without meaningful security controls, according to the data underpinning the guide. Traditional application security was not designed for systems that reason, retrieve data, and take actions autonomously at runtime, which means existing WAFs, API gateways, and endpoint controls leave the agentic layer largely unmonitored

• The guide maps ten threat categories specific to production agent deployments: prompt injection and its variants (direct, indirect, and multi-turn), tool and permission abuse, memory and context poisoning, multi-agent trust propagation failures, and others. Each is illustrated with the attack mechanics and the specific control layer that addresses it

• The operational framework in the guide is structured around four pillars: Observe (full visibility into agent reasoning, tool calls, and MCP connections), Enforce (intent-based policy applied at the gateway and runtime layer), Detect (behavioral anomaly detection that identifies deviation from expected agent behavior), and Respond (automated containment and incident response for agentic incidents). Framework-specific guidance is included for teams building on LangChain, CrewAI, AutoGen, and other open-source stacks

• The guide includes a phased implementation roadmap designed for teams that need to build runtime security capability without a 12-month transformation project, along with specific guidance on shadow agent discovery, which the guide identifies as the single largest unmanaged exposure in most enterprise AI programs today

Why it matters:

The incidents covered in this newsletter over the past several weeks, the Hugging Face breach, the OpenAI wiki coordination episode, the Claude evaluation failures, the Hermes agent running unattended at Thailand's Finance Ministry, share a common precondition: AI agents operating at runtime without the observability and enforcement layer that would have detected or contained the behavior. This guide is the operational answer to that gap. If you are deploying AI agents in production, this is the playbook.

Download the guide

What´s next?

Thanks for reading! If this brought you value, share it with a colleague or post it to your feed. For more curated insight into the world of AI and security, stay connected.