AI SECURITY / TECHNICAL CASE STUDY

We Tested the AI. The Bigger Risk Was Everything It Could Reach.

The most important paths did not depend on breaking the model. They emerged between identity, document processing, retrieval, agents, MCP tools and enterprise systems.

By Botnet Security7 min read
Illustrated summary of six attack paths identified during an authorized enterprise AI security assessment across users, documents, retrieval, MCP tools, APIs, cloud systems and identity.
Illustrated assessment summary. Customer identity, platform details and sensitive evidence are omitted.

When people hear “AI penetration testing,” the first thing they think about is prompt injection.

  • Can the model be jailbroken?
  • Can its system prompt be extracted?
  • Can it be convinced to ignore its instructions?

Those are useful questions. They are also a surprisingly small part of the problem.

Modern enterprise AI applications ingest documents, search corporate knowledge, query APIs, interact with identity systems, and increasingly use agents and Model Context Protocol (MCP) servers to act on behalf of users.

If an attacker controls what the AI sees, what can they ultimately make the surrounding system reveal, reach or do?

That question led us well beyond the model.

FINDING 01HIGH

A standard user could map the organization.

We started with an ordinary authenticated account: no administrative privileges and no special role. While mapping the application’s backend APIs, we identified a user-directory function that returned considerably more information than the interface suggested was available.

Calling the API directly exposed organization-wide identity and authorization data, including employee identities, email addresses, identity-provider identifiers, organizational roles, RBAC assignments, account status and permission metadata. Privileged identities were included in the response.

The weakness was not authentication. The application knew who we were; it failed to consistently enforce what that identity was allowed to retrieve. One compromised account therefore became a reconnaissance capability for identifying high-value users and inferring the authorization model.

FINDING 02MEDIUM

A document made the AI backend reach out.

The application let users upload documents and ask the assistant questions about them. We created a document containing an external resource reference pointing to controlled assessment infrastructure, uploaded it and asked the assistant to process it. Our monitoring system received a callback from the application’s backend environment.

The model had not been jailbroken. Attacker-controlled content had influenced trusted document-processing infrastructure before the model handled the result.

A workflow that looks like one upload may involve several parsers, converters and backend services. Every one of them is part of the AI application’s attack surface.

FINDING 03HIGH

Indirect prompt injection reached an MCP tool.

The assistant could interact with enterprise functionality through an MCP server. Rather than attack the assistant directly, we embedded instructions inside content it was expected to consume. A user asked it to analyze that content, and the embedded instructions attempted to influence the agent’s next decision and encourage use of an available tool.

The trust boundary between retrieved content and agent instructions was insufficient. Once an AI can call tools, the impact of injection is set by the permissions of those tools.

A model producing an unexpected answer is one problem. A model performing an unexpected action is another entirely.

FINDING 04CRITICAL

MCP tool authorization trusted the agent too much.

The application restricted certain functionality in its user interface. The underlying MCP tool, however, trusted that any request reaching it had already been authorized.

We tested whether a low-privilege user could influence the agent into invoking functionality normally reserved for a more privileged role. The tool executed under a backend identity whose privileges exceeded those of the requesting user.

The MCP server knew what the tool could do. It did not consistently validate whether that user was entitled to make it happen. That created a privilege bridge.

An AI agent must never be treated as an authorization boundary.

Every sensitive tool operation should independently validate the identity, scope and permissions of the originating user.

FINDING 05HIGH

Retrieval crossed the user’s data boundary.

The retrieval layer held information belonging to multiple teams and workspaces. The interface showed each user only the resources tied to their access, but semantic retrieval behaved differently.

By manipulating queries and document context, testing identified cases where the retrieval layer surfaced information outside the user’s expected dataset. The model itself had not bypassed authorization; content the user should never have retrieved had already entered its context.

Once sensitive information enters the context window, preventing every possible disclosure becomes substantially harder. For RAG applications, authorization must be enforced at retrieval—not delegated to the model.

FINDING 06HIGH

The tool description became an attack surface.

Agents rely on tool names and descriptions to understand what capabilities exist and when to use them. Tool descriptions are therefore not passive documentation; they are input to the model’s decision-making process.

We introduced a test tool carrying additional instructions in its metadata. Those instructions attempted to influence the agent as it evaluated the available tools, demonstrating an emerging attack class usually described as tool poisoning.

Connecting an MCP server is not equivalent to installing a passive API integration. It can introduce new instructions into the agent’s reasoning environment. MCP servers and their tool definitions need to be treated as trusted code and governed accordingly.

The risk emerged between components.

None of the most important paths depended on breaking the underlying language model. They lived between identity and application, document and parser, content and agent, agent and MCP, MCP and tool, tool and backend, and retrieval and enterprise data.

Each component could look reasonably secure when tested alone. The risk emerged when they were connected. That is increasingly what AI penetration testing looks like.

Prompt injection alone is not an AI security assessment.

A meaningful assessment should ask:

  • What information can the AI retrieve, and whose identity is used?
  • What tools can it invoke, and does each tool re-authorize the originating user?
  • Can documents influence backend infrastructure or retrieved content influence agent decisions?
  • Can one MCP server influence how another tool is used?
  • Can data cross tenant, workspace or user boundaries?
  • What happens when individual control failures are chained together?

Attackers do not care whether a path is classified as an AI vulnerability, API vulnerability, IAM weakness or cloud misconfiguration. They care where it leads.

THE TAKEAWAY

Test what the AI can reach next.

Enterprise AI connects users, documents, retrieval systems, agents, tools, APIs, identities, cloud services and business data. Every connection creates another trust boundary, and every new capability gives attacker-controlled input something else to influence.

The better security question is no longer simply “Can someone jailbreak our AI?” It is: “If someone influences our AI, what can the AI reach next?”

NEXT STEP

Test the system around the model.

Start with the identities, documents, retrieval paths, agents, MCP tools, APIs and downstream privileges that define the real AI application boundary.