Skip to content
frunch

AI Gateways Are Part of Your Security Boundary—Whether You Treat Them That Way or Not

A researcher’s canary experiment reportedly found prompt theft, credential misuse and privacy-setting failures across parts of the AI service supply chain. For developers, the lesson is to assess every provider and reseller that can handle prompt data—not just the model vendor.

What happened

A report describing security researcher HD Moore’s experiment says he sent nearly two million prompts containing unique, trackable secrets through AI services. The test used credentials tied to empty AWS accounts, passwords and private URLs, allowing Moore to trace where exposed information appeared. According to the report, the findings included evidence that attackers at three hosting providers copied prompts, inference resellers logged prompts and passed them to people who used the credentials, and some providers did not honor privacy settings such as Zero Data Retention (ZDR). [1]

The report also describes a provider’s systems fetching secret URLs from more than half of prompts sent to one AI lab’s models during a period from July to October—including many requests marked as “no data collection” or ZDR. Some US providers resold those models, so routing a request through a US company did not necessarily prevent it from reaching the lab. [1]

This article is based on a supplied excerpt, not the full report. The excerpt does not identify all providers or provide the complete experimental methodology, so the findings should not be treated as a comprehensive measurement of the entire AI market.

Why it matters for developers

An AI request can pass through several organizations: an app, a gateway or router, an inference provider, and sometimes hosting infrastructure. The report says prompts are decrypted at each hop so services can route or process them. That means encryption in transit does not stop a service that handles a request from seeing its contents. [1]

For developers, the practical security boundary is therefore wider than the model API named in the code. A gateway may offer useful model selection or pricing, but it also becomes another party with potential access to prompts. Likewise, a privacy flag is a provider’s declared handling preference—not protection against a compromised system or a party that ignores the setting, as the reported incidents illustrate. [1]

A sensible response is to treat prompts as data shared with every intermediary in the route. Avoid putting live credentials, private keys or other reusable secrets into prompts; where a task requires sensitive context, consider whether it can be redacted, substituted or processed in a more controlled environment. Keep credentials used by AI-connected systems narrowly scoped and revocable. These are defensive recommendations based on the risks described, not guarantees that any particular architecture is safe.

Teams should also map which vendors and resellers can receive prompts, review their retention and routing terms, and check that actual routes match data-residency and privacy requirements. The experiment’s use of unique canaries suggests a further option for security teams: controlled, non-production test data can help reveal unexpected access or downstream handling. Such tests need careful design and should never use real credentials as bait.

What remains unproven

The excerpt reports serious findings, but it does not show the full set of tested providers, how representative they were, or the rate of exposure across all requests and services. It also does not establish that every AI gateway or model provider behaves similarly. The results support scrutinizing the chain, not assuming that every vendor is compromised.

My take

The most important engineering lesson is that AI convenience layers are also data-handling layers. Gateways can be genuinely useful for routing and model choice, but the tradeoff is another service that may see sensitive input. I expect mature teams to increasingly evaluate the entire request path—not just model quality, latency and price—before sending confidential data. That is an inference, not a demonstrated industry shift. The report gives a strong reason to make that review now, while leaving the scale of the problem across the market uncertain. [1]

Sources