Tag: Dragos

  • Dragos Canary Study Finds AI Prompt Leaks and Provider Breaches (2026)

    Dragos Canary Study Finds AI Prompt Leaks and Provider Breaches (2026)

    Takeaway: On November 19, 2026, Dragos published AI Supply Chain Security: What Canary Tokens Found, a study led by chief research officer HD Moore. From July through November 2026, the team sent approximately two million prompts containing unique, trackable secrets to widely used AI services. The post reports three main findings: evidence that data from at least three AI providers was exposed, marketplace resellers that could view and retain customer prompts through ordinary logging, and multiple cases in which privacy settings or published retention policies appeared not to be honored. Dragos says this does not mean teams should stop using AI services, but that where prompts go is a decision.

    The details below come from that official Dragos blog post. The article does not name the three providers it says were compromised, and this write-up does not invent those names. It also does not reconstruct attack steps or publish indicators beyond the high-level findings Dragos already disclosed.

    What Dragos tested

    Most people picture inference as a question and a reply. Dragos argues that the prompt and the response are often more sensitive than the compute used to produce them. Providers may store that data for operations, training, safety, abuse monitoring, or business purposes, and it may pass through model developers, cloud operators, inference platforms, and third-party resellers.

    To see what happens after a prompt leaves a customer’s environment, the researchers created unique canaries — URLs and credentials that legitimate systems would never use. Any later attempt to access those assets would show that someone in the AI supply chain had seen the original prompt. The team started in July with Thinkst Canarytokens. Each request carried an AWS key, a DNS name, and a URL that had not been seen elsewhere. Tokens began firing within days, and Dragos says tokens from July were still firing when the post was published.

    The project later used a dedicated seeding service that derived unique hostnames, URLs, and credentials for every prompt and ran the services those credentials pointed to. The AWS keys were real keys tied to empty accounts with full logging. Each later request that touched a token could be tied back to the exact prompt, model, router, and privacy setting that carried it.

    Three provider compromises

    Dragos reports activity indicating that data from at least three AI providers was exposed. Seeded test prompts were later used in attempts to access Dragos-controlled test systems and canary AWS accounts. The earliest incident dates to mid-July 2026. According to the post, none of the three providers knew about the breach until Dragos notified them.

    The largest case involved a couple hundred of the team’s AWS keys. Each key was sent in exactly one request to one provider, all within short windows in late October, and every request sent during those windows was stolen. Some stolen keys came from requests the provider rejected before a model ran. Dragos reads that as evidence the copies were taken at the provider’s front door, not deeper in the inference stack. In the other two cases, some of the stolen requests had been marked for zero data retention (ZDR). “A privacy flag doesn’t help when the attacker is already inside,” the post says.

    Once the credentials were in someone else’s hands, Dragos observed them being tested, often within hours, then used in follow-on access attempts against the canary AWS accounts. In total, the bait keys were used for tens of thousands of AWS API calls. The post also says one adversary appeared to have compromised two of the providers at the same time, because secrets sent to those two providers were used together, almost immediately, from a single rented server.

    Reseller logging did not require an exploit

    The second finding, Dragos writes, did not require exploiting a vulnerability or bypassing platform controls. Multiple resellers operating through an AI inference marketplace were able to view and retain customer prompts using standard logging. In the researchers’ account, a seller could list a model at a low price, enable request logging, and wait.

    The post describes this as a routing problem as much as a logging problem. AI routers offer a single API endpoint that forwards requests to a pool of models, sometimes at different hosts, with unified billing. Every hop can see the full prompt and response. In some setups, a request may pass through more than one router before it reaches the server that runs the model. Dragos also says it is common for the lowest-price seller in an open marketplace to win the request.

    In one example, two prompts sent through a single seller’s offer led, about a day and a half later, to both secret URLs being opened and login guesses against a canary page. Prompts for the same model that went through a different seller were not touched. Dragos concludes that the leak followed the seller, not the marketplace as a whole. In another case, a router that was not advertised as part of the marketplace was quietly forwarding some traffic there.

    Privacy settings and retention promises

    The third finding did not involve stolen credentials. Secret URLs in the prompts were later fetched, repeatedly, and git repositories referenced in those prompts were targeted for cloning — often, Dragos says, by organizations that were never supposed to see the prompt, and sometimes in conflict with the privacy settings on the request.

    The clearest example in the post involves a Chinese AI lab. That lab’s systems fetched the secret URLs from more than half of the prompts Dragos sent to its models between July and October, including roughly two-thirds of the requests marked as “no data collection” or ZDR. The scraper typically appeared two or three days after the request. The same scraper also appeared after prompts sent to several U.S. hosting providers that resell the lab’s models. In Dragos’s words, a customer who picked a U.S. provider specifically to keep data in the United States did not get what they paid for.

    The post also flags issues at large U.S. providers. One continued to crawl a secret URL more than two months after it was used in a single inference call to that provider’s search endpoint. Another ran a scraper that returned for the URLs days after each request, regardless of which router was used to reach it.

    The strangest incident, Dragos says, involved company chat. Twenty-four days after a prompt was sent to one Chinese provider, the secret link from that prompt was posted in a Feishu (Lark) chat channel. Within two minutes, the link was opened from three desktops and three phones in Beijing and Shanghai. Dragos’s disclosure to the provider included only the first eight characters of that link, so whoever shared it was reading from a stored copy of the full prompt — more than three weeks after the request had been sent to a U.S.-based provider with the ZDR flag set.

    What Dragos did, and what it recommends

    Dragos says it stored evidence for each incident and notified providers. For the mid-July case, the provider first treated the report as a misconfiguration, then as a partner data-processing issue, before finding evidence that internal credentials had also been compromised. That provider had still not publicly disclosed the incident when the post was published, and stolen data continued to lead to attempted logins as of late October. For the late-October intrusions, Dragos reached out after confirming third-party access; one of those providers had already published a preliminary security notice. For less severe data-sharing issues, notifications started in early November. The team said it was then working with about a dozen providers.

    Reactions were mixed. One router that had been forwarding traffic to the marketplace told Dragos it had stopped. Several providers started their own investigations. Stolen keys and secret URLs were still being accessed, in some cases months after the original incident.

    The recommended mitigations in the post are operational, not a call to abandon hosted models:

    • Host sensitive workloads yourself if you can afford it. Dragos calls this the only option where nobody else reads the prompts, and it gives hardware price examples (a Mac Studio with 512 GB of memory starting at $22,299, an NVIDIA DGX Spark at $6,950, a DGX Station starting at about $105,000, and an eight-GPU B300 server listing for $570,000 or more). The post says this only makes sense for most teams on the most sensitive work.
    • Go directly to a first-tier provider, with one company, one contract, and one privacy policy. This can still leak through data-sharing agreements or a breach at that provider, but Dragos calls it the smallest exposure you can buy.
    • Use routers with care. Pick upstream providers yourself instead of accepting a default pool, and verify that privacy settings actually change where requests go. The post advises staying away from reseller marketplaces.
    • Seed your own canaries in the context that agents and pipelines already send, and treat any use of those secrets as an incident. A free Canarytoken from Thinkst is the starting point Dragos names.
    • Enable secret filtering when a provider offers it, with the caveat that it is not a panacea because much sensitive prompt data is not shaped like an API key.

    The same week, questions about how AI systems retain and use chat data have also been in the news for consumer products, including Google’s updated Gemini history and training controls. Provider-side incidents and policy disputes are not new to this site either; see the earlier coverage of California’s lawsuit against OpenAI over cybersecurity incidents and OpenAI’s GPT-6 Cyber model. Those are separate stories. The Dragos post is specifically about what happened to inference prompts after they left the customer.

    Key facts

    ItemWhat official sources say
    DateNovember 19, 2026, on the official Dragos blog.
    AuthorHD Moore, chief research officer at Dragos.
    ScopeApproximately two million prompts containing unique URLs and canary credentials, sent to widely used AI services from July through November 2026.
    Finding 1Evidence that data from at least three AI providers was exposed. Seeded prompts were later used against Dragos test systems and canary AWS accounts. The providers are not named in the post.
    Finding 2Marketplace resellers could view and retain customer prompts through standard logging, without exploiting a vulnerability or bypassing platform controls.
    Finding 3Multiple cases in which privacy settings or published policies appeared not to be honored, including ZDR-marked requests and prompts sent via U.S. resellers.
    NotificationsDragos notified providers. One late-October provider had published a preliminary security notice. The mid-July provider had not publicly disclosed the incident as of the post. The team was working with about a dozen providers.
    AdviceDo not assume a privacy flag binds every hop. Prefer direct first-tier providers or self-hosting for the most sensitive work, be careful with routers and marketplaces, and seed canaries in prompts you already send.

    FAQ

    What did the Dragos canary study find?

    The November 19, 2026 Dragos post reports three findings: evidence of compromises at at least three AI providers, marketplace resellers that could view and retain customer prompts through ordinary logging, and multiple failures to honor privacy settings or published retention policies.

    How many prompts did Dragos send, and over what period?

    Dragos says it submitted approximately two million prompts containing unique URLs and canary-token credentials to widely used AI services over five months, from July through November 2026.

    Which AI providers were breached?

    The official post does not name the three providers. It says none of them knew about the breach until Dragos notified them, that the earliest incident dates to mid-July, and that one of the late-October providers had published a preliminary security notice. This article does not add names that Dragos did not publish.

    Does zero data retention protect prompts in this study?

    Not reliably, according to Dragos. In two of the provider compromises, some stolen requests had been marked as ZDR. In the retention findings, a Chinese lab’s systems fetched secret URLs from roughly two-thirds of the “no data collection” or ZDR requests sent to its models, and a secret link later appeared in a company chat more than three weeks after a ZDR-flagged request to a U.S.-based provider. Dragos’s conclusion is that a privacy flag only binds the first hop, and it does not help if an attacker is already inside the provider.

    How did marketplace resellers get access to prompts?

    Dragos says they did not need to exploit a vulnerability. Resellers operating through an inference marketplace could view and retain customer prompts using standard logging after winning traffic, often by listing a model at a low price. Every hop in a router chain can see the full prompt and response.

    Did Dragos notify the providers?

    Yes. The post says Dragos notified the compromised providers, began notifying others about data-sharing issues in early November, and was working with about a dozen providers at the time of publication. One late-October provider had published a preliminary security notice. The mid-July provider had not made a public disclosure when the post appeared.

    What does Dragos recommend if you use AI APIs?

    The post recommends self-hosting only the most sensitive work if you can afford it, going directly to a first-tier provider when you use a hosted API, choosing router upstreams yourself and avoiding reseller marketplaces, seeding canary secrets in prompts you already send, and turning on secret filtering where a provider offers it, with the understanding that filtering is incomplete.

    Should teams stop using hosted AI services?

    Dragos says no. The post’s point is that where prompts go is a decision, and that if you do not make it, the lowest price will make it for you. The recommended path is to narrow exposure and to detect leaks, not to abandon hosted models altogether.