Good Bots, Bad Bots and AI Agents: How to Verify Agent Traffic Without Blocking Customers

Treat AI agents as a third category alongside humans and bad bots. Signed requests make that practical: ChatGPT agent and other operators sign their HTTP requests (RFC 9421 HTTP Message Signatures) and publish keys, the IETF's Web Bot Auth working group is standardizing the approach, and Visa's Trusted Agent Protocol uses it for agent checkout. Verify signatures at your CDN or edge, allow verified agents on browsing and cart paths, keep payment and account changes behind customer confirmation and normal fraud checks, and rate-limit or challenge unsigned automation. Do not rely on user agent strings, and do not block all automation by default.

For years the safe default was simple: anything that is not a browser operated by a person is suspicious. Credential stuffing, scalping, scraping and card testing made that a reasonable rule, and bot-protection products are good at enforcing it.

Agentic commerce breaks the rule. When a shopper asks an assistant to find a product, compare options and put it in a cart, the request reaching your store comes from automation acting with permission. If your defences treat it like a scraper, you lose the sale and never see it in analytics. The same applies to travel bookings, restaurant reservations, appointment scheduling and B2B reordering. Our agentic commerce guide covers how these purchases work end to end.

A user agent string is self-declared; any script can claim to be ChatGPT. IP allowlists are better for crawlers (OpenAI, Google and others publish their crawler ranges), but they are brittle for agents that run on shared cloud infrastructure, and they say nothing about which operator or product sent a specific request. Verifying claimed crawlers by IP or reverse DNS remains good practice, but for agents acting on behalf of people you need something stronger.

HTTP Message Signatures (RFC 9421) let a client sign selected parts of an HTTP request with a private key. The server verifies the signature with the matching public key. For bots and agents, the operator publishes its public keys in a directory at a well-known URL and adds headers that tell the server where to find them.

OpenAI documents this for ChatGPT agent: each request carries `Signature` and `Signature-Input` headers plus a `Signature-Agent` header set to `"https://chatgpt.com"`, and the keys are published at a well-known HTTP message signatures directory on chatgpt.com. A server verifies that the Signature-Agent value matches, fetches the key, and checks the signature as defined in RFC 9421.

The IETF chartered the Web Bot Auth (webbotauth) working group to standardize these methods for crawlers, archivers and AI agents, with milestones through 2026 for authentication techniques, conveying bot information and operational best practice. Several CDN and bot-management providers already verify signed agents for their customers, which means many merchants can enable verification as a setting rather than writing code.

Card networks have the same problem at checkout: was this purchase made by a legitimate agent acting for a real cardholder? Visa announced its Trusted Agent Protocol in October 2025, developed with Cloudflare and with feedback from payment providers and platforms including Adyen, Checkout.com, Stripe, Shopify, Worldpay and Microsoft. It lets approved agents pass signed information to merchants so they can tell legitimate agents from malicious bots, for both guest and logged-in checkout. Visa describes it as built on HTTP Message Signatures and aligned with Web Bot Auth.

Mastercard has a parallel programme (Agent Pay), and Google's Agent Payments Protocol (AP2) focuses on proving the user authorized a purchase. For merchants, the practical message is the same: these schemes are arriving through payment providers and CDNs, so ask yours what they support rather than building to a protocol directly.

Write the policy by path and action, not by bot name.

Signed agents are new. Not every operator signs requests yet, standards are still being finalized, and the long tail of smaller agents will take time to adopt them. Signatures also do not answer every question: an agent might be operated by a reputable company and still be misused by its user. Expect to run signature verification alongside behavioural bot detection for some time, and to adjust the policy as payment networks and platforms settle on their schemes.

For crawler-specific controls, see AI crawlers and robots.txt. For general store protection, see ecommerce security and ecommerce fraud detection.

The old rule (humans good, automation bad) no longer fits a world where customers delegate shopping and booking to assistants. Verify agents cryptographically where operators support it, allow them on low-risk paths, keep customers in control of payment and account changes, and keep blocking the abuse you were blocking before. That keeps the door open for AI-assisted customers without opening it to everyone.