This is for ecommerce, right? Does it apply to my SaaS?

**It is primarily for products sold online: DTC, ecommerce, and marketplaces.** That is where Product schema and Merchant feeds do the heavy lifting. **For SaaS:** the crawler-access and structured-data thinking still applies, but you would lean on SoftwareApplication schema and answer-engine visibility rather than a Merchant feed. If you sell a physical or clearly catalogued product, this tactic fits directly. If you sell a service with no catalogue, start with Answer Engine Optimization instead.

Why does the schema have to be server-side rendered?

**Because many crawlers read the HTML your server returns, before JavaScript runs.** If your Product schema is injected by client-side JavaScript (for example in a useEffect), an agent fetching the raw page may see an empty shell with no price, no availability, and no product data. Server-side rendering puts the schema in the initial HTML response, which is what the crawler actually reads. Verify with a `curl` request using the crawler's user agent.

Is opening robots.txt to AI crawlers risky?

**Allowing search and shopping crawlers is low risk and how you get recommended.** They fetch public pages you already publish. **The separate decision** is training crawlers like GPTBot, which use your content to train models. You can allow the shopping/search bots (OAI-SearchBot, PerplexityBot, Google-Extended) while still blocking training bots. Decide the two independently. Never block Googlebot, that breaks classic Search and Merchant listings.

How long until an agent actually recommends my product?

**Typically 4 to 8 weeks after the technical work is live.** **Why the lag:** crawlers have to re-fetch your pages, your Merchant feed has to be re-processed, and the agents build confidence in your data. Deploy the changes, wait a few days, run the step 5 test prompts, then re-test a couple of weeks later. Getting into ChatGPT and Perplexity answers usually happens before Gemini for most catalogues.

What is the single most common reason products get filtered out?

**A price or availability mismatch between your page, your JSON-LD, and your Merchant feed.** Agents cross-check these three sources. If they disagree, you look untrustworthy and get dropped. Keep all three driven from one source of truth so a price change updates everywhere at once. The second most common reason is missing product identifiers (gtin/mpn) in the feed.

Can I outsource this or does my team have to do it?

**It is partially outsourceable.** A developer or technical SEO can handle robots.txt, the server-rendered JSON-LD, and feed cleanup, that is the bulk of the work. **Keep in-house:** the source-of-truth for prices and availability, and the monthly agent testing, because those need someone who knows the catalogue. Do not hand the whole thing to a generalist agency that only does classic SEO; ask specifically about structured data and Merchant Center feed quality.

🛒 AI Shopping Agents

Agentic Commerce Readiness

AI shopping agents read structured data and feed quality, not page design. This tactic gets you three things: AI crawlers allowed in robots.txt, server-side-rendered Product JSON-LD schema, and a Google Merchant Center feed at 95%+ attribute completion, so agents can consider you when a buyer asks them what to purchase.

13 min read

Last updated: Jul 24, 2026

💬

"We stopped optimising for how our product pages looked and started optimising for how machines read them. Clean Product schema and a complete Merchant feed got us into ChatGPT's shopping answers. That traffic converts far better than anything else we run."

DTC founder, home goods

📅 Your First Month

Setup Warm-up Outreach

Week 1

Audit whether AI agents can see you

⏱ ~3 hours

Week 2

Open robots.txt to AI crawlers + ship SSR Product JSON-LD

⏱ ~1 day dev

Week 3

Push Merchant Center feed to 95%+ completion

⏱ ~1 day

Week 4

Test in ChatGPT/Perplexity and set up AI-referral tracking

⏱ ~half day

Week 1

Audit whether AI agents can see you

⏱ ~3 hours

Week 2

Open robots.txt to AI crawlers + ship SSR Product JSON-LD

⏱ ~1 day dev

Week 3

Push Merchant Center feed to 95%+ completion

⏱ ~1 day

Week 4

Test in ChatGPT/Perplexity and set up AI-referral tracking

⏱ ~half day

Is this for you?

Great fit if...

✓You sell physical products or a paid product online (DTC, ecommerce, marketplace)
✓You have a product catalogue with prices, availability, and images
✓You can get a developer to edit robots.txt and add server-rendered schema
✓Your buyers are starting to ask AI assistants for product recommendations
✓You already run Google Merchant Center or can set it up

Try something else if...

You sell a bespoke service with no catalogue or fixed pricing
You cannot render structured data server-side (schema injected only by client-side JS is often not read)
Your product is confidential or sold only through gated sales calls
You have no product images, specs, or stable pricing to expose
You want same-week results (agent indexing and trust build over weeks)

Optimise for AI answer engines more broadly →

Not the right fit?

If Agentic Commerce Readiness doesn't match your situation, consider these alternative tactics that achieve similar goals:

Content SEO

Create and optimize content to rank for valuable search keywords.

intermediate•low•long term

What to expect

4-8 weeks

Time to first agent recommendations

95%+

Target feed attribute completion

42%

Higher conversion from AI-referred traffic

Quick math: AI-referred traffic is small today but growing fast and converts better than most channels. This is a foundations tactic: you are removing the technical blockers that keep agents from considering you at all. Do it once, maintain it, and every future launch is machine-readable by default.

Before you fix anything, find out what an AI agent sees. Shopping agents (the automated assistants inside ChatGPT, Gemini, and Perplexity that answer buying questions) do not render your page like a human. They read the raw HTML, the structured data, and your product feed. If those are empty or blocked, you are invisible, no matter how good the page looks in a browser.

Why this matters now:

📈 AI-referred traffic to US retail sites rose 393% year on year in Q1 2026📈 That traffic converts 42% better than other channels

Run the three-minute visibility check

Do all four. Each one tells you a different blocker.

Open your product page, right-click, View Source. Search for the price and product name in the raw HTML. If they are missing, the data is injected by JavaScript and agents likely cannot read it
Visit yoursite.com/robots.txt and read it. Note any Disallow rules and any AI crawler names (OAI-SearchBot, GPTBot, PerplexityBot, Google-Extended)
Paste your product URL into the Google Rich Results Test and the Schema.org validator. Note whether valid Product schema is detected
Check whether you have a live Google Merchant Center account and, if so, the current feed status and disapproval count

Write down your baseline

You will compare against this in four weeks.

Is price + product name present in raw server HTML? (yes/no)
Are AI crawlers currently blocked in robots.txt? (yes/no/unclear)
Does valid Product JSON-LD validate today? (yes/no)
Current Merchant Center feed completion %
Current monthly sessions tagged as AI-referred (may be zero)

ℹ️ What is JSON-LD?

JSON-LD is a small block of structured data in your page HTML that describes the product in a format machines read directly: name, price, availability, rating, brand. It is invisible to humans and critical to agents. Schema.org defines the vocabulary.