Build it right once

Sanity and Next.js are the stack we build on and sell, so read our enthusiasm for them accordingly. Every figure below is attributed, and nearly all of them link straight to the source. Vendor-funded studies get flagged where they're cited. Where the evidence is contested, we've said so instead of picking the flattering number. Go and check the work yourself; we would.

The ground moved

For years, being found online meant one thing: rank on Google, get the click. That deal is coming apart. In the first four months of 2026, 68% of US Google searches ended without a click to any website (SparkToro).

Pew tracked 900 US adults' Google searches through March 2025. When the results page showed an AI summary, people clicked through to a website 8% of the time. When it didn't, they clicked 15% of the time (Pew). That's the AI summary roughly halving the chance a search sends anyone to a website. The comparison isn't clean, since the two sets of searches aren't like for like, and Google disputes the read. So take it as the direction of the drop, not a measured cause.

Some publishers went to court anyway. Penske Media and Chegg each filed antitrust suits against Google in 2025, both pointing to falling search referrals among their claims.

At the same time, a new channel showed up. People now click through from ChatGPT, Perplexity, and Claude. It's real and it's growing. It's also small: one peer-reviewed study of 973 stores put ChatGPT referrals at under 0.2% of visits, and whether that traffic converts well is a live fight between credible people on both sides (Marketing Science). We'll let them get on with it.

So that's the state of play. Classic search still sends most of the traffic, but it's shrinking and getting harder to win. AI answers are a promising new channel that's still unproven. And underneath both sits the thing that makes either one work: a fast, crawlable, well-built site. That part hasn't changed at all. It just has more judges now.

The reflex is to treat that as a brand-new problem with a brand-new product bolted on. Buy an AEO tool. Generate an llms.txt file. Sprinkle on some schema. Call it a strategy. We think that's backwards, and we can show you why.

One foundation, four judges

Google's crawler, ChatGPT's crawler, a screen reader, a person on a slow phone: every audience that sizes up your site ends up reading the same thing, your page's structure once it has rendered. What separates them is how much work each one will do to get there.

Google's crawler runs your JavaScript, slowly and unpredictably. A screen reader builds its picture from your HTML and your headings. A human just wants the page to load fast and not jump around while they're trying to tap a button. And the crawlers behind ChatGPT, Claude, and Perplexity don't execute JavaScript at all.

Vercel's crawler study, run over a single month in late 2024, logged 569 million GPTBot fetches and 370 million from Claude's crawler, and found neither executed any JavaScript (Vercel). The volumes have grown a lot since. The finding hasn't moved: as of mid-2026 no ChatGPT, Claude or Perplexity crawler renders JavaScript.

If the main content on your page only appears after JavaScript runs, those bots don't see it late. They don't see it at all.

One fair exception, since we'd rather you hear it from us: Googlebot feeds Google's AI Overviews, and Googlebot does render JavaScript. That particular surface can reach JS-only content, eventually, through the same slow and unpredictable queue as normal Google indexing. It's not a safe thing to lean on, and it doesn't help you with ChatGPT, Claude, or Perplexity at all.

You don't have to take Vercel's word for any of this. Here's our own crawl ledger, rebuilt every day from our server logs.

Now look at what fixes it: shipping complete HTML in the first response instead of assembling it in the browser. Server-side rendering does that. So does static generation, and so does ISR. The dividing line isn't server versus browser, it's pre-rendered versus client-rendered. Make that one call and your page is visible to AI crawlers, cheaper for humans to load, and legible to screen readers. Nobody ran three projects to get there. Someone picked the right rendering strategy once.

Be precise about what it buys you, though, because this is where people oversell it. Pre-rendering makes your content present. It takes semantic HTML and real labels to make it accessible, and a lean bundle to make it fast. Being readable makes you eligible for citations; it doesn't earn them. Build the foundation properly (pre-rendered, semantic, structured, accessible, fast) and it pays off in search, AI answers, speed scores, and accessibility law all at once. Build it right once.

What holds up and what doesn't

Because the foundation is shared, most of the "do this for AI" checklist turns out to be stuff you should be doing anyway. But plenty of the popular advice is simply wrong, and we'd rather tell you than sell you.

Holds up under testing

• Pre-rendered HTML, for the reason above. It's the cheapest thing on this list, because you decide it once when you pick a rendering strategy and every judge downstream inherits the answer.
• Real speed, measured on real users, not a lab score. It's the one signal all four judges score directly. A clean lab result can hide a bad real one, which is the whole reason field data exists.
• Putting your answer first. In a hand-mapped sample of 100 AI Overviews, CXL found 55% of citations came from the top 30% of the page (CXL). Small study, but bigger citation analyses point the same way.
• Concrete evidence in your writing. In the 2023 GEO paper (peer reviewed at KDD 2024), adding quotations lifted visibility about 41%, statistics about 33%, and cited sources about 28%, while keyword stuffing made things slightly worse (Aggarwal et al.). Worth knowing those tests ran against GPT-3.5, so the exact percentages are dated.
• Genuine accessibility, which is now a legal requirement in the EU, not a nice-to-have. The markup that makes a page work for a screen reader is the same markup a crawler reads, so the work lands twice. The law is below.

Here's what the GEO paper measured:

The same numbers as a table, since assistants read those better than charts:

The row pointing the wrong way is the one to remember. Stuffing keywords left the page worse off than leaving it alone.

Doesn't hold up

• Schema markup as a way to push already-cited pages higher. Ahrefs tracked 1,885 pages that added JSON-LD against 4,000 matched controls, all of them already earning 100+ AI Overview citations, and found no uplift plus a small drop in AI Overviews (Ahrefs). They're careful to say schema may still help pages AI systems haven't found yet, so this isn't an argument for ripping it out. It's still doing its job for ordinary Google rich results.
• llms.txt as a growth lever. The file itself is a cheap courtesy index that real agents read when they're handed your domain, so serve one, because it costs you one route handler. What it doesn't do is drive discovery, ranking, or citations, and Google Search has said it won't use the file (Search Engine Land). Expect nothing from it.
• Serving Markdown to bots as a citation play. It saves real bandwidth, and a randomized A/B test across 381 pages found no statistically significant lift in crawler traffic (Profound, an AEO vendor, testing on its own analytics product and publishing a result that cuts against its own pitch). Three weeks rules out a big effect, not any effect, and nobody has shown Markdown earns you citations either way.

The boring, durable work pays off. The clever new tricks mostly don't.

One model, many readers

We just said serving Markdown to bots won't buy you citations. So why does every block in our Sanity starter ship a Markdown serializer? Because two different things go by that name, and only one of them is a trick. The trick is generating a Markdown export and hoping the answer engines reward you for it (see above: they don't).

The other version is plumbing, and it's easier to show than describe. Everything below is real code from our Sanity starter, pinned to the commit we're writing against, so you can open any of it and check us.

It starts with where the code lives. One block, one folder, and every output it owes anyone:

faq-accordion/
├── faq-accordion.schema.ts          # the shape, defined once in Sanity
├── faq-accordion.groq.ts            # the query that fetches it
├── index.tsx                        # HTML, for the person
├── json-ld.ts                       # FAQPage schema, for Google
├── markdown.ts                      # structured Markdown, for the agent
├── faq-accordion.test.tsx
├── json-ld.test.tsx
├── faq-accordion-markdown.test.tsx
└── thumbnail.png

That's the whole folder, all nine files. You define the shape once and query it once. Everything else is a projection of it, and each projection has a test holding it honest.

Here's the Markdown one, with its import block cut and nothing else:

export function faqAccordionToMarkdown(
  block: MarkdownBlock,
  options: MarkdownOptions
): string {
  const faqs = (block.faqs ?? [])
    .filter((faq) => faq?.title)
    .map((faq) =>
      joinSections([
        headingToMarkdown(faq.title, 3),
        portableTextToMarkdown(faq.richText, options),
      ])
    );

  const link = block.link;
  const linkLabel = (link?.description || link?.title || "").trim();
  const linkMarkdown = linkLabel ? mdLink(linkLabel, link?.href) : "";

  const subtitle = (block.subtitle ?? "").trim();

  return joinSections([
    eyebrowToMarkdown(block.eyebrow),
    headingToMarkdown(block.title, 2),
    subtitle ? escapeMarkdown(subtitle) : "",
    ...faqs,
    linkMarkdown,
  ]);
}

Full source, if you'd rather read it in place.

The block's title becomes an h2. Each question becomes an h3. Each answer runs through the Portable Text serializer, which is what keeps the bullets and the links alive. The eyebrow, the subtitle and the trailing link come along too, because they're fields on the block and an agent should get the same ones a person gets. Nothing scrapes the rendered page, and nothing gets flattened.

So an agent asking for that block gets this:

## Frequently asked questions

### Do you support multiple content types?

Yes, pages, posts, and product blocks all serialize through the same pipeline.

### How is this exposed to AI agents?

Every block ships three projections automatically:

- HTML for the rendered page
- JSON-LD for search engines
- Markdown for agents

Full spec: [schema.org/FAQPage](https://schema.org/FAQPage)

Every question is a real heading. Every answer keeps its list and its link. An agent reading that knows which answer belongs to which question, because the structure never left.

Skip the serializer and here's what you hand over instead:

{"title":"Frequently asked questions","faqs":[{"title":"Do you support multiple
content types?","richText":[{"_type":"block","style":"normal","markDefs":[],
"children":[{"_type":"span","text":"Yes, pages, posts, and product blocks all
serialize through the sam...

That's your CMS's internal plumbing. The meaning is in there somewhere. You're just making a language model dig for it. Your content is already structured, or it is if you've modeled it properly (that's what the Sanity AEO/SEO defaults we ship are for), so let each reader take the shape it can parse.

A whole page is the same idea one level up. Walk the blocks and ask each one:

function blockToMarkdown(
  block: MarkdownBlock,
  options: MarkdownOptions
): string {
  switch (block?._type) {
    case "hero":
      return heroToMarkdown(block, options);
    case "richTextBlock":
      return richTextBlockToMarkdown(block, options);
    case "faqAccordion":
      return faqAccordionToMarkdown(block, options);
    // four more cases, one per block type
    default:
      return "";
  }
}

export function pageBuilderToMarkdown(
  blocks: MarkdownBlock[] | null | undefined,
  options: MarkdownOptions = {}
): string {
  if (!Array.isArray(blocks)) {
    return "";
  }

  return blocks
    .map((block) => blockToMarkdown(block, options))
    .filter((markdown) => markdown.trim())
    .join("\n\n");
}

Four of the seven case arms are cut there, along with the imports. Full source.

Look at the default. An unknown block type returns an empty string, so a new block is invisible to agents until someone writes its markdown.ts. That's a deliberate cost, not an oversight: the alternative is guessing at a shape nobody defined, and a bad guess is worse than a gap.

Then serve it from the address the humans already use:

/**
 * Content negotiation for Markdown: a `.md` URL or `Accept: text/markdown` is
 * rewritten to the `/api/markdown` route handler; everything else passes through.
 * The Markdown route sets `Vary: Accept` so a shared cache never serves Markdown
 * to a browser; App Router HTML pages can't carry it (Next owns that header), so
 * the `.md` suffix is the cache-safe surface.
 */
export function proxy(request: NextRequest): NextResponse {
  if (request.method !== "GET" && request.method !== "HEAD") {
    return NextResponse.next();
  }

  const { pathname } = request.nextUrl;
  const hasMdSuffix = pathname.endsWith(".md");
  const wantsMarkdown =
    hasMdSuffix || prefersMarkdown(request.headers.get("accept") ?? "");

  if (!wantsMarkdown) {
    return NextResponse.next();
  }

  const rawPath = hasMdSuffix ? pathname.slice(0, -3) : pathname;
  const contentPath = normalizeMarkdownPath(rawPath);

  const url = request.nextUrl.clone();
  url.pathname = "/api/markdown";
  url.search = "";
  url.searchParams.set("path", contentPath);
  return NextResponse.rewrite(url);
}

export const config = {
  matcher: ["/((?!api/|_next/|favicon.ico|robots.txt|sitemap.xml|llms\\.txt).*)"],
};

The docblock there is verbatim and it's about to do some work. Cut from the body: the guard that skips asset files when negotiating by header, and the request header that carries the path through the rewrite. It's called proxy.ts because Next.js 16 renamed middleware.ts; on 15 and earlier this is your middleware. Full source.

That's content negotiation. It's about as old as HTTP, and you're pointing it at a new kind of reader. The full Next.js implementation is a couple of rewrites and a route handler. Do it for bandwidth and fidelity, not for a ranking bump that isn't coming.

Why there's a .md suffix at all

Now read that docblock again, because it's the part we'd most want challenged.

We key off the Accept header and a .md suffix, not a list of bot user-agents, because that list is stale the week you write it. The header is the elegant half, and on its own it's what our Next.js guide tells you to use. The suffix is there for a duller reason. In the App Router, Next owns the Vary header on an HTML response, so the page can't advertise that it varies on Accept. Put a shared cache in front of that and it can hand a browser the Markdown it cached for an agent.

The route handler can set Vary: Accept because we own that response. The page can't. So the suffix is the cache-safe surface. The Markdown twin ships x-robots-tag: noindex, nofollow plus a content-location header pointing back at the canonical page, which is what keeps it out of search (route handler).

The cost is real, so here it is up front. Bing's Fabrice Canel made the case against in one line: "really want to double crawl load? We'll crawl anyway to check similarity." Every .md URL is a second URL per page, our own guide tells you not to advertise them, and the starter's llms.txt lists them anyway. Yes, those two things contradict each other. We picked cache correctness over crawl budget. If you're running a thousand pages and Bing is the crawler you care about, pick the other way.

None of this is an AI strategy, and we won't dress it up as one. One content model, three projections: HTML for the person, JSON-LD for Google, Markdown for the agent. The fields are the same ones every time; nothing gets scraped from anything else. It's the foundation from earlier, emitting one more honest view of itself.

What this looks like in practice

Take brightonSEO, the conference, which is about as SEO-literate an audience as exists. They moved off WordPress onto Sanity and Next.js. The second site they needed, for their first US conference, went up in two days (Sanity's own case study).

Kelvin credits headless in general. We'll be more specific: the second site took two days because there was nothing left to build, only content to point at. The first build was pre-rendered and structured from day one, so the foundation was already finished. Two days is how long it takes when the only job left is pointing content at it. And it's the same foundation the crawlers, the screen readers, and the people on slow phones are all reading.

Migrations are where it breaks

We're not going to hand you a tidy success story and stop there. Replatforming carries real risk. Dan Taylor at SALT.agency has been tracking this: his June 2026 study of 1,052 domain migrations found a median recovery time of 304 days for organic traffic, with a mean of 489 dragged up by a long tail. About 14% of sites were still short of their old numbers three years on. The data is crowdsourced from SEO practitioners, not randomly sampled, and it measured domain moves, not CMS replatforms. So it tells you how bad a botched move can get, not what your own odds are.

We once inherited a site that had lost more than half its traffic after a headless migration. The cause was a single robots.txt rule carried over from the old stack, blocking the very scripts the new pages needed to render. Every page was technically live and effectively blank.

None of that is a reason to avoid replatforming. It's a reason to do it deliberately, with someone who has mapped every redirect, audited the crawl rules, and modeled the content before cutover, not after. Ours is written down as a pre-launch migration checklist, and it exists because the failure above is boring and preventable.

The gap between the 272% ROI case (Forrester, commissioned by Sanity) and the traffic-halving disaster is almost entirely execution.

Accessibility stopped being optional

One more shift is worth naming, because it's now law. The European Accessibility Act took effect on 28 June 2025 (EU Commission). If you put products or services on the EU market, accessibility is a legal requirement, and it keys off where you sell, not where you're incorporated.

The carve-out is narrower than it looks, and this is the part most write-ups get wrong. A microenterprise means under 10 staff and under €2M turnover or balance sheet, and even then only your services are exempt. A microenterprise that makes or sells a covered product still complies.

Each country sets its own penalties and the range is enormous. Spain's very serious infringements run up to €1M. For the rest of the EU, the numbers you'll find online contradict each other, so we won't quote a figure we haven't checked against the actual law. This is general information, not legal advice, so check your actual obligations for the markets you sell into.

A warning that catches people out: an accessibility overlay widget isn't a reliable substitute for accessible code. In 2025, 24.9% of US accessibility lawsuits named sites that already had a widget installed (EcomBack's lawsuit tracker). That suggests an overlay on its own hasn't reliably headed off claims. A script bolted on at runtime can't invent a heading order that was never authored, and it can't label a control your build left unlabeled.

The good news is the convergence, again. The same semantic HTML a screen reader depends on is the structure an AI crawler reads and Google rewards. Do it properly and you're covered on three fronts with one piece of work.

No spam, only good stuff

More posts that check the receipts

Only god knows why anybody would purposefully subscribe themselves to a newsletter that moans about development. These poor souls did though

Email address

Subscribe

Knowing whether you're winning

Judge your speed on real-user field data (Google's CrUX, Search Console), not a one-off Lighthouse run. CrUX grades you at the 75th percentile, so by definition a quarter of your visits are worse than that number, and a single lab run doesn't capture that spread at all (web.dev).

Your AI traffic won't show up cleanly in analytics yet. Build a custom channel for the AI referrers (our PostHog setup is where ours lives), and remember your server logs are the only place those crawlers actually appear. That's the data behind the scoreboard further up this page, and it costs nothing to start collecting today.

Watch the order it arrives in, too. Crawling comes first, citations second, clicks last. A rise in AI crawler hits with no referrals behind it yet is what you should expect, not a failed experiment. Reading it as failure is how teams talk themselves out of the work a year before it pays.

How we do this

Everything above is the order we spend a client's budget in. Rendering strategy first, because every judge downstream inherits that decision and it's the one that hurts to revisit. Content model second, because the projections are only as good as the fields behind them. Semantics and speed third, which is the work that turns up in a field report and in a legal letter. AEO line items last, and some of them we talk clients out of.

Two builds where that order held. We picked up Mario Testino's site in a state of crisis: a botched migration had left 160+ redirects missing and Sanity API overages running at 200%. The fix was unglamorous, back to vanilla Sanity, rebuild the model, recover every redirect, clean up the navigation so crawlers could read it. Tray.ai was the rendering call at scale: 500,000 pages onto Vercel in six weeks, hybrid static and server rendering across 25 page types, builds down from a full workday to two minutes.

Client build

Mario Testino

From Sanity overages to instantaneous publishing, we brought Mario Testino into the fast lane, and did it in style.

View case study

Client build

Tray.ai

Migrating hundreds of thousands of pages, re-platforming and extending for the leading composable AI integration platform

View case study

Where this leaves you

There's no silver bullet here, and anyone selling you one is worth ignoring. The fundamentals became the strategy while everyone was out shopping for tools. A site that's fast, pre-rendered, structured, and accessible competes in Google, shows up in AI answers, and leaves you on far firmer ground if you're challenged, because all of those judges are reading the same thing.

If you want a shopping list, it's short and it's unfashionable. Fund the rendering strategy, the content model, the semantics, and the speed. Fund the migration properly if you're doing one. Then spend what's left on writing things worth quoting. That's the half no vendor can sell you, and it's where the evidence keeps pointing.

The scoreboard further up this page rebuilds from our server logs every day, so when the picture changes, we'll update the post to match. Until then, the work is the list above.