Why it matters.
Vercel and MERJ analysed crawler traffic across Vercel’s network and reported on 17 December 2024 that “none of the major AI crawlers currently render JavaScript”. ChatGPT’s and Claude’s crawlers do fetch JavaScript files (11.50% and 23.84% of their requests) but do not run them. Google’s Gemini, which uses Googlebot’s infrastructure, and AppleBot were the exceptions.
So if your service descriptions, results or FAQ answers are drawn in by a script, the crawler reads an empty shell. The study is from 2024 and crawler behaviour can change, so test rather than assume.
The 10-minute self-check.
Pick your most important service page and:
- Open the page source (Ctrl+U or Cmd+Option+U), not the inspector, and search for a sentence from the middle of the page.
- Repeat for text inside tabs, accordions, carousels and “load more” sections.
- Or ask your developer to run curl on the URL and search the output for the same sentence.
Any sentence missing from the source is invisible to most AI crawlers.
Steps.
- List the pages that answer buyer questions: services, case studies, method, company facts.
- Check the raw HTML of each for its key passages.
- Where text is missing, ask your developer for server-side rendering, static generation or pre-rendering on those templates.
- Keep tabs and accordions if they help readers, but make sure their text is in the HTML when the page loads.
- Re-check after every redesign or theme change.
If a rebuild is the fix, that is our website development work.
What we saw on prolinkage.com.
How to measure it.
- Coverage: share of key passages present in raw HTML, by template.
- Release check: the same test after every site release.
- Outcome: whether engines describe your services correctly when asked by name, monthly.
Sources.
- Vercel, The rise of the AI crawler, 17 December 2024.
- Prolinkage crawler and markup check of prolinkage.com, 6 November 2026.