In the HTML vs. injected by JavaScript
What matters is where a piece of content lives at the moment the server responds. If it's in the initial HTML, a raw-HTML crawler can read it. If a browser has to run scripts to put it on the page, a crawler that doesn't render will miss it.
| Content source | In initial HTML? | Seen by raw-HTML crawler? |
|---|---|---|
| Server-side rendered (SSR) | Yes | Yes |
| Static generation (SSG) | Yes | Yes |
| Client fetch after load | No | No — appears only post-hydration |
| Rendered by client-side React/Vue | No | No — DOM built in the browser |
| Lazy-loaded on scroll or click | No | No — event never fires for a crawler |
Rule of thumb: if it survives with JavaScript disabled, it survives for crawlers too.
How to test your own page
You don't have to guess. Two checks tell you what a non-rendering crawler sees, and they take under a minute.
First, disable JavaScript in your browser (DevTools settings, or a toggle extension) and reload the page. Whatever disappears is content a raw-HTML crawler probably can't read.
Second, curl the URL — something like curl -sL https://yoursite.com/page — and read the raw markup that comes back. That's close to what a crawler fetches before any script would run. If the product name, description, price, or FAQ text isn't in that output, it isn't reliably retrievable.
Why server-side rendering is the safe default
Rendering behavior varies by crawler and shifts over time, so betting your visibility on a specific crawler executing your JavaScript is fragile. Server-side rendering or static generation removes the bet entirely: the content is in the response no matter who fetches it.
This is also cheaper for the crawler, which raises the odds the substance is captured on the first pass rather than skipped. The words you most want an assistant to quote — what you do, who you serve, what it costs — should never depend on a script running.
Behavior is not uniform across crawlers and can change; treat raw-HTML retrieval as the planning baseline.
Common questions
Does GPTBot render JavaScript like a browser?+
Not reliably. GPTBot largely fetches the raw HTML your server returns rather than running a full browser render, so content injected after client-side hydration is often missed. Behavior can change over time, so server-side rendering the important content is the safe default.
What about ClaudeBot, PerplexityBot, and CCBot?+
The same caution applies. These crawlers largely read raw HTML and don't reliably execute client-side JavaScript. Handling isn't identical across them and can shift, but planning around raw-HTML retrieval covers all of them at once.
How do I know if my content is JavaScript-dependent?+
Load the page with JavaScript disabled, or curl the URL and read the raw markup. If the product names, descriptions, prices, or FAQ text aren't present in that output, they only appear after hydration and a crawler probably can't see them.
keep reading
See what AI says about your brand right now — free, on a live engine.
Scan your domain