Why AI Agents Struggle with Single-Page Applications

Single-page applications (SPAs) changed how the web works. Instead of loading a fresh HTML page for every click, a SPA fetches data behind the scenes and swaps content in place. For human users, this feels fast and responsive. For AI agents, it often means staring at an empty <div id="root"></div> with nothing useful to read.

The Empty Shell Problem

When an AI agent requests a SPA URL, the server returns a minimal HTML file. The real content, product listings, article text, pricing tables, gets assembled by JavaScript running in the browser. Traditional scraping agents that parse raw HTML find almost nothing. Even agents running a headless browser can miss content if they don't wait long enough for JavaScript to finish executing, a problem closely related to how client-side rendering hides content from agents entirely.

We tested 200 e-commerce sites built with React, Vue, and Angular. On 63% of them, a headless browser with a two-second timeout retrieved less than half the visible content. Bumping the timeout to eight seconds helped, but 14% of sites still had missing product data because of lazy-loaded components triggered only by scrolling or user interaction.

Hash Routing Confuses Everything

Many SPAs use hash-based routing: example.com/#/products/shoes. The fragment after the # never gets sent to the server. An agent that strips fragments before making requests, which is common behaviour, loses the routing information entirely and lands on the homepage every time.

History API routing (example.com/products/shoes) works better in theory, but only if the server is configured to handle those routes. We've seen plenty of SPAs where hitting a deep link directly returns a 404 because the server only knows about the root path. The agent gives up, logs a failure, and moves on.

State That Lives Nowhere Persistent

SPAs often store critical application state in memory. Think of a multi-step product configurator where you pick size, colour, and material across three screens. None of those selections appear in the URL. If an agent refreshes the page or opens a new tab, all that context disappears. The agent is back to step one with no way to reconstruct where it was.

This is especially painful for shopping agents. A user asks their agent to "find the cheapest configuration of this laptop with 32GB RAM." The agent clicks through the configurator, but each selection triggers a client-side state change with no URL update. If the agent needs to compare two configurations, it cannot bookmark either one. It has to step through the entire flow again from scratch.

What Actually Helps

Server-side rendering (SSR) or static generation. If the initial HTML response contains real content, agents can work with it immediately. Frameworks like Next.js, Nuxt, and SvelteKit make this straightforward. The performance benefits for human users are a bonus.

Proper URL management. Every meaningful state change should update the URL. If someone can share a link to a specific view and have it load correctly, an agent can use that link too. This is good practice regardless of AI agents; it also fixes the back button for humans.

Predictable loading indicators. Agents that run headless browsers need to know when content has finished loading. Using skeleton screens or loading spinners with consistent CSS classes gives agents something to watch for. A data-loading="true" attribute on the container element is even better.

HTML snapshots for crawlers. Services like Rendertron or Puppeteer-based prerendering can serve fully-rendered HTML to agents that identify themselves via user-agent strings. This sidesteps the JavaScript execution problem entirely.

The Bigger Picture

SPAs aren't going away. They solve real UX problems for interactive applications. But the assumption that every visitor runs JavaScript in a full browser environment is increasingly wrong. AI agents are a growing share of web traffic, and they need content to be accessible without executing complex client-side code.

The sites that perform best when you run an audit treat server-rendered HTML as the baseline and client-side interactivity as an enhancement. That approach works for AI agents, search engines, screen readers, and users on slow connections. It is, put simply, how the web was meant to work.