How Infinite Scroll Breaks AI Agent Browsing
Infinite scroll is everywhere. Social media feeds, product listings, search results, news sites. Instead of clicking "Next Page," you just keep scrolling and new content appears through dynamic content loading. Humans barely notice the loading. AI agents hit a wall.
The First Page Is All You Get
An AI agent loading a product listing page sees whatever renders in the initial viewport, typically 20 to 50 items. The rest of the catalogue loads only when a scroll event triggers a JavaScript function to fetch the next batch. Since most agents don't simulate scrolling, they work with that first batch and nothing else.
We ran a test across 50 e-commerce sites using infinite scroll. The median site had 1,200 products in a category but showed only 24 on the initial load. That means agents could see just 2% of available products. For a shopping agent asked to "find the cheapest wireless keyboard," checking only the first 24 out of hundreds of options is almost useless.
Scroll Position as Pagination
Traditional pagination gives agents clear, followable links: /products?page=2, /products?page=3. Each page has a stable URL. Agents can request pages in parallel, compare results, and systematically work through a catalogue.
Infinite scroll replaces all of that with scroll position. The only way to access item 500 is to scroll past items 1 through 499 first. There's no URL, no link, no way to jump ahead. Some implementations don't even keep earlier items in the DOM; they remove off-screen elements to save memory. An agent scrolling down might lose access to items it already passed.
The API Workaround (When It Works)
Clever agents can sometimes bypass infinite scroll by finding the underlying API. If the page loads products from /api/products?offset=0&limit=24, the agent can call that API directly with different offset values. This works when the API is publicly accessible and doesn't require authentication tokens that only the frontend JavaScript can generate.
About 30% of the infinite scroll implementations we've tested expose a clean, unauthenticated API endpoint. The rest use session tokens, CSRF tokens, or GraphQL queries that are difficult for agents to replicate without reverse-engineering the frontend code.
"Load More" Buttons: A Half Solution
Some sites use a "Load More" button instead of automatic scroll loading. This is better. The button exists in the DOM, and agents can click it. But there are still issues.
Each click loads another batch and typically removes the button until the new content finishes loading. The agent has to click, wait, confirm new content loaded, then click again. Loading 1,200 products in batches of 24 means 50 clicks with pauses between each. That's slow and fragile; any timeout or failed load breaks the chain.
More practically, many "Load More" implementations don't update the URL. If the agent's browser crashes on click 40, it starts over from the beginning.
What Works for Both Humans and Agents
Offer pagination alongside infinite scroll. This is the simplest fix. Keep infinite scroll as the default experience, but include rel="next" and rel="prev" links in the HTML <head>. Agents and search engines follow those links. Humans never see them. Everyone wins.
URL-based state. Update the URL as the user scrolls, something like /products#page=5. Better yet, use query parameters: /products?page=5. This gives agents a way to request specific pages directly and gives humans shareable links to specific positions in the list.
Expose a documented API. If your product data is accessible via a clean REST or GraphQL API, agents can fetch what they need without dealing with scroll mechanics at all. Public API documentation helps agents discover these endpoints.
Sitemaps with product URLs. A complete XML sitemap lets agents discover individual product pages without ever needing to browse category listings. This sidesteps the infinite scroll problem entirely.
The Real Cost
Sites using infinite scroll without any fallback are effectively hiding most of their content from AI agents. For retailers, that means AI shopping assistants can only recommend products from the first screenful of results. The product that best matches what the user wants might be sitting at position 300, completely invisible. Your competitors who offer pagination are getting that recommendation instead. Audit your site to see whether agents can actually browse your full catalogue.