How Agents Decide Which Links to Follow
A typical e-commerce homepage might have 80 or more links. Navigation menus, footer links, promotional banners, product cards, social media icons, legal pages. A human visitor ignores most of them and heads straight for what they need. AI agents have to make the same decision, but without years of web browsing intuition.
The Selection Problem
An agent tasked with "find the return policy for this store" lands on the homepage and sees every link on the page. It cannot click all of them. Each click costs time, tokens, and potentially takes the agent further from its goal. The agent needs to rank these links by relevance and follow only the most promising ones.
This is fundamentally a language understanding problem. The agent reads each link's anchor text, considers its position on the page, evaluates the surrounding context, and makes a judgment call.
Anchor Text Is the Strongest Signal
The text inside a link is the single most important factor. An agent looking for a return policy will immediately rank "Returns & Refunds" higher than "About Us." This seems obvious, but a surprising number of sites use vague link text that gives agents nothing to work with.
"Click here" tells an agent nothing. "Learn more" is only slightly better. "Read more" repeated six times on a page with six product cards forces the agent to analyse the surrounding content for each link to figure out which one leads where. Descriptive anchor text like "View the full wool jumper collection" is unambiguous.
This directly parallels accessibility best practices. Screen readers face the same problem when a user tabs through links: a string of "click here" links is useless without context. What helps screen readers helps agents.
Position and Page Structure
Link position matters. Agents understand that primary navigation tends to sit at the top of the page or in a sidebar. Footer links typically contain legal, policy, and administrative pages. Links within the main content area are more likely to be contextually relevant to the page topic.
This understanding comes from the language model's training data. Having seen millions of web pages, the model has internalised common layout patterns. It knows that a mega menu at the top of the page is navigation, not content. It knows footer links labelled "Privacy Policy" and "Terms of Service" are standard administrative pages.
Semantic HTML reinforces these patterns. A link inside a <nav> element is treated as navigation. A link inside a <main> element is treated as content. A link inside a <footer> is treated as secondary. Without these structural hints, the agent relies more heavily on visual position, which is less reliable.
Surrounding Context
When anchor text is ambiguous, agents look at the surrounding content. A "View Details" link inside a card that mentions "organic cotton t-shirt, £25" gets associated with that product. The agent builds a mental model of what each link probably leads to by combining the link text with nearby headings, prices, descriptions, and images.
Schema.org markup provides an even stronger signal. If a product card contains structured data identifying it as a Product with a name, price, and URL, the agent can skip the guesswork entirely. It knows exactly what the link points to.
Depth Budget and Prioritisation
Agents do not explore indefinitely. They work with a depth budget: a limit on how many pages they will visit before completing or abandoning a task. A research agent comparing prices across five stores might allocate 3-4 page visits per store. If the agent cannot find what it needs within that budget, it moves on.
This creates a direct incentive for clear site architecture. If your return policy is three clicks deep (homepage, customer service page, then returns sub-page), an agent with a shallow depth budget might not reach it. If it is linked directly from the footer or main navigation, the agent finds it on the first page.
Flat site architectures, where important pages are reachable within one or two clicks from any page, work better for agents than deeply nested hierarchies. This also happens to improve SEO and human usability.
What You Can Do
Write descriptive anchor text. Not "click here" or "learn more," but text that tells the reader (human or machine) exactly where the link goes.
Put important pages in your main navigation or footer. Do not bury critical information like pricing, policies, or contact details behind multiple clicks.
Use semantic HTML for navigation structures. Wrap your nav links in <nav>, your main content in <main>, your footer in <footer>. These elements cost nothing to implement and give agents clear structural signals.
Add structured data where it matters. Product pages, service pages, and FAQ pages benefit the most from JSON-LD markup that makes content machine-readable without any guesswork.