Research

We benchmarked seven open-weights embedding models on a corpus of 135 live web pages and 405 stratified queries, using an LLM judge on a 0/1/2 relevance rubric. The chunker was held constant and only the embedder was varied. Total: 28,350 judgements. The top-to-bottom spread in retrieval quality was 0.32 on mean judge — larger than the spread between any two chunkers in our previous study. Two Pareto frontiers emerged when cost was introduced, and they contained different models. Two mid-tier models produced identical aggregate scores to ten decimal places from observably different retrieval behaviours.

Chunking — how you split a source document into retrievable units — is a core configuration choice in any retrieval-augmented generation pipeline. It sets the unit of retrieval and therefore the unit of context available to the reader model. Despite its centrality, we could not find a published benchmark that compared a broad set of chunking strategies across multiple content types and query-specificity levels in one place. We built one.

We ran 3,348 AI navigation tasks across 269 websites and 77 distinct task types to understand how AI agents actually navigate the web. The findings challenge conventional wisdom about AI visibility: search-first approaches either succeed instantly (90%) or fail badly (27%), 91% of successful navigation completes within two clicks, and position in the DOM matters more than semantic relevance. These findings suggest that SEO success remains essential for AI discoverability, but it's no longer sufficient on its own.