We tried to archive a competitor's website. The Internet Archive recorded a 404 five times, while the page loaded fine for everyone else.

This is the third post in our research program, and it is the most procedural. The first two were about how AI assistants build answers and who they name. This one is about a narrower question that turns out to sit underneath both: can the record of a webpage be independently verified? When an AI assistant, a journalist, or a buyer wants to know what a page said last month, the Internet Archive is usually where they look. So we went looking, for one specific site in our own category, and we could not find it. What follows is exactly what we observed, on one day, with the commands anyone can rerun.

The site is listerleo.com, a competitor product in the auction-cataloging space (ListerLeo is operated by Vendidit; its domain was registered 2026-03-13). We name it because vendors are the subject of this program and because the observation is only useful if it is checkable against a real URL. Everything below is a dated observation. We are not making a claim about anyone's intent, and we ask you not to read one in. A configuration that produces these results can arise from a firewall rule, a bot-management vendor's default, a CDN misconfiguration, or a deliberate setting, and we cannot tell which from the outside. What we can do is record what happened and show our work.

Observation one: no snapshots exist, for the domain's entire lifetime

The Wayback Machine exposes a query interface, the CDX server, that lists every capture it holds for a URL or domain. We queried it for the whole listerleo.com domain on 2026-08-08 at 19:20 UTC. It returned HTTP 200 with an empty JSON array: []. Zero rows. Not an error, not a block — an affirmative, successful answer that the Archive holds no snapshot of any page on that domain across the roughly five months since the domain was registered. For comparison, the same query against an established site in the same category returns hundreds of rows. We captured this reading first, before doing anything else, so that it stands as a clean before-picture uncontaminated by our own later requests.

Observation two: five save attempts, five archived 404s

Next we asked the Archive to save the site, using its public Save Page Now tool, one URL at a time. Five of the six requests completed. Each one produced a snapshot — and every snapshot recorded an origin 404 (page not found) for a URL that, as observation three shows, was serving a normal page at that same hour. The five completed captures, at their real archive addresses and timestamps:

Five of five completed captures recorded a 404. The sixth request hit a rate limit (HTTP 429) on three tries and never completed; we disclose it as an incomplete cell rather than round it into the pattern. And a note on what these captures did not change: after the saves, the domain-level CDX query still returned [] — the individual 404 snapshots exist at their direct addresses but do not surface in the default index — so the before-picture in observation one remains the pristine zero-archive reading.

Observation three: the same URLs served a normal page to everyone we asked

At the same hour, from an ordinary residential internet connection, every one of those URLs returned HTTP 200 with real page content. We checked under four different user agents: a default command-line client, a normal Chrome browser identity, the OpenAI crawler's identity (GPTBot), and — the one that matters most here — the Internet Archive's own crawler identity, archive.org_bot. All four got 200. So from where we sat, the page existed and was served to the Archive's own stated crawler identity, yet the Archive's saving infrastructure recorded it as missing. We also checked the site's robots.txt, the file that tells crawlers what they may access: it permits everything (its effective content is 'User-agent: *' with an empty Disallow). Nothing in the site's published rules asks the Archive to stay away.

The site's own rules invite every crawler in. Its own crawler identity gets a normal page from our connection. The Archive, trying to save it, recorded a 404 five times out of five.

The plain description of these three observations together: the Internet Archive cannot capture this site as it is served to the Archive. Whether that is by accident or design, and by whose hand, we do not know and do not assert. The observation stands on its own, and it has a consequence regardless of cause.

Why this matters for citation trust

Our whole program rests on records that outlive the moment they were made. When we report that a page was published before it was first cited, or that a competitor's comparison page framed a rival a certain way, the value of that report is that a third party can go check it against an independent archive. A site with no archival footprint removes that check. Its operator can change any page, at any time, and there is no neutral record of what it said before. In a category where AI assistants increasingly retrieve and quote comparison pages as if they were reference material — we documented exactly that pattern in our first two posts — a page that cannot be archived is a page whose past cannot be audited. That is a property worth measuring and disclosing, in the same spirit as everything else we publish: not as an accusation, but as a fact about the record.

How to verify this yourself

Every observation above is reproducible with public tools. Query the archive index for the domain: curl "https://web.archive.org/cdx/search/cdx?url=listerleo.com&matchType=domain&output=json" — an empty result means no snapshots. Ask the Archive to save a page by visiting https://web.archive.org/save/ and entering a URL, then read the status the resulting snapshot records. Check the origin yourself from your own connection under any user agent, for example: curl -A "Mozilla/5.0 (compatible; archive.org_bot +http://archive.org/details/archive.org_bot)" -I https://listerleo.com/ and compare the status code to what the Archive captured. And read the crawler rules: curl https://listerleo.com/robots.txt.

What would overturn this reading, honestly stated: our checks ran from a single residential IP on a single day, so they establish what that vantage point saw, not a universal property of the site. A successful Archive capture of any listerleo.com URL at any later date — whether because a configuration changed, a different crawl path succeeded, or our reading was incomplete — would supersede this observation, and we would update the record to say so. We invite exactly that retesting. The point of publishing the commands is that you do not have to take our word for any of it.

Where this sits in the program

This is a companion to our audit of the citation supply and our finding on branded-versus-discovery visibility, and it follows the same rules as our public standard: dated observations, reproducible method, disclosed limitations, and a correction policy that treats an updated reading as routine rather than embarrassing. The running numbers behind all of it, including our own, are on the scoreboard. If a later capture changes what is true here, this page will change with it.

Sources

1. Internet Archive — Wayback CDX Server API documentation
2. Internet Archive — Save Page Now
3. Internet Archive — archive.org_bot crawler identification

Every external figure above was verified against the primary source before publication. Our own figures come from the instrument described on the methodology page; the claims ledger for this post is part of our research record.