A sitemap lists a domain’s URLs in a structured file that can be read by search engines or visitors. Submitting this file is not enough: Google treats a sitemap as one signal among many, without guaranteeing that the listed pages will be crawled or indexed. The question that follows is measurable: what is the gap between the URLs declared in a sitemap, those actually indexed, and those visitors can really access through internal links?
XML sitemaps, internal links and indexing: measuring the gaps
Three channels can help a page get discovered: the XML sitemap signals it to crawlers, internal links make it accessible from other pages, and the Search Console indexing report confirms whether it is in the index. These three sources do not always align.
| Discovery channel | What it guarantees | What it does not guarantee |
|---|---|---|
| XML sitemap | The crawler knows the URL | That the page will actually be crawled and indexed |
| Internal links | Visitors and crawlers can reach the page | That it will be included in the index if the page is low quality |
| Sitemap report (Search Console) | The file has been read and processed | That every listed URL is in the index |
A sitemap file can be processed without all its URLs being indexed. The Search Console Sitemap report lets you check whether the file has been processed and identify errors, but the indexing status of each page must be checked separately.
Cross-referencing these three sources reveals a common blind spot: pages listed in the sitemap and known to the search engine, but with no internal links. These pages exist for Google but not for visitors browsing the site. To see this type of structure, Klottra’s sitemap shows how an HTML sitemap makes a domain’s full structure visible.

Orphan pages in the sitemap: finding what visitors never find
An orphan page is a URL listed in the sitemap that no internal links point to. A crawler can discover it through the XML file and index it, yet visitors have no navigation path that leads to it.
The diagnosis involves a simple cross-check:
- Export the list of URLs from the XML sitemap and compare it with a full site crawl to identify pages with no incoming internal links
- Check Search Console to see whether these orphan pages are indexed, pending or excluded
- Identify which of them generate impressions in the performance report, indicating that they match queries despite being inaccessible through site navigation
An indexed page with no internal links receives few relevance signals from the site itself. Google may eventually remove it from the index, considering that the publisher has not given it enough prominence.
The reverse problem also occurs. Some pages have strong internal links but are not listed in the sitemap. A page can still be indexed without being in the sitemap if its internal linking is good, but this deprives the webmaster of a centralized monitoring tool.
XML sitemap metadata: lastmod, priority and their actual impact
The sitemap protocol provides several optional fields for each URL. Their usefulness varies considerably across search engines.
The lastmod field as the only reliable signal
Google says that the lastmod date is useful only if it reflects a real and significant change. A CMS that updates this date every time it regenerates the file, without any content changes, sends a misleading signal. A sitemap that is updated less often but remains accurate is preferable to one that artificially changes the dates of every page.
Google does not take the priority and changefreq fields into account. Setting a priority of 1.0 for every page has no effect on crawl frequency.
Organizing sitemaps by section for targeted diagnosis
Google allows a sitemap index to reference multiple files. Separating products from blog posts or corporate pages is useful for more than just staying within the URL limit per file. This segmentation helps identify which section of the site is having processing issues in Search Console.
If sitemap-products.xml has a low indexing rate while sitemap-blog.xml is almost fully indexed, the problem lies in the quality or structure of the product pages, not in the overall sitemap configuration.

Visual resources and sitemap extensions
A sitemap is not limited to HTML pages. Sitemaps can include extensions for images, videos and news. These extensions help search engines identify resources that might otherwise remain invisible in specialized results (Google Images, the Videos tab).
Submitting an image through the sitemap extension does not replace the technical access requirements: if the image file is blocked by robots.txt or the host page is not indexed, listing it in the sitemap has no effect. The sitemap extension signals the resource; it does not force it to appear in search results.
For sites where visual content accounts for a significant share of organic traffic, the sitemap image extension provides an additional discovery opportunity, provided the alt tags and the page’s textual context are consistent with the target query.
HTML sitemap and XML sitemap: two maps for two audiences
An HTML sitemap is a page on the site for visitors, listing sections and key pages as clickable links. An XML sitemap is a technical file for crawlers that lists URLs and their metadata.
- An HTML sitemap improves human navigation and strengthens internal linking by providing an access point to every section
- An XML sitemap helps crawlers discover pages faster and provides a diagnostic tool through Search Console
- The two formats complement each other: HTML addresses navigation gaps, while XML addresses crawl gaps
A site with only an XML sitemap but a deep or poorly linked structure loses some of its visitors. An HTML sitemap acts as a safety net for navigation, providing a direct path to pages that are not prominent in the menus.
A sitemap remains a mapping tool, not a guarantee of indexing. Its value lies in the gap it helps bridge between what search engines know and what visitors can actually find.



