What you get
A sitemap.xml file in the sitemaps.org 0.9 format, ready to upload, and an inventory of every address we met: its HTTP status, final URL after redirects, content type, canonical tag, noindex directives, robots.txt rule and whether it went into the sitemap, with the reason when it did not. Both download from the report page; the inventory is a CSV file for Excel, Numbers or Google Sheets.
We do not submit anything to Google or Bing on your behalf. You upload the file to your own site and tell search engines where it is.
Which pages are included
- The page answers HTTP 200 directly; redirects are replaced by their destination.
- It is an HTML page on the same host and protocol as your homepage.
- It has no noindex in a robots meta tag or X-Robots-Tag header.
- robots.txt does not block it.
- It is its own canonical, or has no canonical tag. A page whose canonical points to another of your URLs is left out; that other URL is listed if it qualifies itself.
Honest limits
The free crawl follows links up to two clicks from your homepage, plus the pages in your current sitemap, and stops at 40 pages or 25 seconds. When it stops early, the report says the crawl is partial and lists the addresses it found but could not check. Pages nothing links to cannot be found by any crawler.
A lastmod date is added only when your server sends a real Last-Modified header. We never invent dates, and we leave out changefreq and priority, which Google ignores.
Publishing the sitemap
Upload sitemap.xml to the root of your site so it opens at yourfirm.com/sitemap.xml, add the line Sitemap: followed by its full address to robots.txt, and submit it once under Indexing, Sitemaps in Google Search Console and in Bing Webmaster Tools. If your CMS already generates a sitemap, keep that one and use this report to fix what it lists.
Generate the file again after adding or removing pages, and after any redesign or migration: a new theme or CMS often changes addresses, and a sitemap that still lists the old ones sends crawlers into redirects and errors. The inventory is also a quick checklist for a developer: every row left out says exactly what to change.