Sitemap Generator: Build an XML Sitemap for Any Site
Generate an XML sitemap for search engines, not a visual site map for design work. Enter a site, let it crawl up to 500 pages, then download sitemap.xml and submit it.
How to create a sitemap
- 1
Enter your site address
Start from the home page, or from a section such as /blog to map just that part of the site.
- 2
Let it crawl
The generator follows internal links on the same host, obeys robots.txt for Googlebot, and shows each page it reaches.
- 3
Download sitemap.xml
Only indexable pages go in. Redirects, errors, noindex pages and pages with a canonical elsewhere are listed separately with the reason.
How to submit your sitemap
Generating the file is half the job. Search engines only use it once they know where it is.
- 1
Upload it to your site root
Place the file so it loads at yourdomain.com/sitemap.xml. Search engines look there first.
- 2
Open Google Search Console
Select your property, open Indexing > Sitemaps, enter sitemap.xml under "Add a new sitemap" and press Submit.
- 3
Add it to Bing Webmaster Tools
Open Sitemaps, choose Submit sitemap and paste the full URL of the file. Bing checks it on its own schedule after that.
- 4
Reference it in robots.txt
Add a line such as Sitemap: https://yourdomain.com/sitemap.xml so every crawler, AI crawlers included, can find it without a submission.
XML, HTML and visual sitemaps
-
XML sitemap
A machine-readable list of URLs for search engines, following the sitemaps.org protocol. This is what the generator builds.
-
HTML sitemap
A page of links for human visitors. Useful on large sites, but search engines do not need it once an XML sitemap exists.
-
Visual sitemap
A diagram of site structure used in UX and design planning. It shares the name and nothing else.
What the sitemap protocol allows
-
50,000 URLs per file
A single XML sitemap holds up to 50,000 URLs and must stay under 50MB uncompressed.
-
Sitemap index files
Larger sites split their URLs across several sitemaps and list them in one index file, which is what gets submitted.
-
An honest lastmod
lastmod should say when a page genuinely changed. The generator leaves it out rather than stamp every URL with today.
-
Only indexable URLs
A noindex page or one whose canonical points elsewhere will not be indexed, however many sitemaps list it.
When this free tool is not enough
-
500 pages per crawl. The crawl stops at 500 pages and warns you if URLs were still queued. A truncated sitemap understates the site, so crawl one section at a time instead.
-
JavaScript-rendered links. The crawler reads the HTML the server sends. Pages linked only by client-side routing in React, Vue or Angular apps will be missed.
-
Sites that block automated requests. Pages that answer 403 or 429 are listed as skipped with the status code, not silently dropped.
-
One run, no schedule. Nothing regenerates the file after a deploy. Rerun the crawl whenever pages are added or removed.
Crawling past 500 pages, rendering JavaScript or regenerating on a schedule is what MrScraper's Web Scraper API is for.
Sitemap generator FAQ
How do I generate a sitemap?
A sitemap is generated by entering your website address into a sitemap generator, which crawls the site and collects every reachable page. The tool writes those URLs into an XML file in the sitemaps.org format. Download the file, upload it to the root of your site, then submit its address in Google Search Console.
What is the best free sitemap generator?
The best free sitemap generator depends mostly on the size of your site, because most free tools cap the crawl somewhere between 500 and 5,000 URLs. Check the limit before you start, since a truncated sitemap is worse than none. This generator states its cap of 500 pages and warns you when a crawl hits it.
Can AI be used to create a sitemap?
AI is not needed to build the XML file itself, which is the mechanical result of a crawl. AI helps with the harder parts around it: rendering JavaScript pages that a simple crawler cannot see, and deciding which URLs deserve a place in the sitemap. A plain crawler plus a clear inclusion rule covers most sites.
Is a sitemap still relevant in 2026?
A sitemap is still relevant in 2026 for large sites, new sites with few backlinks, and any site whose pages are poorly linked internally. Google finds well-linked pages without one, so a sitemap matters most where internal linking is weak. The file also gives AI crawlers a complete URL list instead of relying on discovery.
How do I submit a sitemap to Google?
A sitemap is submitted to Google by uploading sitemap.xml to your site root and entering its path in Search Console. Open your property, go to Indexing and then Sitemaps, type the path and press Submit. A Sitemap line in robots.txt also lets Google and other crawlers discover the file without a manual submission.
Where should I put my sitemap file?
The sitemap file belongs in the root of your website, so it is reachable at yourdomain.com/sitemap.xml. Search engines look there first, and a sitemap can only list URLs at or below its own location. Referencing the file from robots.txt makes it discoverable by every crawler without a manual submission to each search engine.
How many URLs can an XML sitemap contain?
A single XML sitemap can contain up to 50,000 URLs and must not exceed 50MB uncompressed. Larger sites split their URLs across several sitemap files and list those files in a sitemap index, which is what gets submitted to Search Console. Gzip compression is allowed, but the 50MB limit applies to the uncompressed size.
Do I need a sitemap if I have robots.txt?
Robots.txt and a sitemap do different jobs, so most sites benefit from having both. Robots.txt tells crawlers which paths they may not fetch, while a sitemap lists the pages you want indexed. Robots.txt can reference the sitemap with a Sitemap line, which is how many crawlers find it in the first place.
Do AI crawlers like GPTBot use sitemaps?
AI crawlers including GPTBot and ClaudeBot fetch robots.txt, where a Sitemap line is visible to them, and they generally follow links across a site. A sitemap gives them a complete URL list instead of relying on discovery. Being listed does not guarantee inclusion in any training set or answer engine result.
Why is my sitemap not being indexed?
A submitted sitemap does not guarantee indexing, because it guarantees discovery and nothing more. Common causes are noindex tags, canonical tags pointing elsewhere, pages blocked in robots.txt, thin content and URLs that return errors. The Page indexing report in Search Console names the specific reason for each URL that was not indexed.
Can a sitemap generator crawl JavaScript pages?
Most free sitemap generators read only the HTML the server returns, so pages linked by JavaScript may be missed. Sites built as single-page apps with React, Vue or Angular usually need a crawler that renders each page in a real browser before collecting links. Server-side rendering or static generation avoids the problem entirely.
What is a sitemap index file?
A sitemap index file is an XML file that lists several sitemap files instead of individual pages. Sites with more than 50,000 URLs split their sitemap into parts and reference every part from one index. The index is the single address submitted to Search Console, and it can list up to 50,000 sitemaps.