Crawler
Crawler Info
The crawler reads your sitemap.xml file and stores the content of each discovered URL in the database for further analysis.
How it works
- • Enter the sitemap.xml URL of the website you want to analyze
- • The crawler reads all URLs from the sitemap
- • Each URL is fetched and its content is stored
- • The data is then available for SEO analyses