Crawl the way a search spider does

Visual SEO Studio spider explores web sites like a search engine bot would, showing in real-time the taken crawl paths.

Visual SEO Studio spider explores sites by following their internal links. Crawl order resembles what a web search engine crawler would do. The program fully respects the Robots Exclusions Protocol, and shows in real-time the crawl paths taken.

Real-time Crawl Session - Visual SEO Studio
Simulated G-time shows how long a search engine spider would take to visit the site

  • Fine tune your crawl option if you need

URL lists can be merged from multiple sources
URL lists can be merged from multiple sources (click to enlarge)

Three-level crawling of XML Sitemaps: robots.txt, index Sitemap and normal sitemaps - Visual SEO Studio
Three-level crawling of XML Sitemaps: robots.txt, index and normal sitemaps
(click to enlarge)