Website Crawling
Crawl site URLs and resources to identify broken links, errors, redirects, indexability problems, and other technical SEO issues.
Screaming Frog
Technical SEO crawler for site audits, broken links, redirects, metadata, internal linking, and crawl analysis.
Write a ReviewAbout
Screaming Frog SEO Spider is a technical SEO website crawler designed to help users audit websites, identify SEO issues, and analyze site architecture at scale. Its core capabilities include crawling websites to detect broken links and redirects, reviewing page titles and meta descriptions, identifying duplicate content, auditing canonicals and directives, analyzing internal linking, and generating XML sitemaps.
The tool also supports JavaScript rendering, structured data validation, custom extraction and search, crawl visualizations, and integrations with Google Analytics, Google Search Console, PageSpeed Insights, and external link-data providers. Screaming Frog can crawl anything from small websites to millions of URLs depending on available hardware and storage, making it suitable for SEO specialists, technical SEO teams, agencies, and website owners conducting detailed site audits. Licensed users also gain scheduling, crawl saving, advanced configuration, and API integrations for more complex technical SEO workflows.
Features
On-Page SEO Auditing
Analyse page titles, meta descriptions, canonicals, directives, hreflang, duplicate content, structured data, and internal linking.
Advanced Crawl Analysis
Use JavaScript rendering, custom extraction, crawl comparison, visualisations, and external API data for complex technical audits.
Screenshots
Video
Reviews
O
Oleg
Screaming Frog is a great tool for technical SEO. It quickly helps identify broken links, duplicate content, meta tag issues, and other technical problems on a website. 👍
Share your experience
Write a Review
Your review will be published after moderation by the Softor.io team.
Pros and Cons
Pros
- Detailed technical SEO crawling with granular issue detection and filtering.
- Highly configurable crawls with custom search, extraction, and JavaScript rendering.
- Supports scheduling, saved crawls, crawl comparison, and large-site analysis.
- Integrates crawl data with major Google and backlink analysis platforms.
Cons
- Free version is limited to 500 URLs and excludes many advanced crawling features.
- Licences are sold per user, so teams need separate licences for multiple users.
Use Cases
Integrations
Pricing
Free
$0—- crawl up to 500 URLs
- find broken links, errors, and redirects
- analyse page titles and metadata
- review robots directives
- audit hreflang
- find exact duplicate pages
- generate XML sitemaps
Paid
$279year- unlimited crawl limit subject to memory and storage
- scheduling
- save and open crawls
- JavaScript rendering
- crawl comparison
- custom extraction
- structured data validation
- Google Analytics, Search Console, and PageSpeed integrations
FAQ
It crawls websites to identify technical SEO issues involving links, redirects, metadata, indexability, duplicate content, directives, and site architecture.
Yes, the free version can crawl up to 500 URLs, while the paid licence removes this limit subject to available memory and storage.
Yes, the paid version includes JavaScript rendering for crawling and analysing dynamically generated content.
It integrates with Google Analytics, Google Search Console, PageSpeed Insights, Ahrefs, Majestic, Moz, and supported AI APIs.