Screaming Frog SEO Spider Review 2022: Best SEO Tools?

Scam Risk is here to help you protect yourself and fight back

Updated: December 01

By: Scam Risk - Expert Reviewer

Screaming Frog SEO Spider Review 2021 Best SEO Tools

SEO Spider is an SEO tool developed by Screaming Frog, a website search agency based in the UK. It allows you to evaluate website pages and extract essential information that can be used to determine their SEO characteristics.

This software, in my opinion, is exceptional because it presents complex SEO data in an easily understandable format. There are a few issues, such as the lack of cloud-based access or the heavy feature restrictions on the free package.

This review will go over Screaming Frog SEO Spider to see if it really is the best website crawling tool out there.

Now, before we go any further…

This Screaming Frog SEO Spider review has been thoroughly researched with information and testimonials that are available online to anyone in the public. Any conclusions drawn by myself are opinions.

Table of Contents


What I Liked
  • Easy user interface
  • Segmented reporting
  • Top-notch sitemap
The Good


What I Didn't Like
  • Outdated interface
  • Limited documentation
The Bad

And at the end, I’ll answer some of the most often asked questions regarding Screaming Frog SEO Spider and website crawling tool in general.

But most importantly, you’ll see the exact step-bey-step system that many others have used to grow their own successful online business to more than $40,000 a month in passive income.

This system made them swear off SEO for good, because it uses some of the same skills but in a much more powerful and profitable way!

SEO Spider Tool

The Screaming Frog SEO Spider helps you improve your on-site SEO by extracting data and auditing common issues. You can download and crawl 500 URLs free of charge, or you can purchase a license to unlock advanced features.

What Can You Do With The SEO Spider Tool?

SEO Spider is a flexible and powerful site crawler that can crawl small and large websites quickly and allows you to analyze the results in real-time. It provides key onsite data to help SEOs make informed decisions.

Find Broken Links

Find Broken Links

Instantly crawl a website and identify broken links (404s) and server errors. Bulk exports the errors and URLs to either fix or send to a developer.

Audit Redirects

Audit Redirects

Locate temporary and permanent redirects. Identify loops and chains of redirects. Upload a list to be audited in site migrations.

Analyze Page Titles & Meta Data

Analyze Page Titles & Meta Data

You can analyze both page titles and meta descriptions during a crawl to identify duplicates or pages that are too long, too short, missing, not present, or duplicated throughout your site.

Discover Duplicate Content

Discover Duplicate Content

Use the md5 algorithmic scan to find exact duplicate URLs. You can also search for partially duplicated elements, such as page titles and descriptions, including low content pages.

Extract Data With XPath

Extract Data With XPath

Any data you find in the HTML of a webpage can be collected using CSS Path, XPath, or regex. This could include social meta tags, additional headings, and prices.

Review Robots & Directives

Review Robots & Directives

You can view URLs that are blocked in robots.txt, meta robotics, or X-Robots -Tag directives like ‘noindex’ or ‘nofollow’ and canonicals, as well as, rel=”next”, and rel=”prev.”

Generate XML Sitemaps

key onsite seo elements

You can quickly create XML Sitemaps and Image XML Sitemaps with advanced configuration URLs, which include last modified, priority, and change frequency.

Integrate With GA, GSC & PSI

screaming frog seo spider alternative

For more insight, you can connect to Google Analytics, Search Console, and PageSpeed Insights APIs. To fetch performance and user data for all URLs crawled in a crawl.

Crawl JavaScript Websites

Crawl JavaScript Websites

To render web pages, you can use the integrated Chromium WRS. This allows you to crawl dynamic, JavaScript-rich websites and frameworks like React, Angular, and Vue.js.

Visualize Site Architecture

Visualize Site Architecture

Interactive crawl, directory force-directed diagrams, and tree graph site visualizations can be used to assess internal linking and URL structure.

Schedule Audits

google analytics integration

You can schedule crawls to run at specific intervals or export crawl data to any location, including Google Sheets. You can also automate the entire process via the command line.

Compare Crawls & Staging

You can track the progress of SEO opportunities and issues and what has changed between crawls. Advanced URL Mapping allows you to compare staging and production environments.

The SEO Spider Tool Crawls & Reports

The SEO Spider Tool Crawls & Reports

The Screaming Frog SEO Spider is an auditing tool created by real SEO experts with thousands of worldwide users.

Here is a quick overview of some data that are collected during a crawl:

  • Errors – Client errors such as broken links & server errors (No responses, 4XX client & 5XX server errors).
  • Redirects – Permanent, Temporary, JavaScript redirects, and meta refreshes
  • Block URLs – View and audit URLs that are blocked by robots.txt.
  • Blocked Resource – View and audit the blocked resources in rendering mode.
  • External Links – View all links, including their status codes and source page.
  • Security – Find insecure pages, insecure forms, and missing security headers.
  • URL Issues – Non-ASCII characters.
  • Duplicate pages – Use advanced algorithmic checks to find exact duplicates and close copies of pages.
  • Page Titles – Missing, duplicated, long, or multiple title elements.
  • Meta Description – Missing, duplicate, long, short, or multiple descriptions.
  • Meta Keywords – These keywords are used primarily for regional or reference search engines, not by Google, Bing, or Yahoo.
  • File Size– Size of URLs and Images
  • Response time – See how long it takes pages to respond to your requests.
  • Last-Modified Header – View the HTTP header’s last modified date.
  • Crawl Depth – See how deep a URL is in a website’s structure.
  • Word Count – Analyze the number of words per page.
  • H1 – Missing, duplicated, long, or multiple headings
  • H2 – Missing, duplicated, long, or multiple headings
  • Meta Robots – Index, noindex, follow, nofollow, noarchive, nosnippet, etc.
  • Meta Refresh – Includes target page and time delay.
  • Canonicals – Link elements and canonical HTTP headers
  • HTML-Robots Tag – See directives via the HTTP Header.
  • Pagination View rel=”next” and rel=”prev” attributes.
  • Follow and Nofollow View meta nofollow and nofollow link attributes.
  • Redirect chains – Find redirect chains and loops.
  • Attributes – Audit missing confirmation links, inconsistent and incorrect languages codes, non-canonical hreflang, and more.
  • Inlinks – View all pages that link to a URL. See the anchor text, and decide if the link is follow- or nofollow.
  • Outlinks – View all pages to which a URL links, as well resources.
  • Anchor text – All links. Alt text for images that have links
  • Render – Crawl JavaScript frameworks such as AngularJS or React by crawling the rendered HTML once JavaScript has been executed.
  • AJAX – Select to follow Google’s now obsolete AJAX Crawling Scheme.
  • Images – All URLs that include an image link and all images on a page. Images over 100kb, missing alt text, alt text over 100 characters.
  • User Agent Switcher – Crawl as Googlebot or Bingbot! You can also use mobile user agents and your own custom UUA.
  • Custom HTTP headers – Any header value can be supplied in a request.
  • Custom Source code Search – Find any information you need in a website’s source code! It doesn’t matter if it’s Google Analytics code or specific text.
  • Custom Extraction – Scrape any data from a URL using XPath or CSS Path selectors.
  • Google Analytics Integration – Connect to the Google Analytics API to pull in conversion and user data during a crawl.
  • Google Search Console Integration – Connect to the Google Search Analytics API to collect impression, click, and average position data against URLs.
  • PageSpeed Integration – Connect to the PSI API to get Lighthouse metrics, speed possibilities, and Chrome User Experience Report (CrUX) data.
  • External link Metrics – Pull out external link metrics from Majestic and Ahrefs APIs into a crawl to perform content audits, profile links, or other related tasks.
  • XML Sitemap Generator – Use the SEO spider to create an XML sitemap or an image sitemap.
  • Custom Robots.txt – Download, Edit, and Test a Site’s Robots.txt Using the New Custom Robots.txt.
  • Rendered Screenshots – View, analyze and fetch the generated pages crawled.
  • View HTML & Store HTML – This is essential for analyzing the entire DOM.
  • AMP Crawling & Validation – Crawl AMP URLs to validate them using the official AMP Validator.
  • XML Sitemap Analysis – Use an XML Sitemap to crawl missing pages, orphan pages, or as part of a crawl.
  • Visualizations – Analyze the internal linking and URL structure using crawl and directory tree force-directed diagrams and tree graphs.
  • Structured Data & Validation – Get structured data and validate it against specifications.
  • Spelling & Grammar – You can spell and check your website in more than 25 different languages.
  • Crawl Comparative – Use crawl data to compare the issues and track technical SEO progress. Comparing site structure and detecting changes in critical elements, metrics, and URL mapping allows you to compare staging with production sites.

About The Tool

The SEO Spider is an advanced and fast SEO site audit tool. It can crawl small and large websites. Manually checking each page can be very labor-intensive, and you may miss a meta refresh, redirect or duplicate page issue.

The program’s interface allows you to view, filter, and analyze the crawl data as it is gathered. It also keeps the data updated constantly.

SEO Spider enables exporting key onsite SEO elements such as a URL, meta description, page title, headings, etc., to a spreadsheet. This spreadsheet can be used to make SEO recommendations.

Crawl 500 URLs For Free

You can download the ‘lite’ free version. This version can only crawl up to 500 URLs per crawl. It does not allow you to save crawls or configure advanced features like JavaScript rendering, custom extract, Google Analytics integration, and many other valuable features. You can crawl as many URLs as you want from one website or 500 URLs from multiple websites.

A license can be purchased for just PS149/year. This license allows you to remove the 500 URL limit, save crawls, and gives you access to advanced features and configuration options.


  • It is easy to use and configure.
  • A reliable output of detailed lists with errors, issues, and optimization suggestions –
  • Segmented reporting allows users with experience to narrow down specific issues.
  • Sitemap.xml solves large-scale site implementation issues and is top-notch.


  • The interface is outdated; it takes some time to learn where to find different views or data.
  • Needs regular updating
  • There is limited documentation available in the tool. You will need to consult Knowledgebase.

What Is Our Top Recommendation For Making Money Online In 2021?

screaming frog seo spider review

Our review team has come across a program in the real estate industry that is next level!

Although it’s not real estate in the traditional sense, it’s all digital.

Yup, Digital Real Estate!

Where Screaming Frog falls short is in scalability.

You can’t realistically expect to be able to do SEO all day everyday.

It just takes too many resources (money included) for any regular person to bootstrap, not to mention some clients can be real headaches!

But what if you could make even more money off of little local websites without having to spend all your day monitoring different campaigns?

With this digital real estate program, you can profit off LOCAL traffic to your website every single day!

Sound too good to be true? Of course it does! But it isn’t…in fact, business owners wish they had this skill!

All you have to do is build and rank a LOCAL website and forward the jobs off to a business owner in town, you could even email it to them!

This works for literally any service based business, tree service, plumbing, towing, etc.

How do you get paid and how much?

Simple, after you forward the jobs off to a business owner and he makes some money off of them, you simply ask to make the deal beneficial for each other.

A fair price to charge per lead, depending on the industry is 10-20%…let’s just use the tree service industry for example and go by worst case scenario.

Let’s say you build and rank the site and only 10 jobs a month come in. The average tree service job is anywhere from $500-$2000!

That means at bare minimum you have an asset worth $500 a month!

See why they call it digital real estate now? That’s a rent payment. Because It’s YOUR ASSET

The great thing is how easy it is to scale. You don’t have to worry about clients that are headaches because you own the website.

Back to SEO, why land a small $500 SEO client that is treating you like an employee, when you can own your own asset with leverage?

This model actually allows you to collect HUGE FLAT RATE DEALS. Truly passive income!

The training program takes making money online to a whole other level. The owner of the program walks you through how to build and rank a site hand in hand, with the occasional voice over when he is sharing his screen.

You will learn the importance of keywords, website name, how to send call notifications via email, backlinking, etc.

Once the training program is completed you will also have access to a Facebook group much better than the other SEO groups in my opinion. This group is much more active.

Unlike SEO, where you’re getting maybe $250 per month, you could be getting 10-20X THAT.

screaming frog seo spider tool

A business will always want more leads and another job. In fact it doesn’t even matter that the job isn’t coming from their website name…they see it as it is…expanding digital real estate.

Unlike SEO, more people have been able to walk away from their 9-5 job as well.

Now, I know you probably have tons of questions…

So, check this out to learn more.

ScamRisk's #1 Online
Business Recommendation:
You know deep down what you're meant for. Now they will show you how to go get it.
We will be happy to hear your thoughts

Leave a reply

Scam Risk