Overview

Overview

Before learning how to use each tool, it’s worth knowing where we get our data from.

At Ahrefs, we crawl the web much like search engines do: using crawlers.

Our crawler is called AhrefsBot. It’s the #1 most active SEO crawler (and 8th most active crawler after Google’s and Bing's), visiting over 8 billion web pages every 24 hours and updating its index every 15–30 minutes.

Here’s how it works:

  1. Our crawler starts from a big list of known URLs.
  2. These URLs are then sent to the Scheduler, which decides when a URL should be crawled.
  3. When a URL is ready to be analyzed, AhrefsBot will send the URL to the Crawler and download its contents; all while strictly respecting the allow/disallow rules set in the robots.txt file.
  4. The crawler will deliver the raw data to the Parser, which extracts links on that page, the title, and other relevant metadata.
  5. The extracted data is then sent to the Indexer.
  6. It’s then added to the Link Index and made available in various reports in Ahrefs.

This is a linear and simplified explanation of how our crawler works. In order to continuously discover new pages and increase the size of our backlink index, we feed our list of URLs by surfacing new links from:

Currently, we show live link data and links that were deleted in the last 60 days.

Module 1. Ahrefs’ Crawler and Index

Module 2. Ahrefs SEO metrics

Module 3. Exploring your Ahrefs account

Module 4. Dashboard

Module 5. Site Explorer

Module 6. Keywords Explorer

Module 7. Site Audit

Module 8. Rank Tracker

Module 9. Content Explorer

Module 10. Competitive Analysis

Module 11. Alerts

Module 12. Toolbar

About this course How to Use Ahrefs

What you’ll learn

Course by

Andrei Țiț is a product marketer at Ahrefs. He develops content to help you make the most out of Ahrefs.

Sam Oh is VP of Marketing at Ahrefs. He incorporates his commitment to education and love for entrepreneurship into actionable and easy-to-digest tutorials.

Certification Exam

Take the course exam to test what you’ve learned. Pass it and you’ll get a personal certificate signed by our educators and Ahrefs.

Take exam An Ahrefs account is required to take this exam. Sign up for an Ahrefs Free account or paid subscription if you don’t have one.