How do I configure Cloudflare to allow the Claritybot user agent to crawl my site?

How do I configure Cloudflare to allow the Claritybot user agent to crawl my site?


Please note that the following settings are part of the Cloudflare platform and could be changed by them without notice.
  1. In https://dash.cloudflare.com,/ select the domain name that will be the crawl target
  2. Navigate to the “Firewall” tab and click “Firewall Rules”
  3. Click "Create Firewall rule"
  4. Input: a Rule name, for example “Allow Claritybot”
    Field: User Agent
    Operator: equals
    Value: {select a user agent string from below}
  5. Choose an action: Allow
Desktop User-Agent String:  
     Mozilla/5.0 (compatible; ClarityBot/9.0; +https://www.seoclarity.net/bot.html)
Mobile User-Agent String:
      Mozilla/5.0 (Linux; Android 9; SM-G960F Build/PPR1.180610.011; wv) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0
      Chrome/74.0.3729.157 Mobile Safari/537.36 (compatible; ClarityBot/9.0; +https://www.seoclarity.net/bot.html)

Compare your page to the diagram below. When you are happy, click "Deploy". 




    • Related Articles

    • Why did my crawl return an error?

      Crawl Error Video Malformed or Incorrect URL Check that the starting URL is correct and is accessible (200 response) to the user agent setup for the crawl. User Agent We recommend adding our crawler as an allowed crawler based on the user agent name, ...
    • Allowing The seoClarity Crawler To Crawl Your Site

      Overview The seoClarity crawler can only crawl your site if you allow it to. With the volume of bad bots increasing day by day, most sites are enhancing their security to block unknown bots from accessing their site. In that respect, it is important ...
    • Site Audit Projects

      Site Audit Projects Overview The Site Audit Projects List gives you a high level view of the different crawls that have been setup for the domain. Watch the video below: "How to Create a Clarity Audit Project" Background & Requirements Some sites ...
    • Site Audit Settings

      Site Audit Settings Overview Site Audits provides a variety of reports and analysis based on a crawl, that can impact the health of a site. Site Audit Settings allow for the customization of those Site Audit reports by enabling the ability to ...
    • Site Audit Details

      Site Audit Details Overview Site Audit Details provides a variety of reports and analysis based on a crawl, that can impact the health of a site. Site Audit Details Use Cases Identify and fix potential user experience issues. Learn more Identify ...