Cloudflare Now Blocks AI Crawlers by Default—What Law Firms Need to Know

What’s Changed

Cloudflare now automatically blocks known AI bots like:

• GPTBot (OpenAI)
• ClaudeBot (Anthropic)
• Google-Extended
• CCBot (Common Crawl)

And this blocking happens before the bot ever sees your website or robots.txt file, using Cloudflare’s application-layer firewall.

Unless you manually opt in through Cloudflare settings, your content will not be accessible to these crawlers—even if you’re trying to be included in AI search results, chatbots, or voice assistants.

Why This Matters for Law Firms

Modern search behavior is driven by AI:

• Google’s Search Generative Experience (SGE) often answers legal questions without showing a single organic link.
• Chatbots and AI apps pull information from LLM-accessible websites.
• Clients searching for legal guidance may never land on your site if LLMs can’t crawl it.

If your law firm’s content isn’t accessible to these crawlers, your visibility in AI-powered tools is at risk, even if you’ve done everything else right with SEO.

Is Your Website Affected?

Very likely. Many law firm websites are already using Cloudflare—even if you didn’t configure it directly.

This happens because popular web hosting platforms and agencies often include Cloudflare behind the scenes for security, speed, and reliability.

Common Web Hosts & Platforms That Use Cloudflare:

• Squarespace – built-in integration with Cloudflare’s CDN
• Webflow – uses Cloudflare under the hood
• Wix – uses Cloudflare in its infrastructure
• WordPress – often connected via plugins like WP Rocket, SiteGround Optimizer, or  direct connection through hosts like:
  • Rocket.net
  • WP Engine
  • Kinsta
  • Bluehost
  • SiteGround
  • DreamHost
• GoDaddy (for managed WordPress or SSL-enabled sites)
• Shopify (especially if custom domains are managed via Cloudflare)

In many cases, your firm may not even have access to the Cloudflare dashboard—it could be managed entirely by your hosting provider or web team.

What Law Firms Should Do

If your law firm does not manage Cloudflare directly, contact your website hosting provider or developer and ask the following:

Email or Support Request Template:

Hi [Hosting Provider],

I understand that Cloudflare is now blocking AI crawlers by default at the application layer. We want to allow trusted AI bots (such as GPTBot, ClaudeBot, and Google-Extended) to crawl our law firm’s website.

Can you please confirm:

• Is our website using Cloudflare through your platform?
• If so, can you enable AI crawler access in our Cloudflare settings?
• If we manage Cloudflare separately, can you provide credentials or guidance for access?

Thank you,
[Law Firm Name]
[Contact Info]

If You Do Control Your Cloudflare Account

1. Log in to your Cloudflare dashboard
2. Go to Security → Bots
3. Scroll to the AI Crawler Access section
4. Selectively allow trusted AI bots
5. Save changes

Reminder: Simply editing your robots.txt file will not allow AI bots if Cloudflare is still blocking them at the application layer.

Should Law Firms Allow AI Crawlers?

It depends on your goals, but most law firms focused on lead generation, SEO, and legal content marketing should strongly consider opting in.

Pros of Allowing AI Crawlers:

• Appear in AI-powered search results (SGE, ChatGPT, Bing Copilot)
• Boost brand visibility and authority
• Increase indirect traffic from citations and summaries

Risks of Blocking:

• No visibility in AI-generated answers
• SEO investments may be undercut
• Competitors could dominate AI-driven discovery

Ask Your Web Team or Agency

Even if you’re working with a web developer or legal marketing agency, they may not be aware of this policy change. Be proactive:

• Ask them to review Cloudflare bot access settings
• Clarify whether your site’s AI access is currently blocked
• Determine if additional opt-in steps are needed through your host

Control Over Your Site

Cloudflare’s move gives law firms more control over who can crawl their site—but with that control comes responsibility. If you don’t act, your site may stay hidden from the very tools that shape the future of online legal search.

Key Takeaways for Lawyers:

• Cloudflare blocks AI crawlers at the application layer by default
• This affects many law firm websites via platforms like Squarespace, Webflow, WordPress, and Wix
• You must opt in via Cloudflare—robots.txt changes won’t override the block
• Contact your hosting provider or developer to allow trusted AI bots