Google Says IT's Bot Shield Can Quietly Pull Your Shopping Ads
Google warned on November 11, 2026 that blanket CDN and firewall bot protection can block its crawlers, causing pages to drop out of Search and, per the accompanying video, products to vanish from Google Shopping and ads to stop serving. The risk peaks right now: bot defenses get tightened ahead of Black Friday (November 27), usually by IT, while the Merchant Center disapprovals land on marketing.
The warning came in a Google Search Central blog post about moving to a CDN and a companion episode of Google's video series with Gary Illyes and Cherry Prommawin. PPC Land's writeup is where the Shopping and ads angle gets spelled out: the video says products can disappear from Shopping and ads products can stop working when the crawler hits a wall it can't get through.
Most of the post is aimed at site reliability people. Cold caches, origin servers getting hammered, that kind of thing. But buried in the middle is the part that matters to anyone running Shopping or PMax, and it's the part I'd forward to whoever owns your Cloudflare account today.
Bot protection is somebody else's toggle and your revenue
Google's post says that blanket rules treating all automated traffic the same way will catch Google's crawlers too. A CAPTCHA challenge looks like a wall to Googlebot, and if it persists, pages start dropping out. Google's line on this is pretty good: "Googlebot has many talents; spotting traffic lights and chickens is not one of them."
The fix Google recommends is unglamorous. Most major CDNs and WAFs ship a "verified bots" setting that lets known crawlers through, and Google publishes its crawler IP ranges and a reverse DNS verification method so security teams can confirm a request really is Google and not a scraper wearing its user agent.
This is where the org chart gets in the way. In most companies I've seen, the person who flips on aggressive bot mode in November is an engineer responding to a credential-stuffing spike or a scraper flood, and fair enough. Bot traffic is a real problem. One Shopify merchant documented on r/shopify more than 500 fake cart additions per hour earlier this year, with Cloudflare challenge mode and rate limiting failing to stop it. Nobody in that situation is thinking about Storebot-Google.
And they probably should be, because Google's list of common crawlers says preferences addressed to Storebot-Google affect all Google Shopping surfaces, including the Shopping tab. Googlebot handles landing pages for Search. Block either one and you've got a problem, just in different reports.
The person who can break your Shopping campaign this month probably doesn't have a login to Merchant Center.
The "blocked" page that returns a 200 is the nasty one
This isn't new advice, to be fair. Google covered it back in its December 2024 post on CDNs and crawling, which split CDN blocks into two kinds. Hard blocks return an error code like 503 or 429, which Google reads as "come back later." Soft blocks return a normal 200 status with an interstitial ("are you human?") instead of your content.
Google called the soft version "particularly bad." If it can't tell the challenge page is an error, every URL serving that same page can get folded together as duplicates and dropped. Recovery is slow, too, because Google doesn't have much reason to recrawl what it thinks are duplicates. The new video makes the same point: a 403 wall, or a "you've been blocked" message served with a 200 code.
Think of it like a bouncer who doesn't refuse your best customer outright but hands them a fake menu. They leave thinking the restaurant serves nothing, and they don't come back to check for a while.
One more wrinkle that I didn't expect. If you're on Cloudflare's free plan, Cloudflare's own docs say you cannot bypass or skip Bot Fight Mode using WAF custom rules or Page Rules, because it runs in a separate pipeline. So the "just add an allow rule for Google" fix a lot of people reach for may do nothing at all. Super Bot Fight Mode (Pro plans and up) supports skip rules; the free toggle doesn't. That's a meaningful detail if your store sits on a cheap plan with the bot toggle switched on.
How the Merchant Center side actually fails
On the Merchant Center end, a crawler that can't reach your pages shows up as a "product page unavailable" style issue in Needs attention, and the affected products stop serving. FeedArmy's Emmanuel Flossie, quoted by PPC Land in March, has a useful rule of thumb: if roughly 10% of products get flagged, it's often Google's crawler having a bad day; if everything gets flagged, the problem is probably your site. He says a manual "Request website check" usually gets products approved again within about 24 hours.
Google's Merchant Center help page on blocked product pages is slower and more conservative. It says fixes can take 24 to 72 hours to clear from the Needs attention page, and errors should resolve within a couple of days after a new feed upload. It also mentions that after a robots.txt change, products may keep showing for up to 30 days using previously approved images, which is a nice cushion for images and not much of one for landing pages.
Do the math on that against the calendar. Black Friday is November 27. If a bot setting goes on November 24 and nobody notices until the 25th, a 72-hour recovery puts you back online roughly on Black Friday itself, or just after. That's the scenario I'd plan against.
A 20-minute check before November 19
I'd want this done by Thursday next week, November 19, leaving a full week of slack before the peak. Here's the order I'd go in:
- Merchant Center, Needs attention tab. Filter for any landing page or "page unavailable" issue. Note the percentage of products affected. Zero is the benchmark. Anything above roughly 10% (Flossie's line) deserves a look at your own setup rather than blaming Google.
- Ask who owns the CDN or WAF. Literally one Slack message: "Is verified-bot allowlisting on, and are we planning to tighten bot rules before Black Friday?" Get a name.
- Check the security event log for Google user agents. In Cloudflare that's the Security events view. Search for Googlebot and Storebot-Google and look for challenges or blocks. If you find any, verify the IPs against Google's published ranges before panicking; fake Googlebots are common.
- Run URL Inspection on five top product URLs in Search Console. If the rendered page shows a challenge screen instead of your product, you've found it.
- Agree on an escalation rule. Any new bot rule between now and Cyber Monday gets a heads-up to the paid search lead. That's it. Not a process, just a heads-up.
If crawling actually overloads your origin (less likely for most mid-size stores, but it happens), Google's post says to return 429 or 503 temporarily, for "a few hours, or 1-2 days at most." That's the safe brake. A challenge page served with a 200 is the unsafe one.
Google left out the part we'd most like to know
Something that bugs me a little here: Google gives no numbers. As PPC Land points out, there's nothing on how often this happens, how long a block has to last before products are pulled, or how big the crawl spikes get. So you can't really calculate your exposure. You can only reduce it.
And honestly, the timing of the post feels deliberate. Google publishing "please don't block our crawlers" two weeks before the biggest retail weekend of the year suggests it's seen enough of this to bother. My guess (and it is a guess) is that at least 1 in 10 mid-size Shopping advertisers will see a bot-related disapproval spike somewhere between now and Cyber Monday, mostly from settings changed in the final week.
This also fits a pattern we've been tracking. Bot traffic is now big enough to distort GA4 direct traffic, and companies like United are trying to ban AI agents outright without a reliable way to tell who's who. Every team is building walls, which is reasonable. The catch is that Google's shopping crawler has to get through the same door as everyone else, and it isn't very good at puzzles.
Stack that on top of the November core update that may still be rolling on Black Friday and it's a slightly uncomfortable month to be relying on Google's view of your site being accurate.
One Slack message is cheaper than a lost weekend
None of this requires a budget or a new tool. It requires the paid search person and the infrastructure person to talk once before Thanksgiving, which, in a lot of companies, is apparently the hardest part.
I don't think most stores will lose their Black Friday to this. But the ones that do probably won't find out from a dashboard alert. They'll find out on Saturday morning, when someone asks why Shopping revenue looks like a Tuesday.