Publishing an AI-written post is not the finish line. A page Google has not indexed cannot rank, cannot earn traffic, and cannot be cited in AI Overviews or other retrieval-augmented answers. The number that should control your pipeline is not posts per week — it is the share of your posts that are actually indexed.
An Ahrefs study of roughly one billion pages found that about 90% get no organic search traffic from Google (Ahrefs, 2020). Unindexed pages are guaranteed to be in that group. This article describes how Knowhow Seller uses a Google indexing gate to decide when to keep publishing and when to stop, based on hands-on experience running our own content pipeline and on Google’s published documentation.
When should you stop publishing AI content?
In our pipeline, we stop publishing new AI content when Google’s indexed share of our mature posts falls below roughly 60%. We read that as a sign the site has hit its crawl ceiling: new URLs compete with old ones for a limited crawl budget. Fix or consolidate existing posts first, then resume.
What is a Google indexing gate?
A Google indexing gate is a rule that ties how much you publish to how much of your existing content is indexed. When the indexed share is healthy the gate allows more posts. When it drops, the gate throttles output to zero and switches the pipeline into reinforce mode — improving existing pages instead of adding new ones.
| Indexed share of mature posts | Daily publish allowance | Pipeline mode |
|---|---|---|
| 80% or higher | 2 posts | Normal publishing |
| 60–79% | 1 post | Cautious publishing |
| Below 60% | 0 posts | Reinforce existing pages only |
These tiers are our own operating thresholds, not numbers published by Google.
How do you measure your indexed share?
Pull every published URL, then check each one against Google Search Console — the URL Inspection API for automation, or the Page Indexing report by hand. Count a post only after a 7-day grace period, since normal crawl lag makes fresh URLs look unindexed. Indexed share is indexed URLs divided by mature URLs.
A few practical notes from running this check regularly:
- The URL Inspection API is subject to a per-property daily quota (Google documents it at roughly 2,000 calls per day) — cache confirmed-indexed URLs so you only re-check the uncertain ones.
- Exclude rows where the API itself failed (auth errors, 500s). A failed check is not an unindexed page, and mixing the two corrupts the ratio.
- Match both the English and localized
coverageStatestrings if your properties span languages. - Google removed the sitemap ping endpoint in 2023 (Google Search Central), so resubmitting in GSC and keeping
<lastmod>honest is how you signal changes now.
What does a real index-gate snapshot look like?
Here is one morning’s reading from our own WordPress engine — a single point-in-time snapshot from our pipeline, not a controlled test. Knowhow Seller runs a self-reinforcing content pipeline, and the gate audits it each morning. At the time of this snapshot the site had 37 live posts and the gate was sitting just inside the cautious band.
| Metric | Value | What it means |
|---|---|---|
| Live posts | 37 | Total published, past the 7-day grace window |
| Indexed | 79% | Above the 60% throttle line, below the 80% “publish 2” line |
| Discovered – currently not indexed | 5 posts | Google knows the URL but has not crawled or indexed it |
| Ghost URLs | 2 | Deleted posts still listed as submitted in Search Console |
That 79% put the engine on a one-post-per-day allowance. The five “discovered – currently not indexed” URLs are the early-warning signal: Google’s documentation describes this state as pages it found but has not crawled, often because of crawl-budget limits or perceived site quality (Google Search Central Help). Left alone, those five tend to drift toward a lower indexed share and can eventually trip the gate.
Where did the 60% throttle threshold come from?
The threshold was set from the engine’s own history, not from a blog post. At low publishing volume, indexed share of mature posts sat near 80%. When output ramped to several posts a day, indexed share fell into the low 20s — new URLs were crawled instead of old ones, and the total indexed count dropped. In our data, 60% is roughly the point where “publish more” began making the indexed share worse rather than better.
The logic behind it is Google’s own: crawl demand is shaped by how popular and how high-quality Google considers your URLs (Google Search Central, “Large site owner’s guide to managing crawl budget”). A low-authority site that floods thin pages can teach Google that its new URLs are not worth crawling. The gate assumes that ceiling is real and treats the indexed share as the throttle.
What are the three index-failure states, and how do you fix each?
Google’s Page Indexing report surfaces several failure states, and each needs a different fix. “URL is unknown to Google” means Google never found the URL. “Discovered – currently not indexed” means it found the URL but has not crawled it. “Crawled – currently not indexed” means it crawled the page and did not index it, often on quality grounds.
| Search Console state | Likely cause | Fix |
|---|---|---|
| URL is unknown to Google | Sitemap not submitted, or the page is orphaned | Submit the sitemap in GSC; add internal links from indexed pages |
| Discovered – currently not indexed | Publishing volume outruns site authority; crawl budget stretched | Cut publishing volume; strengthen and interlink existing posts |
| Crawled – currently not indexed | Thin or low-value page | Expand with unique information, or merge into a stronger post |
Why do deleted posts still show as “submitted” in Search Console?
Because removing a page from your CMS does not tell Google it is gone. Until Google recrawls the URL and receives a 404 or 410, or you drop it from your sitemap, Search Console can keep listing it as submitted. The Removals tool only hides a URL from results for about six months (Google Search Central).
Our snapshot had two of these ghost URLs. The fix is unglamorous: make sure deleted slugs return 410 Gone, remove them from the sitemap, and let the next crawl clear them. They do not hurt rankings, but they inflate the submitted count and make the indexed-share math look worse than it is.
What else should gate your AI publishing?
Two checks belong alongside the indexing gate: a duplicate-topic filter that runs before generation, and a verification step built to fail loudly. Both exist in our pipeline because the absence of each caused a silent outage — stretches of zero useful output while every scheduled job reported success.
Block duplicate topic clusters before you generate
Before writing, the engine compares each candidate topic against existing posts and rejects anything that would cannibalize one. On one run, every one of the seven candidate topics was rejected. That is not a bug — it is a practical signal that a narrow niche seed list is exhausted, and the honest move is to stop rather than publish near-duplicates that split their own rankings.
Make the verification step fail loudly
Our pipeline once ran a single verification LLM as its publish gate. It failed twice — an auth failure once, a usage-quota failure once — and publishing silently went to zero for several weeks, then again for a few days, while every scheduled task still reported success. Exit code 0 hides “did nothing” longer than any crash does.
The fix: split the single verification gate across two different vendors, and alert even when the fallback path succeeds, so a quiet failover still surfaces. We also keep the watchdog process outside the system it watches.
One caveat on our setup: Knowhow Seller is not a developer team and does not read code. Every failure here was diagnosed by pasting the error back to an AI assistant, and every blockage was written down the moment it happened. The gate exists because the failures were logged, not because anyone was watching dashboards.
What does a working index gate check every week?
A working gate re-audits every URL, recalculates the indexed share on mature posts only, maps that share to a publish allowance, and routes below-threshold days into reinforce mode. It also clears ghost URLs and blocks duplicate topics before generation. The trend across weeks matters more than any single reading.
- Audit every published URL against GSC weekly, using a 7-day grace window.
- Set publish tiers: 80%+ = normal, 60–79% = half rate, below 60% = reinforce only.
- Split any content-verification gate across two vendors and alert on fallback success.
- Serve 410 on deleted slugs and remove them from the sitemap.
- Reject candidate topics that overlap existing posts before you generate.
- Log every indexed-share reading so you can see the direction, not just the level.