“Indexed, though blocked by robots.txt” in Google Search Console
What the status means, whether it's a problem, and how to fix it, step by step.
Indexed, with a warning · Checked against Google's documentation on 2026-10-30
Short answerDepends
Google indexed the URL without reading it, usually because other pages link to it. If you want the page in search, remove the robots.txt rule. If you want it out, remove the rule too and add noindex, because Google can't see a noindex on a page it isn't allowed to crawl.
Google: “The page was indexed despite being blocked by your website's robots.txt file.” Page indexing report help
What the status means
The page is in Google's index, but Google hasn't read it. Your robots.txt blocks crawling, and Google indexed the URL anyway from what other pages say about it, such as links pointing to it.
Google's help explains why both can be true: it always respects robots.txt, but that doesn't prevent indexing if someone links to the page. The URL can appear in results with a very limited snippet or none at all.
Is it a problem?
The report only asks one question: do you want this page in search?
| You want the page… | What's wrong now | What to do |
|---|---|---|
| In search | Google can't read it, so it ranks on links alone and has no proper snippet | Remove the robots.txt rule |
| Out of search | It's already there, and the rule stops Google from seeing a noindex | Remove the rule and add noindex |
| Don't care (junk parameter URLs with few impressions) | Little | Leave it, or clean up the links that create these URLs |
Check impressions before you spend time on it. In Performance, filter by the URL or folder. A URL with no impressions costs you nothing today.
How to find the cause
- Open Indexing → Pages and find Indexed, though blocked by robots.txt in the warnings section. Export the examples and group them by folder or pattern.
- Find the rule. Paste the file and the URL into our robots.txt tester to see which line blocks it.
- Find what points to the URL. Check your internal links and your sitemap first: those you control. A URL in the sitemap and in a
Disallowrule sends Google two opposite signals.
Common cases
A rule added to “remove” pages. Someone wanted old pages gone and added a Disallow. Those pages can stay indexed, because the block stops Google from seeing that anything changed. This is the most useful case to catch.
WordPress. The usual URLs are internal search (/?s=), tag and archive pages, and parameter URLs blocked by hand-written rules or an SEO plugin. Also look for /wp-admin/admin-ajax.php, which sometimes gets linked by themes and plugins.
Shopify. Filtered collection URLs and internal search pages. The default robots.txt blocks them, and the theme's filter links keep exposing them. If they get no impressions, they usually aren't worth a custom theme change.
Pages you actually want. A service page or article blocked by an over-broad rule. These often look “fine” because they appear in search, while they rank far below what they could.
How to fix it
To keep the page in search
- Delete or narrow the rule, and check the result in the tester.
- Test the URL live in URL Inspection, then request indexing for the few that matter most.
To take the page out of search
- Add a noindex directive: a robots meta tag or the
X-Robots-Tagheader. - Remove the robots.txt rule. This is the step people skip. Google's help is direct: to avoid being indexed, remove the block and use noindex.
- Keep the URL crawlable until it's gone from the index. The page moves to Excluded by 'noindex' tag.
- Only then, if you need to save crawling on a big set of URLs, consider blocking them again. New links can bring them back the same way.
For both: remove the blocked URLs from your sitemap, unless you're using the sitemap temporarily to get noindex URLs recrawled.
How to set noindex by platform
- WordPress: set the page to noindex in Yoast (Advanced → Allow search engines to show this content in search results?) or Rank Math (Advanced → Robots Meta).
- Shopify: use the
seo.hiddenmetafield on the product, page or collection. - Wix: turn off Let search engines index this page in the page's SEO settings.
- Squarespace: use Hide page from search results in the page's SEO settings.
- Next.js: set
robots: { index: false }in the page's metadata, or sendX-Robots-Tagfrom the server.
After fixing
Run Validate fix for the issue. Google says validation typically takes up to about two weeks, and in some cases much longer. URLs leave this status only after Google recrawls them, so old pages with few links can take a while.
What not to do
- Don't add noindex and keep the block. Google can't read a tag on a page it isn't allowed to fetch.
- Don't treat the Removals tool as the fix. It hides URLs temporarily and doesn't change what happens once the removal expires.
- Don't remove blocks in bulk without checking what they protect. Unblocking internal search or filter URLs can open up a huge number of low-value pages to crawling.
Check a URL
Paste one of the URLs from the report. The checker follows redirects and tests the status code, robots.txt, noindex and canonical.
Related statuses
- DependsBlocked by robots.txtA robots.txt rule stops Google from crawling the page.
- DependsExcluded by 'noindex' tagA noindex directive tells Google not to index the page.
- DependsCrawled - currently not indexedGoogle fetched the page but hasn't added it to the index.
FAQ
robots.txt controls crawling, not indexing. Google's help says it respects the rule and won't crawl the page, but it can still index the URL using information from pages that link to it.
It isn't a penalty. The cost is limited to the listed pages: Google can't read them, so their snippets are very limited and they can't rank on their content.
No. Google's documentation says the page must not be blocked by robots.txt for noindex to work, because the crawler never sees the directive otherwise.
Want us to look at your Search Console?
We go through the Pages report, crawl the site and tell you which statuses cost you traffic and which are harmless, with the fixes in priority order.