Robots.txt Linter
Free toolPaste your robots.txt and a list of URLs. For each one, we show whether it is allowed or blocked, the exact line and pattern that wins, and why. The linter also catches real mistakes: accidental full-site blocks, "noindex" in robots.txt (Google has not supported it there since 2019), sitemaps with an invalid URL, and contradictory rules. Everything is processed in your browser: nothing is uploaded to RA Software Apps servers, and the sitemap URL is only checked for valid syntax, never fetched.
Type a user-agent (e.g. "Googlebot") or leave "*" for every crawler.
Simulation results
| URL | Status | Rule | Why |
|---|
Linter findings
Report
Markdown report
How to use it
- Paste the contents of your robots.txt.
- Add the URLs or paths you want to simulate, one per line, and choose the user-agent (for example Googlebot, or * for all).
- Read the verdict for each path, with the winning line and pattern, plus the file-wide warnings.
What it works out and how
For each path, the linter applies the rules of the matching user-agent group and resolves conflicts the way search engines do: the most specific rule wins, meaning the one with the longest pattern. It also catches common mistakes, such as a Disallow that blocks the whole site, noindex directives inside robots.txt, sitemaps with an invalid URL or rules that contradict each other.
Frequently asked questions
Does noindex work inside robots.txt?
No. Google stopped supporting it in 2019. To remove a page from the index, use the meta robots tag or the X-Robots-Tag header, and keep the page crawlable.
If I block a URL in robots.txt, does it disappear from Google?
Not necessarily. Robots.txt prevents crawling, not indexing: a blocked URL can still show up if other pages link to it.
Does the tool fetch my sitemap?
No. It only checks that the sitemap URL has valid syntax; it makes no network request.