Free SEO tool
Robots.txt checker
Check robots.txt for risky Disallow rules, a Sitemap directive and crawl instructions that may hide important pages.
Before you start
One check, three clear steps
Run the check
Check robots.txt
Use public URLs only. Do not submit private admin pages, tokens or internal services.
Your result will appear here
You will see whether the file blocks the whole site, includes a sitemap and contains the expected directives.
How to use the result
Follow the check in order
- Enter the website URL, robots.txt URL or paste the file.
- Check whether Disallow: / blocks the entire site.
- Confirm important URLs are also indexable by status, canonical and robots meta.
Next checks
Continue with a related tool
Robots.txt generator
Create a basic robots.txt file with an open crawl rule and a sitemap reference for a public website.
Open tool →Next checkSitemap.xml checker
Validate sitemap XML, loc entries and URL format before submitting the file to Google Search Console or Yandex Webmaster.
Open tool →Next checkFree SEO audit for a web page
Check title, meta description, H1, canonical, robots meta, Open Graph, links and HTTP status for one public URL.
Open tool →FAQ
Common questions
What should I enter?
Use a public production URL or the exact text requested by the form. Do not enter private admin pages, tokens or internal addresses.
Is every recommendation critical?
No. Fix issues that block crawling, indexing, page understanding or the user path first. Treat the rest as improvements.
How do I confirm the fix?
Run the same check again for the same URL after publishing the change.