Free SEO tool
Robots.txt generator
Create a basic robots.txt file with an open crawl rule and a sitemap reference for a public website.
Before you start
One check, three clear steps
Run the check
Generate robots.txt
Use public URLs only. Do not submit private admin pages, tokens or internal services.
Your result will appear here
You will get a ready-to-review robots.txt template.
How to use the result
Follow the check in order
- Enter the canonical website address.
- Generate the file and review every rule.
- Publish it at /robots.txt and verify it with the checker.
Next checks
Continue with a related tool
Robots.txt checker
Check robots.txt for risky Disallow rules, a Sitemap directive and crawl instructions that may hide important pages.
Open tool →Next checkSitemap.xml generator
Create a simple sitemap.xml from a list of canonical page URLs.
Open tool →Next checkSitemap.xml checker
Validate sitemap XML, loc entries and URL format before submitting the file to Google Search Console or Yandex Webmaster.
Open tool →FAQ
Common questions
What should I enter?
Use a public production URL or the exact text requested by the form. Do not enter private admin pages, tokens or internal addresses.
Is every recommendation critical?
No. Fix issues that block crawling, indexing, page understanding or the user path first. Treat the rest as improvements.
How do I confirm the fix?
Run the same check again for the same URL after publishing the change.