One per line
User-agent: * Allow: / Disallow: /admin Disallow: /api Disallow: /private Sitemap: https://example.com/sitemap.xml
How to use
01
Configure your rules
Choose what to allow and what to block.
02
Add sitemap URL
Point crawlers at your sitemap.xml.
03
Save as robots.txt
Copy the result and upload to your site root.
Why Robots.txt Generator
- One-click block for AI crawlers (GPTBot, ClaudeBot, Google-Extended, CCBot).
- Builds valid robots.txt syntax, no typos.
- Includes sitemap declaration automatically.
Common questions
Is Robots.txt Generator free to use?
Yes. Robots.txt Generator is completely free. There is no sign up, no trial, and no usage cap.
Do I need an account to use Robots.txt Generator?
No. You can start using Robots.txt Generator straight away. We do not ask for an email address or a login.
Is my data uploaded to a server when I use Robots.txt Generator?
No. Robots.txt Generator runs entirely inside your browser, so the URL or details you enter never leaves your device and nothing is sent to us.
Do you store the URL or details I use with Robots.txt Generator?
No. Because everything happens locally in your browser, there is nothing for us to store. Close the tab and it is gone.
Will using Robots.txt Generator change my website?
No. Robots.txt Generator only reads and analyzes. It never edits or publishes anything on your site.
Is the data from Robots.txt Generator fetched live?
Yes. Robots.txt Generator works from the current version of the page or resource you point it at.
How do I use Robots.txt Generator?
Enter or paste your URL or details and Robots.txt Generator updates the result as you go. When it looks right, copy or download it with one click.
Does Robots.txt Generator work on phones and tablets?
Yes. Robots.txt Generator is responsive and works in any modern browser on mobile, tablet, and desktop.
Which browsers does Robots.txt Generator support?
It works in current versions of Chrome, Firefox, Safari, and Edge. For the best experience, keep your browser up to date.
Is there a limit on how often I can use Robots.txt Generator?
No. You can use Robots.txt Generator as many times as you need, at no cost.
Can I use Robots.txt Generator for commercial or client work?
Yes. You are free to use the output in personal, educational, and commercial projects.
Why is Robots.txt Generator free?
The site is supported by advertising, which lets us keep every tool free and open to everyone.
What should I do if Robots.txt Generator gives an unexpected result?
First check that your URL or details is formatted as expected. If something still looks wrong, use the feedback link on the page and we will take a look.