robots.txt Generator
Create robots.txt with indexing rules, Disallow directives and a sitemap.xml link.
Settings
Generated robots.txt
User-agent: *
About this tool — robots.txt Generator
Create robots.txt with indexing rules, Disallow directives and a sitemap.xml link.
Enter your domain, choose a base indexing rule and add custom blocked paths. Get a ready robots.txt file for Google, Yandex and other search crawlers. The tool works directly from the DEVNEW.ORG web page and requires no separate software installation. It is useful for website owners, developers, SEO specialists, designers and anyone who needs a fast online utility. Use it free for work or personal tasks.
See also
sitemap.xml Generator
Create an XML sitemap automatically from a website URL or from your own page list.
Open toolMeta Tags Generator
Create Meta Title, Description, Robots, Canonical and Open Graph tags for web pages.
Open toolWebsite Speed Checker
Check HTTP status, connection time, TTFB, HTML load time and response size for a website.
Open tool