robots.txt: how to check it and not lose traffic
Why check robots.txt
A single wrong line in robots.txt can block the entire site from indexing — the classic case is a “Disallow: /” left over after development. The tool parses the file line by line and highlights dangerous and invalid directives.
It checks User-agent groups, Allow/Disallow rules, Clean-param, Crawl-delay and Sitemap links, as well as common syntax errors.
Practical recommendations
Block service sections from crawling: the cart, the account area, on-site search results, pages with sorting parameters. Do not block CSS and JS — search engines need them to render pages correctly.
After changing the file, test it in Yandex Webmaster and Google Search Console: there you can check a specific URL against the current rules.