Crawling and indexing are separate steps, and most mistakes in this area come from treating them as one. Robots.txt controls whether a crawler may fetch a URL. It does not control whether that URL is indexed: a blocked page can still appear in results, without a description, if enough links point to it. Keeping a page out of the index takes a noindex directive, and the crawler has to be allowed to fetch the page in order to see it.
The tools follow the order of those decisions. The robots.txt generator writes the file from presets, including rules for AI crawlers such as GPTBot, and the robots.txt validator tests a URL against a user-agent using the longest-match rule from RFC 9309, which is why Disallow: / followed by Allow: /public/ does what it appears to. The meta robots tag generator builds noindex, nofollow and the other directives as a meta tag for HTML pages, or as an X-Robots-Tag header for PDFs, CSV exports and other files that have no head to put a tag in.
The sitemap generator turns a list of URLs into a valid sitemap.xml. List only canonical URLs that return 200 and are meant to be indexed, and give lastmod only when the date is true: Google uses it when it is consistently accurate, so a sitemap that stamps every URL with the build date throws away the one signal it had. The canonical tag generator normalises a URL into a rel=canonical line. Protocol, host, trailing slashes and tracking parameters are where duplicate URLs usually come from, and a canonical is a hint that search engines weigh against everything else, not an order.
Two tools handle sites that exist in more than one form. The hreflang generator builds a reciprocal set of alternates, because hreflang only works when every language version lists all the others and itself; a one-way annotation is ignored. The htaccess redirect generator writes Apache and Nginx rules side by side for the moves that come up most — HTTP to HTTPS, www to the bare domain, a renamed page. Use a 301 when the move is permanent, and point it straight at the final URL rather than through a chain of hops.