Robots txt file generator
This is a structured file that tells a search engine the pages to crawl or ignore. You can make robots.txt using an online robots.txt generator or by a plugin like Yoast SEO. Upload it to the root directory and test it in Google Search Console. This ensures perfect control of indexing.