Answer:
The "robots.txt" file in web development is used to instruct web robots (search engine crawlers) about which parts of the site they should or should not access.
The "robots.txt" file in web development is used to instruct web robots (search engine crawlers) about which parts of the site they should or should not access.
You may be interested in:
Web Fundamental MCQs