Tag Archives: robots.txt

Q & A: What is a robot file? 0

http://bit.ly/mhiopZ

The robots.txt file sits on your server at this address www.mysite.com/robots.txt and tells search engines what they are, and aren’t allowed to do on your website. A robots file was traditionally used my websites who didn’t want their pages appearing in a search index. With a robots.txt file you can speak to all robots or [...]