Before i addressing you about Robots.txt,firstly you have to know why it needs to your website ? The simple and smartest answer is some what tricky but easy, to secure your website pages by notifying to search engine that, you don't want to it crawl your important website pages or blog pages . cautious or note : Just use this feature for getting your website pages secure other wise your website or blog ignored by search engine's with out indexing your websites .If it happens, obviously you may miss a lot of traffic to your website from user to access their useful information. For example: Your website pages having some parts of valuable information like log-in and password pages or word-press admin log in pages.so you are not supposed to allow these pages to all visitors and these are secured pages.we need to protect it by adding Robots.txt to your pages, from this you are not allow search engines to crawl or index your pages. and make important pages ...