SEO blog post: "robots.txt tutorial"
One of my SEO blog posts. Based on content curation from the web, my own experience and my clients' feedback. Text preview: "It's amazing how one little text file can make or break your website. If you have a line or two wrong in your robots.txt file, you might actually be telling the search engine robots to not crawl and index your site...which means your web pages won't show up in the search engines. Luckily it's easy to check, and easy to solve. In this blog post you'll learn about what robots.txt is, how to check your own robots.txt, and how you can improve your instructions for the search engine robots." For the full blog post, see http://fortheloveofseo.com/2012/tutorial-robots-txt-your-guide-for-the-search-engines/