How to stop search engines from crawling a file or folder using Robots.txt


Robots.txt is a file which instructs search engine crawlers, spiders and robots which part of the site to be indexed and which part of the site not be indexed.

Basically you don’t need a robots.txt file unless you have content that you don’t want Google or any other search engines to index.  Don’t create a robots.txt file if you like search engines to crawl your entire website.

Using robots.txt file you can easily eliminate a URL from being indexed in search engine search results. Below are some helpful commands that can used to Disallow files and folders on your website in your robots.txt file.

continue reading

Top Domain Auction Sites

Domain Auction sites allow you to buy, sell, bid & search domains listed for sale by owners of the domain. Most domain auction websites allow you to either bid on a listed domain or buy a fixed price domain.

This websites will also allow an owner to list online business with domain for sale. In such cases some auction website will do a free website appraisal that can be viewed by the buyer.

When a buyer interested to buy a domain or an online business the auction website will always act as a mediator for the buyer and seller to make sure the transaction is smooth. The auction website will take a small percentage of the total transaction amount as a service charge.

If you want to sell your domain or online business you may need to consider the below auctions websites where you can list your domain or online business.

continue reading

Top Search Engines

Search Engines are the index page of internet. Most of us know Google as a search engine for a long time but there are also other search engines that are interesting to search the web, images, video & news.

If you are doing SEO for your website or a blog you may be interested to see how your keyword performing on different searches engines. Here is a list of top search engines by page rank.
continue reading

Top 100 Article Submission Sites

Article Submission is one of the excellent strategies to promote your business or product. A well written article may bring you high traffic to your website. Article submission seems easy but there are some vital points you should keep in mind while submitting an article.

continue reading

Top 100 Social Bookmarking Sites

Social Bookmarking is one of the great ways to brings lots of traffic to your website. While Social bookmarking you should keep few of the below points in mind that will bring you fortune (traffic).

continue reading