What is robots.txt ?

Gurmeet Kaur

Total Post:46

Points:324

Posted by  Gurmeet Kaur
SEO  SMO 
 1048  View(s)
Ratings:
Rate this:

Search Engine Optimization 

  1. Re: What is robots.txt ?

    The Robots.txt is one way of telling the Search Engine Bots about the web pages on your website which you do not want them to visit.

    A Robots.txt is useful for preventing the indexation of the parts of any online content that website owners do not want to display.

    IF we want to block all search engine robots from crawling our website, just put the following code:

    User-agent: *

    Disallow: /

    IF we want to block Google from crawling our website, just put the following code:

    User-agent: Googlebot

    Disallow: /

    This is important to decide the location of Robots.txt very carefully, or else errors might occur while displaying the website.

      Modified On Aug-31-2019 05:30:43 AM
  1. Re: What is robots.txt ?

    Robots.txt is a text file webmasters create to instruct web robots how to crawl pages on their website.

  1. Re: What is robots.txt ?

    Robots.txt is one way of telling the Search Engine Bots about the web pages on your website which you do not want them to visit.

    https://sport11.in/

Answer

Please check, If you want to make this post sponsored

You are not a Sponsored Member. Click Here to Subscribe the Membership.