Many bloggers thought is robot.txt and and what is their function, and mostly thought that this just the formality nothing else, even once i thought that this nothing just the waste of time, then i did research and got a very interesting and important knowledge about Robot.txt file.
Work Of ROBOT.TXT File
The work of Robot.txt file is very important and it should be implemented by every blogger for their healthy traffic, YES! you heard it right it helps you to provide healthy traffic for your site.The main wok of Robot.Txt file is to stop the robots to crawl all the pages of your sit or you may say it a it gives path to the robots of search engine that which pages to crawl and which page to notcrawl.If the robots of search like Google,Bing or yahoo will crawl our whole site than it might create duplicate pages and your site may got banned.
How to add Robot.Txt File
Here are the instruction that how to add Robot.txt files in your blogger and how does it look alike.Below is the examplae of robots.Just replace my url with your site's url and you are done with your robot.txt file.Now you need to upload this Robot.Txt file in your blogger directory.
Instruction
- First Go to the blogger dashboard
- Select your site
- Edit Setting
- Go into Search preference
- Go down and find Crawlers and Indexing
- Enable Robot.Txt
- Copy the following code and replaace our url with your url and paste it in Robot.txt file
User-agent:Mediapartners-Google
Disallow:
User-agent: *
Disallow: /Ssearch
Allow: /
Sitemap: http//www.freedsoftwares.blogspot.com/feeds/posts/default?orderby=UPDATED
8.Replace our URL with your url and you are done
Advantages Of Robot.Txt File
Their are many advantages of robot.txt file but the main one is to guide te robots that which pages should be crawled and which one should no be crawled.And the second advantage is that it makes your site omptimized and gives you massive and healthy traffic for your site.If you will not put this Robot.txt file in your blog then it will definately decrease and minimize your google rank, So it is very important for the site admin to place these Robot.txt file in their site directory.I have mentioned many time above my post that if you will not gonna put it in your blog directory then the search engine robots will index the all pages of your site and it will create duplicate content and it will definately effect your site's ranking badly.The above Robot.Txt file is the best of way Robot.Txt fille given by Google.So FreedSoftwares will recommend you to use this Robot.txt file in blog directory.
Codes Description
User-agent: means the Google search, search robots and the sign (*) is used for all the search engine's Robots like: Google, Bing,Yahoo e.t.c.
Disallow: That means that you don't want the search engine's robot to crawl a particular page of your site.
Description Of Disallow:The Disallow, code stop the robots to crawl the particular pages on your site, means if you don't want search engine to index a particular page on your site then you need to put this disallow in following method.See the following example for more clarification.
User-agent:*
Disallow:/labels.html
In above example of Robot.txt file that is disallowing labels means it will not index your labels on your site.
This was some brief description about the code of Robot.txt file by FreedSoftwares.The above Robot.txt file is best one we will recommend you to use that one by the replaced URL of yours.
For more clarification feel free to contact or ask us in comments we will get back to your question very soon.
