|
View previous topic
::
View next topic
|
| Author |
Message |
sarah.brooks
Joined:
11 Apr 2011
Posts:
137
|
Posted: Wed May 18, 2011 6:48 pm
Post subject: robots.txt
|
 |
|
Robots.txt is a regular text file that through its name, has special meaning to the majority of "honorable" robots on the web. By defining a few rules in this text file, you can instruct robots to not crawl and index certain files, directories within your site, or at all. For example, you may not want Google to crawl the /images directory of your site, as it's both meaningless to you and a waste of your site's bandwidth. "Robots.txt" lets you tell Google just that.
It is important to clarify that robots.txt is not a way from preventing search engines from crawling your site i.e. it is not a firewall, or a kind of password protection and the fact that you put a robots.txt file is something like putting a note “Please, do not enter” on an unlocked door – e.g. you cannot prevent thieves from coming in but the good guys will not open to door and enter. That is why we say that if you have really sen sitive data, it is too naïve to rely on robots.txt to protect it from being indexed and displayed in search results.
_________________
Sarah Brooks
Affiliate Manager
BuzzCreators.com - Premium CPA Network |
|
| Back to top |
     |
 |
carl16
Joined:
02 Jun 2011
Posts:
6
Location:
Canada
|
Posted: Fri Jun 03, 2011 5:45 am
Post subject:
|
|
|
Hello Friends........
Robots.txt is a text (not html) file you put on your site to tell search robots which pages you would like them not to visit.
Thanks
_________________
Brampton Real estate agent |
|
| Back to top |
 |
|
georgebiran
Joined:
01 Jul 2011
Posts:
4
|
Posted: Fri Jul 01, 2011 7:53 am
Post subject:
|
|
|
Robots.txt is a regular text file that through its name, has special intending to the giant majority of honorable robots on the net. By defining a few rules in this text file, you can instruct robots to not crawl and index definite files, directories within your site, or at all.
_________________
fendall 1000 |
|
| Back to top |
|
|
James-Smith2011
Joined:
10 Oct 2011
Posts:
3
|
Posted: Mon Oct 10, 2011 9:46 am
Post subject:
|
|
|
| Robot.txt is a text file that is used in a website in order to hide pages. The pages where robot.txt is applied, don't show. |
|
| Back to top |
|
|
mojicerock
Joined:
24 Nov 2011
Posts:
2
|
Posted: Thu Nov 24, 2011 9:18 am
Post subject:
|
|
|
The Benefits of the Robots.txt file are:
- Avoid the things that not be shown to search engine.
- Safe guard from the spam spot.
- You can easily handle what to show Google..
_________________
Send Christmas flowers to India | Christmas Gifts for 2011 |
|
| Back to top |
|
|
StangMan23
Joined:
10 Feb 2012
Posts:
8
Location:
Saint George, UT
|
Posted: Thu Mar 01, 2012 6:12 pm
Post subject:
|
|
|
Are there other methods other than robots.txt to prevent search engine bots from crawling sections of your site?
_________________
Harness the power of mobile marketing and make money as a mobile marketing affiliate! Sign up free! |
|
| Back to top |
|
|
lupinesteve
Joined:
15 May 2012
Posts:
13
|
Posted: Thu May 17, 2012 7:29 am
Post subject:
|
|
|
Basically, robots.txt is a plain text file which is placed in a server's root directory it includes information on whether search engine robots should index the site or parts of the site. This text file contains commands recognized by the search engines that you'll use as rules - rules for handling your site that will help you promote it.
_________________
Magento Customization | Magento Module Development |
|
| Back to top |
|
|
cloud_server
Joined:
02 Jan 2013
Posts:
3
|
Posted: Thu Jan 03, 2013 9:55 am
Post subject: Reply
|
|
|
I gotta bookmark this post of this it seems extremely helpful and very much handy. Actually You have brought up a very good details, thanks you a lot and I truly appreciate this post. Thanks a lot my friend.
_________________
cloud server
http://www.hostwinds.com/cloud-server.html |
|
| Back to top |
|
|
martinsherman
Joined:
23 Nov 2012
Posts:
33
Location:
India
|
Posted: Sat Jan 12, 2013 7:01 am
Post subject:
|
|
|
The robots.txt file tells search engines which directories to crawl and which not to. You can use it to block crawlers from looking at your image directory if you don't want your images showing up on google search.
Be careful not to use this to try and block people from directories you want to keep secret. Anyone can view you robots.txt file. Make sure you password protect directories that need to be secured.
_________________
Womens Clothing Online | Tops for Women |
|
| Back to top |
|
|
| View previous topic :: View next topic |
|
FAQ
Search
Memberlist
Usergroups
Register
Profile
Private Messages
Log in
