This is the mail archive of the
gcc@gcc.gnu.org
mailing list for the GCC project.
Re: robots.txt
- To: Jason Molenda <jsm at cygnus dot com>, egcs at egcs dot cygnus dot com
- Subject: Re: robots.txt
- From: Gerald Pfeifer <pfeifer at dbai dot tuwien dot ac dot at>
- Date: Thu, 11 Mar 1999 23:10:38 +0100 (MET)
- cc: egcs-patches at egcs dot cygnus dot com
On Wed, 10 Mar 1999, Gerald Pfeifer wrote:
> How about the following for now?
I have now installed the following robots.txt file for our egcs.cygnus.com
site as wwwdocs/htdocs/robots.txt.
---- cut ----
# See http://info.webcrawler.com/mak/projects/robots/norobots.html
# for information about the file format.
# Contact egcs@egcs.cygnus.com for questions.
User-Agent: *
Disallow: /cgi-bin/
---- cut ----
Thanks to Jason for his comments...
Gerald
--
Gerald "Jerry" pfeifer@dbai.tuwien.ac.at http://www.dbai.tuwien.ac.at/~pfeifer/