robots.txt

[Gerald Pfeifer]

On Wed, 10 Mar 1999, Gerald Pfeifer wrote:
> How about the following for now?

I have now installed the following robots.txt file for our egcs.cygnus.com
site as wwwdocs/htdocs/robots.txt.

---- cut ----
# See http://info.webcrawler.com/mak/projects/robots/norobots.html
# for information about the file format.
# Contact egcs@egcs.cygnus.com for questions.

User-Agent: *
Disallow: /cgi-bin/
---- cut ----

Thanks to Jason for his comments...

Gerald
-- 
Gerald "Jerry" pfeifer@dbai.tuwien.ac.at http://www.dbai.tuwien.ac.at/~pfeifer/