gcc.gnu.org and IBrowse ("spambot detected" problem)

[Joe Buck]

On Thu, Nov 04, 2004 at 11:02:33AM +0100, Christian Rosentreter wrote:
> "Hi, your web browser has triggered our spambot detection 
> mechanism. A spambot is a computer program designed to 
> scoop up e-mail addresses and bombard them with unsolicited 
> advertisements."
> ...
> this is what I get when I browse with IBrowse (well known
> Webbrowser for AmigaOS/ MorphOS) on gcc.gnu.org.

This happens, I believe, when a hidden link is followed on the
web page, and the robots.txt file forbids spiders from visiting it.
The theory is that no person would follow a link they can't see,
and no search engine would either, so it is likely that someone
has written a spider program that violates the rules, for purpose
like address harvesting.

> Is Internet Explorer required to visit this site now? Sadly
> I don't have a Windows compatible Personal Computer to find
> out.

Of course not.

> It would be nice if you/ someone could fix that. Thanks. 
> 
> Browser ID is: "IBrowse/2.3 (MorphOS 1.4)" or "IBrowse/2.3 
> (AmigaOS 3.1)"

While an exception could be created for that browser, if the problem is
that your browser is pre-fetching every link, opening a hole for it would
simply mean that all the bad guys could have their robots falsely identify
themselves as IBrowse.