Re: [ga] DNSO.org web site and robots.txt

[Marc Schneiders <marc@fuchsia.bijt.net>]

On Sat, 18 May 2002, at 13:29 [=GMT-0700], George Kirikos wrote:

> I was curious as to why the DNSO website has a very restrictive
> robots.txt file:
>
> http://www.dnso.org/robots.txt
>
> With the top 3 lines of:
>
> User-agent: *
> Disallow: /constituency/work/
> Disallow: /clubpublic/
>
> it means that search engines like Google can't index the website
> (including mailing lists), and since the website itself lacks search
> capabilities, it's hard to find material (e.g. old posts) quickly.
>
> PROPOSAL: That the DNSO webmaster delete the current robot.txt file
> entirely.

I am not so sure I agree. I do see the advantages. On the other hand it
means opening up email addresses for spam harvesting, as well as
sacrificing all posters privacy. It might make people think twice before
posting. (I know, some should :-)

--
This message was passed to you via the ga@dnso.org list.
Send mail to majordomo@dnso.org to unsubscribe
("unsubscribe ga" in the body of the message).
Archives at http://www.dnso.org/archives.html