[prev in list] [next in list] [prev in thread] [next in thread] 

List:       wikitech-l
Subject:    Re: [Wikitech-l] Wikipedia blocks grub
From:       "Wikinews Markie" <newsmarkie () googlemail ! com>
Date:       2007-10-27 10:56:23
Message-ID: a6424d870710270356w51207391h9dc875e99c6e1e4f () mail ! gmail ! com
[Download RAW message or body]

hheh nice.  however in the robots.txt file it also has the following

# Don't allow the wayback-maschine to index user-pages
#User-agent: ia_archiver
#Disallow: /wiki/User
#Disallow: /wiki/Benutzer

but its all coded out.  intended??

Mark

On 10/27/07, Mathias Schindler <mathias.schindler@gmail.com> wrote:
>
> On 10/27/07, Jay R. Ashworth <jra@baylink.com> wrote:
> > On Fri, Oct 26, 2007 at 08:21:17PM -0400, Anthony wrote:
> > > Heh, I just noticed that Wikipedia's robots.txt blocks grub.  For
> > > those who don't get the joke, Jimbo's company owns grub.
>
> As far as I know, the grub block was instated in 2004 or 2003 when
> grub was usually very bad-behaving in terms of listening to robots.txt
> advises.
>
> Mathias
>
> _______________________________________________
> Wikitech-l mailing list
> Wikitech-l@lists.wikimedia.org
> http://lists.wikimedia.org/mailman/listinfo/wikitech-l
>
_______________________________________________
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
http://lists.wikimedia.org/mailman/listinfo/wikitech-l
[prev in list] [next in list] [prev in thread] [next in thread]