[prev in list] [next in list] [prev in thread] [next in thread] 

List:       kde-devel
Subject:    Re: robots.txt in quickgit.kde.org
From:       Ben Cooksley <bcooksley () kde ! org>
Date:       2015-12-27 11:08:07
Message-ID: CA+XidOEKBv0a5KLidZyDQP4w=cvVUmXLTZe6aSinY-9tos2y9g () mail ! gmail ! com
[Download RAW message or body]

On Sun, Dec 27, 2015 at 11:53 PM, Ashish Bansal
<bansal.ashish096@gmail.com> wrote:
> Hi everyone,

Hi Ashish,

>
> "quickgit.kde.org" contains robots.txt[0] which is disallowing search
> engines to fetch the project repos. I just wanted to know if this is
> intentional or not?
>
> If I recall correctly, mirror of kde repositories on github was created just
> because it wasn't being indexed by the search engines.

This is intentional, and is done to reduce the server load created by
indexers such as Google on the system hosting quickgit.kde.org.
(Generation of the pages, including the main index is substantially
more expensive than it appears due to the disk access required by
Git/SVN to return the needed information).

>
> [0] https://quickgit.kde.org/robots.txt
>
> --
>
> Regards,
> Ashish Bansal
> http://ashish-bansal.in

Regards,
Ben Cooksley
KDE Sysadmin

>
>
>>> Visit http://mail.kde.org/mailman/listinfo/kde-devel#unsub to unsubscribe
>>> <<
>

>> Visit http://mail.kde.org/mailman/listinfo/kde-devel#unsub to unsubscribe <<

[prev in list] [next in list] [prev in thread] [next in thread]