[prev in list] [next in list] [prev in thread] [next in thread] 

List:       python-catalog-sig
Subject:    Re: [Catalog-sig] PyPI down
From:       "Thomas Wouters" <thomas () python ! org>
Date:       2007-08-01 20:45:30
Message-ID: 9e804ac0708011345oefd1f98rba1870ff8f493720 () mail ! gmail ! com
[Download RAW message or body]

[Attachment #2 (multipart/alternative)]

[Attachment #4 (text/plain)]

On 8/1/07, "Martin v. Löwis" <martin@v.loewis.de> wrote:
>
> > Fixed, I think. The problem was actually a screen-scraper searching the
> > wiki (a third of the last 100k hits on the wiki were from the same host,
> > requesting every possible link on every possible page.) Someone with
> > more wiki knowledge may want to make sure no weird pages were made. (The
> > offending host was somewhere in dynamic.dsl.as9105.com
> > <http://dynamic.dsl.as9105.com>; I have the actual address but don't
> > want to mail it around.)
>
> Strange. According to the logs, it also happened (simultaneously,
> but independently?) that pypi.fcgi would always terminate
> immediately. This happened before, and I could never figure out
> why, so I added a mechanism to restart Apache. This time, the
> restart happened, and pypi.fcgi would again stop right away,
> to be restarted again, and so on.
>
> If this is indeed related to the load on MoinMoin, this is quite
> puzzling: they are separate processes, and separate virtual hosts.
> So they shouldn't "see" each other.


The load is machine load, which is of course shared across all processes. It
was about 15, with a slow response to match. Nullrouting that particular IP
address fixed the problem instantly, so I'm pretty sure that was it.

-- 
Thomas Wouters <thomas@python.org>

Hi! I'm a .signature virus! copy me into your .signature file to help me
spread!

[Attachment #5 (text/html)]

<br><br><div><span class="gmail_quote">On 8/1/07, <b \
class="gmail_sendername">&quot;Martin v. Löwis&quot;</b> &lt;<a \
href="mailto:martin@v.loewis.de">martin@v.loewis.de</a>&gt; wrote:</span><blockquote \
class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt \
0pt 0.8ex; padding-left: 1ex;"> &gt; Fixed, I think. The problem was actually a \
screen-scraper searching the<br>&gt; wiki (a third of the last 100k hits on the wiki \
were from the same host,<br>&gt; requesting every possible link on every possible \
page.) Someone with <br>&gt; more wiki knowledge may want to make sure no weird pages \
were made. (The<br>&gt; offending host was somewhere in <a \
href="http://dynamic.dsl.as9105.com">dynamic.dsl.as9105.com</a><br>&gt; &lt;<a \
href="http://dynamic.dsl.as9105.com"> http://dynamic.dsl.as9105.com</a>&gt;; I have \
the actual address but don&#39;t<br>&gt; want to mail it around.)<br><br>Strange. \
According to the logs, it also happened (simultaneously,<br>but independently?) that \
pypi.fcgi  would always terminate<br>immediately. This happened before, and I could \
never figure out<br>why, so I added a mechanism to restart Apache. This time, \
the<br>restart happened, and pypi.fcgi would again stop right away,<br> to be \
restarted again, and so on.<br><br>If this is indeed related to the load on MoinMoin, \
this is quite<br>puzzling: they are separate processes, and separate virtual \
hosts.<br>So they shouldn&#39;t &quot;see&quot; each other. </blockquote><div><br>The \
load is machine load, which is of course shared across all processes. It was about \
15, with a slow response to match. Nullrouting that particular IP address fixed the \
problem instantly, so I&#39;m pretty sure that was it. <br></div></div><br>-- \
<br>Thomas Wouters &lt;<a \
href="mailto:thomas@python.org">thomas@python.org</a>&gt;<br><br>Hi! I&#39;m a \
.signature virus! copy me into your .signature file to help me spread!



_______________________________________________
Catalog-SIG mailing list
Catalog-SIG@python.org
http://mail.python.org/mailman/listinfo/catalog-sig


[prev in list] [next in list] [prev in thread] [next in thread]