[prev in list] [next in list] [prev in thread] [next in thread] 

List:       opensuse
Subject:    Re: [opensuse] One repo in the pool killing updates - probably utexas.edu - can it be fixed?
From:       Peter_Pöml <peter () poeml ! de>
Date:       2010-09-22 9:23:15
Message-ID: DA9287A4-0966-47BF-908F-BD2474A5C00B () poeml ! de
[Download RAW message or body]

Hi David,

Le 21.09.2010 à 22:30, David C. Rankin a écrit :

> On 09/21/2010 04:24 AM, Peter Pöml wrote:
> > David,
> > 
> > Le 21.09.2010 à 00:18, David C. Rankin a écrit :
> > > Guys,
> > > 
> > > 	It is literally like Russian-Roulette updating openSuSE with the mirror pool \
> > > you get with download.opensuse.org. There is at least 1 maybe more mirrors that \
> > > give no more than 1-4K/s download speed. (utexas.edu is a know problem repo). \
> > > This is getting ridiculous. I have had to kill 3-4 updates in the past 2-3 days \
> > > when I hit one of these mirrors. It would take more than an hour to download an \
> > > 800k file. 
> > > 	I have searched zypper.log and can't find the problem mirror this time. Where \
> > > to look? Also, this has been a problem for more than a year, why aren't these \
> > > mirrors being dropped or fixed? I end up having to hard-code the mirror I want \
> > > just to get reasonable consistent updates. Something should be done to correct \
> > > this.
> > 
> > Two questions:
> > 
> > Have you reported it to those who can fix it?
> > (I don't consider this mailing list as "those who can fix it". So I'm not sure \
> > what you expect by posting here. Does it help others?) 
> > If you didn't: *do* you know where to report it?
> > 
> > If you did (which is what I assume), and it's not fixed after such a long time (a \
> > year!), I think that makes the openSUSE project look pretty bad. :-( 
> > Peter
> 
> Brother Peter,
> 
> 	Yes on all 3. Question: With yast, I could look in the yast log and find the
> offending repo IP and hostname, can I find that same info if I use zypper? If so
> where? I've dug through zypper.log but struck out.... and grepped the remainder
> of /var/log for the package name I was updating when the update hung on the bad
> repo. Is there anywhere else to look to help narrow it down?

Hm, makes me sad to hear that this problem is not being taken care of, and I can well \
understand frustration following that.

It's probably packages from the build service which are the problem. There are two \
"classes" of packages in the build service: those that are more popular and are \
mirrored by 4 mirrors in the US, and those that are a little less popular and are \
mirrored only by 2 mirrors. This includes all packages in the "home:" namespace, but \
also many others.

Recently, one US mirror stopped mirroring the build service, so the latter packages \
are not properly mirrored anymore in the US. The remaining two mirrors in the US are: \
University of Texas, and a mirror operated by susestudio.com. From what I hear, both \
are very slow. I'd think that this is the #1 reason that makes downloads very slow \
for you.

All other mirrors are in Europe (4xde, gb, pt, nl, ro). I don't know if the US \
mirrors are slow as such, or if they are just flooded with too many requests because \
they are the only ones.

Anyway, the situation would improve if somebody in the US would volunteer and \
contribute a mirror. However, nobody there seems to be interested enough to step up \
and contribute spare time and resources for this purpose. There may still be \
possibilities. I have found that chances in finding new mirrors can be considerably \
improved by actively approaching existing mirrors and ask them for help. 

Beyond that, if one of the US mirrors is not performing well, it should be taken out \
of the rotation. But maybe they do perform well but are just overloaded, due to lack \
of other mirrors. I can't tell. (I am assuming that utexas.edu is indeed the \
recurrent problem for you.)


Having stated all that, maybe you can confirm some observations? I guess you are \
using a more recent openSUSE version, like 11.3? As you know, since 11.2, zypper/YaST \
use a download backend which should work around about exactly the above issues, by \
contacting several mirrors in parallel. Maybe that is not configured well.

Take repositories/home:/-miska-/openSUSE_11.3/x86_64/mapnik-0.7.1-1.1.x86_64.rpm as \
an example for a package which is only on few mirrors. Here are they, as seen from \
the US:

 % curl -s 'http://download.opensuse.org/repositories/home:/-miska-/openSUSE_11.3/x86_64/mapnik-0.7.1-1.1.x86_64.rpm.metalink?country=us' \
| grep location  <url type="http" location="us" \
preference="100">http://ftp.utexas.edu/opensuse/repositories/home:/-miska-/openSUSE_11.3/x86_64/mapnik-0.7.1-1.1.x86_64.rpm</url>
                
      <url type="http" location="us" \
preference="99">http://130.57.19.201/repositories/home:/-miska-/openSUSE_11.3/x86_64/mapnik-0.7.1-1.1.x86_64.rpm</url>
  <url type="http" location="de" \
preference="98">http://ftp.halifax.rwth-aachen.de/opensuse/repositories/home:/-miska-/openSUSE_11.3/x86_64/mapnik-0.7.1-1.1.x86_64.rpm</url>
  <url type="http" location="de" \
preference="97">http://ftp5.gwdg.de/pub/opensuse/repositories/home:/-miska-/openSUSE_11.3/x86_64/mapnik-0.7.1-1.1.x86_64.rpm</url>
                
      <url type="http" location="de" \
preference="96">http://widehat.opensuse.org/repositories/home:/-miska-/openSUSE_11.3/x86_64/mapnik-0.7.1-1.1.x86_64.rpm</url>
  <url type="http" location="ro" \
preference="95">http://ftp.roedu.net/mirrors/opensuse.org/repositories/home:/-miska-/openSUSE_11.3/x86_64/mapnik-0.7.1-1.1.x86_64.rpm</url>



Using aria2c directly, we can see what it does with that:

 % aria2c --log=- --log-level=debug '--user-agent=ZYpp 7.7.5 (aria2 version 1.9.1) \
openSUSE-11.3-x86_64' --summary-interval=1 --follow-metalink=mem \
--check-integrity=true --file-allocation=none \
--server-stat-of=/home/poeml/.aria2/aria2.stats \
--server-stat-if=/home/poeml/.aria2/aria2.stats --uri-selector=adaptive \
--use-head=false --max-tries=5 --max-concurrent-downloads=2 --header=Pragma: \
--connect-timeout=180 \
'http://download.opensuse.org/repositories/home:/-miska-/openSUSE_11.3/x86_64/mapnik-0.7.1-1.1.x86_64.rpm.metalink?country=us' \
                | grep Host
Host: download.opensuse.org
Host: widehat.opensuse.org
Host: ftp.halifax.rwth-aachen.de
Host: ftp5.gwdg.de
Host: ftp.roedu.net
Host: 130.57.19.201

It contacts 5 servers, which is the built-in default for concurrency. The job \
finishes after 1 or 2 seconds for me, which is about the same time to download the \
package from a single source in this case. 

Maybe if you can try that out, do you get a working download with that? If you see a \
long hang with that, it might be due to this bug: \
https://bugzilla.novell.com/show_bug.cgi?id=555803


Sigh. When testing the same as above from the US, aria2c does not contact 5 mirrors; \
it contacts only one, and if that is utexas.edu, the download takes a minute instead \
of 2-4 seconds. The log shows that only one server is contacted, and it is \
reproducible:

 % aria2c --log=- --log-level=debug '--user-agent=ZYpp 7.7.5 (aria2 version 1.9.1) \
openSUSE-11.3-x86_64' --summary-interval=1 --follow-metalink=mem \
--check-integrity=true --file-allocation=none \
--server-stat-of=/home/poeml/.aria2/aria2.stats \
--server-stat-if=/home/poeml/.aria2/aria2.stats --uri-selector=adaptive \
--use-head=false --max-tries=5 --max-concurrent-downloads=2 --header=Pragma: \
--connect-timeout=180 \
'http://download.opensuse.org/repositories/home:/-miska-/openSUSE_11.3/x86_64/mapnik-0.7.1-1.1.x86_64.rpm.metalink?country=us' \
                | grep Host
Host: download.opensuse.org
Host: 130.57.19.201

The worst is, I don't know why this differs. Apart from the aria2c version everything \
should be exactly the same. In the latter case, I used the newest version, 1.10.3.

Which aria2 package version do you have installed?

Peter-- 
To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org
For additional commands, e-mail: opensuse+help@opensuse.org


[prev in list] [next in list] [prev in thread] [next in thread]