[prev in list] [next in list] [prev in thread] [next in thread] 

List:       opensuse-buildservice
Subject:    Re: AW: Builds taking forever and not finishing
From:       Adrian =?ISO-8859-1?Q?Schr=F6ter?= <adrian () suse ! de>
Date:       2024-03-07 12:06:38
Message-ID: 4663229.CvnuH1ECHv () rio
[Download RAW message or body]

On Donnerstag, 7. März 2024, 12:13:42 CET Mathias Radtke wrote:
> Hi, me again.
> 
> So what I saw in the rep_server.log was
> 
> 
> 
> 2024-03-07 10:14:13: [19782] maxchild limit reached
> 
> 2024-03-07 10:14:13: maxchild limit reached
> 
> and on a worker itself

that sounds like hanging connections, as usually you should not run
into that limit.

bs_serverstatus /srv/obs/run/$each.status

may give a hint. maybe strace also.

> rpc failed: socket write: Connection timed out
> 
> sleeping one minute just in case...
> 
> send_statehttp://obs.uib.gmbh:5252 <http://obs.uib.gmbh:5252> : rpc timeout
> 
> 
> Any suggestion what I might do ?
> 
> Regards
> 
> Mathias
> 
> 
> 
> 
> Mathias Radtke
> Software Developer
> 
> uib GmbH
> Bonifaziusplatz 1B
> 55118 Mainz
> 
> E-Mail: m.radtke@uib.de <mailto:m.radtke@uib.de> 
> Telefon: +49 6131 27561 0
> 
> https://uib.de
> https://opsi.org
> 
> Sitz der Gesellschaft: Mainz
> Handelsregister: Amtsgericht Mainz HRB 6942
> Geschäftsführer: Erol Ülükmen
> USt-IdNr.: DE 203 394 450
> 
> 
> Save the Date: opsiConf in Mainz am 09. & 10. Juli 2024!
> Die Voranmeldung ist bereits eröffnet. opsi.org/opsiconf/
> 
> 
> 
> 
> 
> -----Ursprüngliche Nachricht-----
> Von: Mathias Radtke <m.radtke@uib.de>
> Gesendet: Donnerstag 7. März 2024 8:04
> An: buildservice@lists.opensuse.org
> Betreff: Builds taking forever and not finishing
> 
> Hi
> 
> 
> 
> since yesterday we have a strange behaviour on our ptivate OBS instance. 
> This instance links our packages published on the official OBS into our internal \
> instance. 
> The new updated packages get downloaded into our local instance. They are \
> "finished" according to the log but then still having the "building" state. The \
> time of the build continues also and it never reaches a success. 
> Here's a Log from one of the workers
> 
> 
> [  166s] obsworker1 finished "build opsi-linux-bootimage.spec" at Thu Mar  7 \
> 06:43:26 UTC 2024. 
> [  166s]
> 
> 
> 
> 2024-03-07 07:43:26: finished 'opsi-linux-bootimage' for project \
> 'home:uibmz:opsi:4.3:development' repository 'openSUSE_Leap_15.4' arch 'x86_64' 
> build succeeded, send everything back...
> 
> rpc failed: socket write: Connection timed out
> 
> 
> 
> In addition in the UI monitoring the Repserver seems to die and the workers \
> dissappear. Systemctl says the obsrespserver is running. Restarting it makes the \
> workers appear again in the UI Montoring page, and they start building until the \
> builds never finish again. 
> Any suggestion what might cause the issue?
> Which log should I look into?
> 
> Regards
> 
> Mathias
> 
> 
> 
> 
> Mathias Radtke
> Software Developer
> 
> uib GmbH
> Bonifaziusplatz 1B
> 55118 Mainz
> 
> E-Mail: m.radtke@uib.de <mailto:m.radtke@uib.de> 
> Telefon: +49 6131 27561 0
> 
> https://uib.de
> https://opsi.org
> 
> Sitz der Gesellschaft: Mainz
> Handelsregister: Amtsgericht Mainz HRB 6942
> Geschäftsführer: Erol Ülükmen
> USt-IdNr.: DE 203 394 450
> 
> 
> Save the Date: opsiConf in Mainz am 09. & 10. Juli 2024!
> Die Voranmeldung ist bereits eröffnet. opsi.org/opsiconf/
> 


-- 

Adrian Schroeter <adrian@suse.de>
Build Infrastructure Project Manager

SUSE Software Solutions Germany GmbH, Frankenstraße 146, 90461 Nürnberg, Germany
(HRB 36809, AG Nürnberg) Geschäftsführer: Ivo Totev


[prev in list] [next in list] [prev in thread] [next in thread]