[prev in list] [next in list] [prev in thread] [next in thread]
List: nginx
Subject: Re: killed child process
From: Alex Samad <alex () samad ! com ! au>
Date: 2017-05-20 22:07:17
Message-ID: CAJ+Q1PWGD03XHB3KkwAVDkRA6o6KFUGcevGxtMy9yNEUtMhnWg () mail ! gmail ! com
[Download RAW message or body]
[Attachment #2 (multipart/alternative)]
Well at least in my case, I can ask the application to make an orderly
reconnect. Where if nginx does it it just closes the connection.
The option to do it seems like better than having no option.
Alex
On 20 May 2017 at 21:11, B.R. via nginx <nginx@nginx.org> wrote:
> ... and you would end up with connections serving different content (as
> per different configuration) on the long run, leading potentially to an
> increased number of problems.
> How would you shut them down, if not gracefully?
>
> If you want to keep long-lived connections open, do not make changes
> server-side, such as asking it to reopen configuration (and thus to apply
> it to every worker).
> ---
> *B. R.*
>
> On Fri, May 19, 2017 at 11:40 PM, Alex Samad <alex@samad.com.au> wrote:
>
>> Yes this exactly, I ended up been schooled by support :)
>>
>> Seems like my understanding of graceful shutdown / reload ..
>>
>> for the list and the archives
>>
>> No keep alive for http1.0, has to be http1.1
>>
>>
>> client -> nginx keep alive session - these are shutdown once the current
>> request is completed
>> nginx -> backend server keep alive session - these are shutdown once the
>> current request is completed
>>
>> client -> nginx - websockets ... these are kept alive until the client
>> closes it
>> nginx -> backend - websocket ... these are kept alive until the client
>> closes it
>>
>>
>> client -> nginx -> backend ... ssl passthrough ? I presume these are
>> kept alive until either the client or the backend closes it.
>>
>>
>> it would be nice to allow a reload/refresh but keep keep-alive session
>> open until the client closes it
>>
>> Alex
>>
>>
>>
>>
>>
>>
>>
>> On 19 May 2017 at 23:10, Maxim Dounin <mdounin@mdounin.ru> wrote:
>>
>>> Hello!
>>>
>>> On Fri, May 19, 2017 at 11:28:05AM +1000, Alex Samad wrote:
>>>
>>> > Hi
>>> >
>>> > so I have lots of clients on long lived tcp connections , getting rp
>>> into 2
>>> > back end app servers
>>> >
>>> > I had a line in my error log, saying one of the upstream was failed
>>> caused
>>> > it timeout -
>>> >
>>> >
>>> > then I got this
>>> >
>>> > 2017/05/18 13:30:42 [notice] 2662#2662: exiting
>>> > 2017/05/18 13:30:42 [notice] 2662#2662: exit
>>> > 2017/05/18 13:30:42 [notice] 2661#2661: signal 17 (SIGCHLD) received
>>> > 2017/05/18 13:30:42 [notice] 2661#2661: worker process 2662 exited with
>>> > code 0
>>> > 2017/05/18 13:30:42 [notice] 2661#2661: signal 29 (SIGIO) received
>>> >
>>> >
>>> > I am not sure what initiated this ? there are no cron jobs running at
>>> that
>>> > time
>>> >
>>> > and it looks like a lot of my long lived tcp session where TCP-FIN'ed.
>>> >
>>> >
>>> > I also checked my audit logs to see what command / process run at that
>>> time
>>> > ... nothing that signaled or initiated a nginx reload ...
>>>
>>> Try searching error log for prior messages from process 2662 (note
>>> that each error message have process ID in it, "... 2662#..."), it
>>> should help to identify why it exited.
>>>
>>> Most likley it was an old worker previously instructed to shutdown
>>> gracefully due to a configuration reload, and it finished serving
>>> all the existing clients and exited. If it's the case, you should
>>> see something like "gracefully shutting down" from the worker
>>> somewhere earlier in logs, and "reconfiguring" from master just
>>> before it.
>>>
>>> --
>>> Maxim Dounin
>>> http://nginx.org/
>>> _______________________________________________
>>> nginx mailing list
>>> nginx@nginx.org
>>> http://mailman.nginx.org/mailman/listinfo/nginx
>>>
>>
>>
>> _______________________________________________
>> nginx mailing list
>> nginx@nginx.org
>> http://mailman.nginx.org/mailman/listinfo/nginx
>>
>
>
> _______________________________________________
> nginx mailing list
> nginx@nginx.org
> http://mailman.nginx.org/mailman/listinfo/nginx
>
[Attachment #5 (text/html)]
<div dir="ltr">Well at least in my case, I can ask the application to make an orderly \
reconnect. Where if nginx does it it just closes the \
connection.<div><br></div><div>The option to do it seems like better than having no \
option.</div><div><br></div><div>Alex</div></div><div class="gmail_extra"><br><div \
class="gmail_quote">On 20 May 2017 at 21:11, B.R. via nginx <span dir="ltr"><<a \
href="mailto:nginx@nginx.org" target="_blank">nginx@nginx.org</a>></span> \
wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px \
#ccc solid;padding-left:1ex"><div dir="ltr"><div class="gmail_default" \
style="font-size:small;color:rgb(51,51,153)">... and you would end up with \
connections serving different content (as per different configuration) on the long \
run, leading potentially to an increased number of problems.<br></div><div \
class="gmail_default" style="font-size:small;color:rgb(51,51,153)">How would you shut \
them down, if not gracefully?<br><br></div><div class="gmail_default" \
style="font-size:small;color:rgb(51,51,153)">If you want to keep long-lived \
connections open, do not make changes server-side, such as asking it to reopen \
configuration (and thus to apply it to every worker).<br></div><div \
class="gmail_extra"><div><div class="m_2671196621031259156gmail_signature" \
data-smartmail="gmail_signature"><font size="1"><span \
style="color:rgb(102,102,102)">---<br></span><b><span \
style="color:rgb(102,102,102)">B. R.</span></b><span \
style="color:rgb(102,102,102)"></span></font></div></div> <br><div \
class="gmail_quote">On Fri, May 19, 2017 at 11:40 PM, Alex Samad <span \
dir="ltr"><<a href="mailto:alex@samad.com.au" \
target="_blank">alex@samad.com.au</a>></span> wrote:<br><blockquote \
class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc \
solid;padding-left:1ex"><div dir="ltr">Yes this exactly, I ended up been schooled by \
support :)<div><br></div><div>Seems like my understanding of graceful shutdown / \
reload ..</div><div><br></div><div>for the list and the \
archives</div><div><br></div><div>No keep alive for http1.0, has to be \
http1.1</div><div><br></div><div><br></div><div>client -> nginx keep alive session \
- these are shutdown once the current request is completed </div><div>nginx -> \
backend server keep alive session - these are shutdown once the current request is \
completed</div><div><br></div><div>client -> nginx - websockets ... these are kept \
alive until the client closes it</div><div>nginx -> backend - websocket ... these \
are kept alive until the client closes \
it</div><div><br></div><div><br></div><div>client -> nginx -> backend ... ssl \
passthrough ? I presume these are kept alive until either the client or the backend \
closes it.</div><div><br></div><div><br></div><div>it would be nice to allow a \
reload/refresh but keep keep-alive session open until the client closes it</div><span \
class="m_2671196621031259156HOEnZb"><font \
color="#888888"><div><br></div><div>Alex</div><div><br></div><div><br></div><div><br></div><div><br></div><div><br></div><div><br></div></font></span></div><div \
class="m_2671196621031259156HOEnZb"><div class="m_2671196621031259156h5"><div \
class="gmail_extra"><br><div class="gmail_quote">On 19 May 2017 at 23:10, Maxim \
Dounin <span dir="ltr"><<a href="mailto:mdounin@mdounin.ru" \
target="_blank">mdounin@mdounin.ru</a>></span> wrote:<br><blockquote \
class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc \
solid;padding-left:1ex">Hello!<br> <span><br>
On Fri, May 19, 2017 at 11:28:05AM +1000, Alex Samad wrote:<br>
<br>
> Hi<br>
><br>
> so I have lots of clients on long lived tcp connections , getting rp into 2<br>
> back end app servers<br>
><br>
> I had a line in my error log, saying one of the upstream was failed caused<br>
> it timeout -<br>
><br>
><br>
> then I got this<br>
><br>
> 2017/05/18 13:30:42 [notice] 2662#2662: exiting<br>
> 2017/05/18 13:30:42 [notice] 2662#2662: exit<br>
> 2017/05/18 13:30:42 [notice] 2661#2661: signal 17 (SIGCHLD) received<br>
> 2017/05/18 13:30:42 [notice] 2661#2661: worker process 2662 exited with<br>
> code 0<br>
> 2017/05/18 13:30:42 [notice] 2661#2661: signal 29 (SIGIO) received<br>
><br>
><br>
> I am not sure what initiated this ? there are no cron jobs running at that<br>
> time<br>
><br>
> and it looks like a lot of my long lived tcp session where TCP-FIN'ed.<br>
><br>
><br>
> I also checked my audit logs to see what command / process run at that time<br>
> ... nothing that signaled or initiated a nginx reload ...<br>
<br>
</span>Try searching error log for prior messages from process 2662 (note<br>
that each error message have process ID in it, "... 2662#..."), it<br>
should help to identify why it exited.<br>
<br>
Most likley it was an old worker previously instructed to shutdown<br>
gracefully due to a configuration reload, and it finished serving<br>
all the existing clients and exited. If it's the case, you should<br>
see something like "gracefully shutting down" from the worker<br>
somewhere earlier in logs, and "reconfiguring" from master just<br>
before it.<br>
<span class="m_2671196621031259156m_-4756504063898370487HOEnZb"><font \
color="#888888"><br>
--<br>
Maxim Dounin<br>
<a href="http://nginx.org/" rel="noreferrer" \
target="_blank">http://nginx.org/</a><br> \
______________________________<wbr>_________________<br> nginx mailing list<br>
<a href="mailto:nginx@nginx.org" target="_blank">nginx@nginx.org</a><br>
<a href="http://mailman.nginx.org/mailman/listinfo/nginx" rel="noreferrer" \
target="_blank">http://mailman.nginx.org/mailm<wbr>an/listinfo/nginx</a><br> \
</font></span></blockquote></div><br></div> \
</div></div><br>______________________________<wbr>_________________<br> nginx \
mailing list<br> <a href="mailto:nginx@nginx.org" \
target="_blank">nginx@nginx.org</a><br> <a \
href="http://mailman.nginx.org/mailman/listinfo/nginx" rel="noreferrer" \
target="_blank">http://mailman.nginx.org/mailm<wbr>an/listinfo/nginx</a><br></blockquote></div><br></div></div>
<br>______________________________<wbr>_________________<br>
nginx mailing list<br>
<a href="mailto:nginx@nginx.org">nginx@nginx.org</a><br>
<a href="http://mailman.nginx.org/mailman/listinfo/nginx" rel="noreferrer" \
target="_blank">http://mailman.nginx.org/<wbr>mailman/listinfo/nginx</a><br></blockquote></div><br></div>
_______________________________________________
nginx mailing list
nginx@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx
[prev in list] [next in list] [prev in thread] [next in thread]
Configure |
About |
News |
Add a list |
Sponsored by KoreLogic