[prev in list] [next in list] [prev in thread] [next in thread]
List: pypy-dev
Subject: Re: [pypy-dev] Mysterious IndexError in service running with PyPy
From: "hubo" <hubo () jiedaibao ! com>
Date: 2018-05-22 7:15:22
Message-ID: 5B03C387.4020506 () jiedaibao ! com+E78C0CB65B51204C
[Download RAW message or body]
[Attachment #2 (multipart/alternative)]
[Attachment #4 (text/plain)]
Thank you for your explaination!
One more thing I would like to confirm: looks like in PyPy3, a "yield from \
my_generator()" expression delegates all exceptions inside, so the finalizer in \
my_generator() is always executed. Am I right?
2018-05-22
hubo
发件人：Armin Rigo <armin.rigo@gmail.com>
发送时间：2018-05-22 14:21
主题：Re: [pypy-dev] Mysterious IndexError in service running with PyPy
收件人："hubo"<hubo@jiedaibao.com>
抄送："William ML Leslie"<william.leslie.ttg@gmail.com>,"PyPy Developer Mailing \
List"<pypy-dev@python.org>
Hi,
On 22 May 2018 at 04:46, hubo <hubo@jiedaibao.com> wrote:
> I think I found the cause, happily it is not a PyPy bug (although it is
> PyPy-related). Some generators are calling some cleanup code in the
> "finally" clause, but they are not correctly collected in PyPy until GC
> (because PyPy does not use reference counting). Apparently when they are
> collected by GC module, the "finally" clause is executed in a separated
> thread.
Unlike Java, we're not purposefully running finalizers in its own
threads: instead, if your program has got more than one thread
running, finalizers will run in the one where the GC happens to run.
There isn't much we can do about that.
All these hard GC issues also exist on CPython, but they are just more
hidden. Typically, if there is a cycle of objects which itself holds,
indirectly, a reference to the generator-iterator, then you get the
same effect: the cycle will be broken by the GC module of CPython at a
random point in time and from a random thread, too. So we can argue
that you potentially have a bug even on CPython, in the sense that an
apparently unrelated change somewhere else can make the same bug
appear on CPython.
A bientôt,
Armin.
[Attachment #5 (text/html)]
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN">
<HTML><HEAD>
<META content="text/html; charset=utf-8" http-equiv=Content-Type><!-- flashmail style \
begin --> <STYLE type=text/css>
body {border-width:0;margin:0}
img {border:0;margin:0;padding:0}
</STYLE>
<BASE target=_blank><!-- flashmail style end -->
<META name=GENERATOR content="MSHTML 11.00.10570.1001"></HEAD>
<BODY
style="BORDER-LEFT-WIDTH: 0px; FONT-SIZE: 10.5pt; FONT-FAMILY: 微软雅黑; \
BORDER-RIGHT-WIDTH: 0px; BORDER-BOTTOM-WIDTH: 0px; COLOR: #000000; MARGIN: 12px; \
LINE-HEIGHT: 1.5; BORDER-TOP-WIDTH: 0px" marginheight="0" marginwidth="0">
<DIV>Thank you for your explaination!</DIV>
<DIV> </DIV>
<DIV>One more thing I would like to confirm: looks like in PyPy3, a "yield from
my_generator()" expression delegates all exceptions inside, so the finalizer in
my_generator() is always executed. Am I right?</DIV>
<DIV> </DIV>
<DIV style="FONT-SIZE: 10pt; FONT-FAMILY: Verdana; COLOR: #c0c0c0"
align=left>2018-05-22
<HR id=SignNameHR
style="BORDER-TOP: #c0c0c0 1px solid; HEIGHT: 1px; BORDER-RIGHT: 0px; WIDTH: 122px; \
BORDER-BOTTOM: 0px; BORDER-LEFT: 0px" align=left>
<SPAN id=_FlashSignName>hubo</SPAN> </DIV>
<HR
style="BORDER-TOP: #c0c0c0 1px solid; HEIGHT: 1px; BORDER-RIGHT: 0px; BORDER-BOTTOM: \
0px; BORDER-LEFT: 0px">
<BLOCKQUOTE id=ntes-flashmail-quote
style="FONT-SIZE: 10pt; FONT-FAMILY: Verdana; PADDING-LEFT: 0px; MARGIN-LEFT: 0px">
<DIV><STRONG>发件人：</STRONG>Armin Rigo <armin.rigo@gmail.com></DIV>
<DIV><STRONG>发送时间：</STRONG>2018-05-22 14:21</DIV>
<DIV><STRONG>主题：</STRONG>Re: [pypy-dev] Mysterious IndexError in service
running with PyPy</DIV>
<DIV><STRONG>收件人：</STRONG>"hubo"<hubo@jiedaibao.com></DIV>
<DIV><STRONG>抄送：</STRONG>"William ML
Leslie"<william.leslie.ttg@gmail.com>,"PyPy Developer Mailing
List"<pypy-dev@python.org></DIV>
<DIV> </DIV>
<DIV>
<DIV>Hi, </DIV>
<DIV> </DIV>
<DIV>On 22 May 2018 at 04:46, hubo <hubo@jiedaibao.com> wrote: </DIV>
<DIV>> I think I found the cause, happily  \
;it is not a PyPy bug (although it is </DIV>
<DIV>> PyPy-related). Some generators are calling some cleanup code in the </DIV>
<DIV>> "finally" clause, but they are not correctly collected in PyPy until GC </DIV>
<DIV>> (because PyPy does not use reference counting). Apparently when they are </DIV>
<DIV>> collected by GC module, the "finally" clause is executed in a separated </DIV>
<DIV>> thread. </DIV>
<DIV> </DIV>
<DIV>Unlike Java, we're not purposefully running finalizers in its own </DIV>
<DIV>threads: instead, if your program has got more than one thread </DIV>
<DIV>running, finalizers will run in the one where the GC happens to run. </DIV>
<DIV>There isn't much we can do about that. </DIV>
<DIV> </DIV>
<DIV>All these hard GC issues also exist on CPython, but they are just more </DIV>
<DIV>hidden. Typically, if there is a cycle of objects which itself holds, </DIV>
<DIV>indirectly, a reference to the generator-iterator, then you get the </DIV>
<DIV>same effect: the cycle will be broken by the GC module of CPython at a </DIV>
<DIV>random point in time and from a random thread, too. So we can argue </DIV>
<DIV>that you potentially have a bug even on CPython, in the sense that an </DIV>
<DIV>apparently unrelated change somewhere else can make the same bug </DIV>
<DIV>appear on CPython. </DIV>
<DIV> </DIV>
<DIV> </DIV>
<DIV>A bientôt, </DIV>
<DIV> </DIV>
<DIV>Armin. </DIV>
<DIV> </DIV></DIV></BLOCKQUOTE></BODY></HTML>
_______________________________________________
pypy-dev mailing list
pypy-dev@python.org
https://mail.python.org/mailman/listinfo/pypy-dev
[prev in list] [next in list] [prev in thread] [next in thread]
Configure |
About |
News |
Add a list |
Sponsored by KoreLogic