[prev in list] [next in list] [prev in thread] [next in thread]
List: pykde
Subject: Re: [PyQt] Possible memory leak with signal connections
From: "Kevin Keating" <kevin.keating () schrodinger ! com>
Date: 2019-09-25 21:05:43
Message-ID: 8b370174-9b43-4a03-8f24-8ed389056135 () getmailbird ! com
[Download RAW message or body]
[Attachment #2 (multipart/alternative)]
On 9/25/2019 9:17:24 AM, Phil Thompson <phil@riverbankcomputing.com> wrote:
On 19/09/2019 19:53, Kevin Keating wrote:
> I've found what appears to be a memory leak caused by connecting
> signals to slots that lack a QtCore.pyqtSlot decorator.=C2=A0 I know that
> connections without the pyqtSlot decorator are expected to use more
> memory than connections with the decorator based on
> https://www.codeproject.com/Articles/1123088/PyQt-signal-slot-connection-=
performance.=C2=A0
> In the script I've pasted below, though, connections without the
> pyqtSlot decorator continue to consume memory even after the signal
> has been disconnected and the QObjects have been discarded.=C2=A0 The
> script does the following:
> =C2=A0 - Instantiates a bunch of SignalObjects, which are QObjects with a
> signal, and stores the SingalObjects in a list.
> =C2=A0 - Instantiates a bunch of SlotObjects, which are QObjects with
> slots.=C2=A0 Each SlotObject slot is connected to the signals from all the
> SignalObjects.=C2=A0 The SlotObjects are then immediately discarded.
> =C2=A0 - Discards all SignalObjects.
> =C2=A0 - Runs gc.collect()
> =C2=A0 - Runs the QApplication's event loop for a second in case there are
> any pending DeferredDelete events.
>
> If the SlotObject class uses the QtCore.pyqtSlot decorator, then
> memory usage at the end of the script is the same as what it was at
> the beginning of the script, which makes sense since all the objects
> that get created should be completely destroyed before the script
> finishes.=C2=A0 Here's the output that I get with pyqtSlot decorators:
> =C2=A0 =C2=A0 =C2=A0 =C2=A0 Memory before signal_objects creation: 17.1MiB
> =C2=A0 =C2=A0 =C2=A0 =C2=A0 Memory before slot objects creation: 17.1MiB
> =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 Difference: 0.0B
> =C2=A0 =C2=A0 =C2=A0 =C2=A0 Memory after slot objects creation: 17.1MiB
> =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 Difference: 0.0B
> =C2=A0 =C2=A0 =C2=A0 =C2=A0 Memory after event loop runs: 17.1MiB
> =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 Difference: 0.0B
> If the SlotObject class doesn't use the QtCore.pyqtSlot decorator,
> though, then memory usage at the end of the script is substantially
> higher.=C2=A0 Here's the output that I get without pyqtSlot decorators:
> =C2=A0 =C2=A0 =C2=A0 =C2=A0 Memory before signal_objects creation: 17.0MiB
> =C2=A0 =C2=A0 =C2=A0 =C2=A0 Memory before slot objects creation: 17.0MiB
> =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 Difference: 0.0B
> =C2=A0 =C2=A0 =C2=A0 =C2=A0 Memory after slot objects creation: 210.6MiB
> =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 Difference: 193.5=
MiB
> =C2=A0 =C2=A0 =C2=A0 =C2=A0 Memory after event loop runs: 210.6MiB
> =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 Difference: 193.5=
MiB
> Is this expected behavior?=C2=A0 Without pyqtSlot decorators, is there
> anything I can do to recover the 193 MiB of memory other than
> terminating the process?=C2=A0 Thanks!
>
> I've tested the script with Python 3.6.2, PyQt 5.12.2, and Qt 5.12.3
> on Windows 10, Linux, and Mac OS.=C2=A0 I've also tested with Python 3.6.=
5,
> PyQt 5.13.1, and Qt 5.13.1 on Windows 10.=C2=A0 All of them give similar
> results.=C2=A0 The script requires the psutil package
> (https://pypi.org/project/psutil/) to monitor memory usage.
Are you expecting that the Python interpreter will return unused memory
to the operating system?
Phil
My assumption was that the memory usage from the signal connections wouldn'=
t "build up" (i.e. that memory usage would be independent of the number of =
SignalObjects instantiated), which seems to be what happens with the pyqtSl=
ot decorated version.=C2=A0 In create_slot_objects, each SlotObject is inst=
antiated, which connects the signals, and then should be immediately garbag=
e collected, which should disconnect the signals.=C2=A0 In the version of t=
he script without the pyqtSlot decorators, the number reported for "Memory =
after slot objects creation" is proportional to the number of SlotObject in=
stances created, which is why I'd assumed memory had leaked, but I could ce=
rtainly be thinking through things wrong.
- Kevin
[Attachment #5 (text/html)]
<div id="__MailbirdStyleContent" style="font-size: 10pt;font-family: Arial;color: \
#000000"><blockquote class="history_container" type="cite" style="border-left-style: \
solid;border-width: 1px;margin-top: 20px;margin-left: 0px;padding-left: \
10px;min-width: 500px"><p style="color: #AAAAAA; margin-top: 10px;">On 9/25/2019 \
9:17:24 AM, Phil Thompson <phil@riverbankcomputing.com> wrote:</p><div \
style="font-family:Arial,Helvetica,sans-serif">On 19/09/2019 19:53, Kevin Keating \
wrote: <br>> I've found what appears to be a memory leak caused by connecting
<br>> signals to slots that lack a QtCore.pyqtSlot decorator. I know that
<br>> connections without the pyqtSlot decorator are expected to use more
<br>> memory than connections with the decorator based on
<br>> https://www.codeproject.com/Articles/1123088/PyQt-signal-slot-connection-performance.
<br>> In the script I've pasted below, though, connections without the
<br>> pyqtSlot decorator continue to consume memory even after the signal
<br>> has been disconnected and the QObjects have been discarded. The
<br>> script does the following:
<br>> - Instantiates a bunch of SignalObjects, which are QObjects with a
<br>> signal, and stores the SingalObjects in a list.
<br>> - Instantiates a bunch of SlotObjects, which are QObjects with
<br>> slots. Each SlotObject slot is connected to the signals from all the
<br>> SignalObjects. The SlotObjects are then immediately discarded.
<br>> - Discards all SignalObjects.
<br>> - Runs gc.collect()
<br>> - Runs the QApplication's event loop for a second in case there are
<br>> any pending DeferredDelete events.
<br>>
<br>> If the SlotObject class uses the QtCore.pyqtSlot decorator, then
<br>> memory usage at the end of the script is the same as what it was at
<br>> the beginning of the script, which makes sense since all the objects
<br>> that get created should be completely destroyed before the script
<br>> finishes. Here's the output that I get with pyqtSlot decorators:
<br>> Memory before signal_objects creation: 17.1MiB
<br>> Memory before slot objects creation: 17.1MiB
<br>> Difference: 0.0B
<br>> Memory after slot objects creation: 17.1MiB
<br>> Difference: 0.0B
<br>> Memory after event loop runs: 17.1MiB
<br>> Difference: 0.0B
<br>> If the SlotObject class doesn't use the QtCore.pyqtSlot decorator,
<br>> though, then memory usage at the end of the script is substantially
<br>> higher. Here's the output that I get without pyqtSlot decorators:
<br>> Memory before signal_objects creation: 17.0MiB
<br>> Memory before slot objects creation: 17.0MiB
<br>> Difference: 0.0B
<br>> Memory after slot objects creation: 210.6MiB
<br>> Difference: 193.5MiB
<br>> Memory after event loop runs: 210.6MiB
<br>> Difference: 193.5MiB
<br>> Is this expected behavior? Without pyqtSlot decorators, is there
<br>> anything I can do to recover the 193 MiB of memory other than
<br>> terminating the process? Thanks!
<br>>
<br>> I've tested the script with Python 3.6.2, PyQt 5.12.2, and Qt 5.12.3
<br>> on Windows 10, Linux, and Mac OS. I've also tested with Python 3.6.5,
<br>> PyQt 5.13.1, and Qt 5.13.1 on Windows 10. All of them give similar
<br>> results. The script requires the psutil package
<br>> (https://pypi.org/project/psutil/) to monitor memory usage.
<br>
<br>Are you expecting that the Python interpreter will return unused memory
<br>to the operating system?
<br>
<br>Phil
</div></blockquote><div><span style="font-size: 10pt;font-family: Arial;color: \
#000000"><span style="color: #3B5998"><br></span></span>My assumption was that the \
memory usage from the signal connections wouldn't "build up" (i.e. that memory usage \
would be independent of the number of SignalObjects instantiated), which seems to be \
what happens with the pyqtSlot decorated version. In create_slot_objects, each \
SlotObject is instantiated, which connects the signals, and then should be \
immediately garbage collected, which should disconnect the signals. In the \
version of the script without the pyqtSlot decorators, the number reported for \
"Memory after slot objects creation" is proportional to the number of SlotObject \
instances created, which is why I'd assumed memory had leaked, but I could certainly \
be thinking through things wrong.</div><div><br></div><div>- Kevin</div></div>
[Attachment #6 (text/plain)]
_______________________________________________
PyQt mailing list PyQt@riverbankcomputing.com
https://www.riverbankcomputing.com/mailman/listinfo/pyqt
[prev in list] [next in list] [prev in thread] [next in thread]
Configure |
About |
News |
Add a list |
Sponsored by KoreLogic