[prev in list] [next in list] [prev in thread] [next in thread]
List: pykde
Subject: Re: [PyQt] Attempting to reduce the memory footprint of my PyQt5 application
From: oliver <oliver.schoenborn () gmail ! com>
Date: 2016-09-08 4:26:03
Message-ID: CAMb+DaXXqYN-WSs_j3TEzMerMRD4JPiQy1oVtqNyCzWG96Yc5g () mail ! gmail ! com
[Download RAW message or body]
[Attachment #2 (multipart/alternative)]
Xavion,
The better way to save memory is to use following techniques:
1. don't import more than necessary; for example, import from
PyQt5.QtCore, not from PyQt5.Qt
2. consider using Python slots (unrelated to Qt slots) for large numbers
of objects stored in lists
3. consider delaying loading of data (from disk or network), and only
load what the app can show/process/edit
4. use generators instead of lists by using range() and itertools module
when possible, and consider creating your own generators using the yield
statement
5. *might* save some mem by using struct module where appropriate
6. only create circular references when necessary (see
http://engineering.hearsaysocial.com/2013/06/16/circular-references-in-python/),
although it seems that python 3.4 is now quite good at deleting objects
that were stuck in cyclical refs even if they have a del method (but the
docs say that there can still be uncollectable garbage once in a while,
without being specific, so if your app happens to generate a lot of objects
with the particular kind of circular reference that is not collectable by
interpreter, you would see steady increase in memory by your app as you
create and delete those objects). Looks like objgraph might be a good
utility for identifying cycles but I have not used it yet (just found out
while researching this topic)
All that being said, you should probably dig deeper into into all of the
above only when you determine you have a memory leak or are using gigs more
memory than necessary.
Oliver
Oliver
Open Source contributions: PyPubSub <http://pubsub.sf.net/>, nose2pytest
<https://github.com/schollii/nose2pytest>, Lua-iCxx
<http://lua-icxx.sf.net/>, iof <http://iof.sf.net/>
StackOverflow <http://stackoverflow.com/users/869951/schollii> contributions
On Tue, Sep 6, 2016 at 12:22 AM, Xavion <xavion.0@gmail.com> wrote:
> Hi All,
>
> I'm trying to cut down on the memory usage of my PyQt5 application. I'm
> assuming that one way to do this is to only import the Qt submodules that
> are actually used.
>
> The three attached files illustrate my point. The Qt5 Designer only
> imports the necessary submodules (e.g. "QtCore/QVariant"). Whereas, the
> PyQt5 UIC and RCC generators import the parent modules in full (e.g.
> "QtCore").
>
> Assuming my memory reduction theory is right, is there any way that I can
> get the UIC and RCC generators to behave like the Qt5 Designer? There's
> probably no point in using this strategy throughout the rest of my code if
> not.
>
>
> --
> Regards, Xavion.
>
>
> _______________________________________________
> PyQt mailing list PyQt@riverbankcomputing.com
> https://www.riverbankcomputing.com/mailman/listinfo/pyqt
>
[Attachment #5 (text/html)]
<div dir="ltr">Xavion, <div>The better way to save memory is to use following \
techniques: <div><ol><li>don't import more than necessary; for example, import \
from PyQt5.QtCore, not from PyQt5.Qt<br></li><li>consider using Python slots \
(unrelated to Qt slots) for large numbers of objects stored in \
lists<br></li><li>consider delaying loading of data (from disk or network), and only \
load what the app can show/process/edit<br></li><li>use generators instead of lists \
by using range() and itertools module when possible, and consider creating your own \
generators using the yield statement<br></li><li>*might* save some mem by using \
struct module where appropriate<br></li><li>only create circular references when \
necessary (see <a href="http://engineering.hearsaysocial.com/2013/06/16/circular-refe \
rences-in-python/">http://engineering.hearsaysocial.com/2013/06/16/circular-references-in-python/</a>), \
although it seems that python 3.4 is now quite good at deleting objects that were \
stuck in cyclical refs even if they have a del method (but the docs say that there \
can still be uncollectable garbage once in a while, without being specific, so if \
your app happens to generate a lot of objects with the particular kind of circular \
reference that is not collectable by interpreter, you would see steady increase in \
memory by your app as you create and delete those objects). Looks like objgraph might \
be a good utility for identifying cycles but I have not used it yet (just found out \
while researching this topic)<br></li></ol></div><div><br></div><div>All that being \
said, you should probably dig deeper into into all of the above only when you \
determine you have a memory leak or are using gigs more memory than necessary. \
</div><div>Oliver</div><div><br></div></div></div><div class="gmail_extra"><br \
clear="all"><div><div class="gmail_signature" data-smartmail="gmail_signature"><div \
dir="ltr"><div><div dir="ltr"><div dir="ltr"><div><font size="2" \
style="font-size:small">Oliver</font><div style="font-size:small"><font size="1">Open \
Source contributions: <a href="http://pubsub.sf.net/" style="color:rgb(17,85,204)" \
target="_blank">PyPubSub</a>,</font><span style="font-size:x-small"> </span><a \
href="https://github.com/schollii/nose2pytest" \
style="color:rgb(17,85,204);font-size:x-small" target="_blank">nose2pytest</a><span \
style="font-size:x-small">,</span><span style="font-size:x-small"> L</span><a \
href="http://lua-icxx.sf.net/" style="font-size:x-small;color:rgb(17,85,204)" \
target="_blank">ua-iCxx</a><span style="font-size:x-small">, </span><a \
href="http://iof.sf.net/" style="font-size:x-small;color:rgb(17,85,204)" \
target="_blank">iof</a></div><div style="font-size:small"><font size="1"><a \
href="http://stackoverflow.com/users/869951/schollii" style="color:rgb(17,85,204)" \
target="_blank">StackOverflow</a> contributions</font></div></div><div><font \
size="1"><br></font></div><div></div></div></div></div></div></div></div> <br><div \
class="gmail_quote">On Tue, Sep 6, 2016 at 12:22 AM, Xavion <span dir="ltr"><<a \
href="mailto:xavion.0@gmail.com" target="_blank">xavion.0@gmail.com</a>></span> \
wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px \
#ccc solid;padding-left:1ex"><div dir="ltr">Hi All,<div><br></div><div>I'm trying \
to cut down on the memory usage of my PyQt5 application. I'm assuming that one \
way to do this is to only import the Qt submodules that are actually \
used.</div><div><br></div><div>The three attached files illustrate my point. The \
Qt5 Designer only imports the necessary submodules (e.g. \
"QtCore/QVariant"). Whereas, the PyQt5 UIC and RCC generators import the \
parent modules in full (e.g. "QtCore").</div><div><br></div><div>Assuming \
my memory reduction theory is right, is there any way that I can get the UIC and RCC \
generators to behave like the Qt5 Designer? There's probably no point in using \
this strategy throughout the rest of my code if not.</div><span class="HOEnZb"><font \
color="#888888"><div><br clear="all"><div><br></div>-- <br><div><div dir="ltr"><span \
style="color:rgb(68,68,68)"><font size="2">Regards, \
Xavion.<br><br></font></span></div></div> </div></font></span></div>
<br>______________________________<wbr>_________________<br>
PyQt mailing list <a \
href="mailto:PyQt@riverbankcomputing.com">PyQt@riverbankcomputing.com</a><br> <a \
href="https://www.riverbankcomputing.com/mailman/listinfo/pyqt" rel="noreferrer" \
target="_blank">https://www.<wbr>riverbankcomputing.com/<wbr>mailman/listinfo/pyqt</a><br></blockquote></div><br></div>
[Attachment #6 (text/plain)]
_______________________________________________
PyQt mailing list PyQt@riverbankcomputing.com
https://www.riverbankcomputing.com/mailman/listinfo/pyqt
[prev in list] [next in list] [prev in thread] [next in thread]
Configure |
About |
News |
Add a list |
Sponsored by KoreLogic