[prev in list] [next in list] [prev in thread] [next in thread]
List: kwin
Subject: Re: Review Request: Fixing massive memory consumption in KWin using
From: Ivan Čukić <ivan.cukic () kde ! org>
Date: 2011-05-19 22:14:57
Message-ID: 20110519221457.25774.92325 () vidsolbach ! de
[Download RAW message or body]
[Attachment #2 (multipart/alternative)]
> On May 19, 2011, 7:34 a.m., Aaron J. Seigo wrote:
> > those are some impressive results. my guess is that we likely have a very similar \
> > issue in plasma. in both cases, we are dealing with lots of small bits of memory \
> > in the form of pixmaps, strings, etc. that are fairly constantly coming and \
> > going. switching between compositing and no compositing is probably the 'worst \
> > case' real world scenario for this.
> > if this does test out well, i'd suggest putting this code in libkworkspace so \
> > that all of our apps with this usage pattern (plasma-desktop, plasma-netbook, \
> > krunner, kwin, ..) can take advantage of a common implementation. a set of static \
> > methods in the KWorkspace namespace should suffice.
> > if you could repeat your experiment with plasma-desktop with and without such a \
> > patch, that would be great and let us know whether we should adopt such a \
> > strategy across the plasma shell components.
> > as for portability ... i'm not overly concerned about the portability of this to \
> > non-glibc systems as those systems are our primary targets. however, instead of \
> > checking for GLIBC the code could simply check for M_TRIM_THRESHOLD. for \
> > comparison, here is how busybox does this same trick for both the trim size and \
> > mmap request size threshold:
> > #ifndef PAGE_SIZE
> > # define PAGE_SIZE (4*1024) /* guess */
> > #endif
> > #ifdef M_TRIM_THRESHOLD
> > /* M_TRIM_THRESHOLD is the maximum amount of freed top-most memory
> > * to keep before releasing to the OS
> > * Default is way too big: 256k
> > */
> > mallopt(M_TRIM_THRESHOLD, 2 * PAGE_SIZE);
> > #endif
> > #ifdef M_MMAP_THRESHOLD
> > /* M_MMAP_THRESHOLD is the request size threshold for using mmap()
> > * Default is too big: 256k
> > */
> > mallopt(M_MMAP_THRESHOLD, 8 * PAGE_SIZE - 256);
> > #endif
> >
> > i doubt we need to tweak the mmap strategy, but i like the checks for the \
> > definitions of mallopt there. for portability to non-unix-y systems, we could \
> > wrap this whole thing in a #ifndef Q_WS_WIN block which should keep the KDE \
> > Windows team from pulling their hair out over the use of such UNIX-y API ;)
sysconf(_SC_PAGESIZE) returns the page size for mmap, no need for (4*1024)
- Ivan
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
http://git.reviewboard.kde.org/r/101385/#review3402
-----------------------------------------------------------
On May 18, 2011, 8:53 p.m., Philipp Knechtges wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> http://git.reviewboard.kde.org/r/101385/
> -----------------------------------------------------------
>
> (Updated May 18, 2011, 8:53 p.m.)
>
>
> Review request for kwin and Plasma.
>
>
> Summary
> -------
>
> The intention of this patch is to lower the heap fragmentation in KWin when using \
> the raster backend.
> One can illustrate the issue with the following testcase: If one currently uses the \
> raster backend and switches back to non-compositing mode one gets easily a similar \
> memory usage like the following:
> Situation: 14 windows, QtCurve
> KWin started with compositing: 40 MB
> KWin switched to non-compositing : more than 70 MB
>
> The first guess might be, that this is due to a memleak because of not properly \
> released pixmaps. But calling malloc_stats() sheds some more light on the subject \
> (non-compositing mode):
> Arena 0:
> system bytes = 72232960
> in use bytes = 8180512
> Arena 1:
> system bytes = 135168
> in use bytes = 4784
> Total (incl. mmap):
> system bytes = 73080832
> in use bytes = 8898000
> max mmap regions = 13
> max mmap bytes = 36343808
>
> In other words, the glibc has allocated more than 70 MB memory although KWin uses \
> only less than 10 MB. The problem is that glibc only resizes the heap if the heap \
> has more than 128 KB of free space at the end, but many decoration pixmaps are \
> smaller. Using mallopt to tune the threshold to 20 KB (i'm open for other \
> suggestions?) fixes the issue. After the patch:
>
> KWin in compositing mode: 19 MB
> KWin switched to non-compositing: 13 MB
>
>
> Arena 0:
> system bytes = 12374016
> in use bytes = 6894544
> Arena 1:
> system bytes = 135168
> in use bytes = 4784
> Total (incl. mmap):
> system bytes = 13750272
> in use bytes = 8140416
> max mmap regions = 67
> max mmap bytes = 99463168
>
> Are there some negative effects?
> The only negative effect i am aware of is, that Glibc free() is calling the sbrk \
> syscall more often. But this should not be a bottleneck.
>
>
> Diffs
> -----
>
> kwin/composite.cpp 9edb99d
> kwin/main.cpp f767f54
>
> Diff: http://git.reviewboard.kde.org/r/101385/diff
>
>
> Testing
> -------
>
> Tested using a standard Fedora 14.
> Would be nice to know, whether other OSes have similar issues?
> Martin Gräßlin had some concerns about the portability?
>
>
> Thanks,
>
> Philipp
>
>
[Attachment #5 (text/html)]
<html>
<body>
<div style="font-family: Verdana, Arial, Helvetica, Sans-Serif;">
<table bgcolor="#f9f3c9" width="100%" cellpadding="8" style="border: 1px #c9c399 \
solid;"> <tr>
<td>
This is an automatically generated e-mail. To reply, visit:
<a href="http://git.reviewboard.kde.org/r/101385/">http://git.reviewboard.kde.org/r/101385/</a>
</td>
</tr>
</table>
<br />
<blockquote style="margin-left: 1em; border-left: 2px solid #d0d0d0; padding-left: \
10px;"> <p style="margin-top: 0;">On May 19th, 2011, 7:34 a.m., <b>Aaron J. \
Seigo</b> wrote:</p> <blockquote style="margin-left: 1em; border-left: 2px solid \
#d0d0d0; padding-left: 10px;"> <pre style="white-space: pre-wrap; white-space: \
-moz-pre-wrap; white-space: -pre-wrap; white-space: -o-pre-wrap; word-wrap: \
break-word;">those are some impressive results. my guess is that we likely have a \
very similar issue in plasma. in both cases, we are dealing with lots of small bits \
of memory in the form of pixmaps, strings, etc. that are fairly constantly coming and \
going. switching between compositing and no compositing is probably the 'worst \
case' real world scenario for this.
if this does test out well, i'd suggest putting this code in libkworkspace so \
that all of our apps with this usage pattern (plasma-desktop, plasma-netbook, \
krunner, kwin, ..) can take advantage of a common implementation. a set of static \
methods in the KWorkspace namespace should suffice.
if you could repeat your experiment with plasma-desktop with and without such a \
patch, that would be great and let us know whether we should adopt such a strategy \
across the plasma shell components.
as for portability ... i'm not overly concerned about the portability of this to \
non-glibc systems as those systems are our primary targets. however, instead of \
checking for GLIBC the code could simply check for M_TRIM_THRESHOLD. for comparison, \
here is how busybox does this same trick for both the trim size and mmap request size \
threshold:
#ifndef PAGE_SIZE
# define PAGE_SIZE (4*1024) /* guess */
#endif
#ifdef M_TRIM_THRESHOLD
/* M_TRIM_THRESHOLD is the maximum amount of freed top-most memory
* to keep before releasing to the OS
* Default is way too big: 256k
*/
mallopt(M_TRIM_THRESHOLD, 2 * PAGE_SIZE);
#endif
#ifdef M_MMAP_THRESHOLD
/* M_MMAP_THRESHOLD is the request size threshold for using mmap()
* Default is too big: 256k
*/
mallopt(M_MMAP_THRESHOLD, 8 * PAGE_SIZE - 256);
#endif
i doubt we need to tweak the mmap strategy, but i like the checks for the definitions \
of mallopt there. for portability to non-unix-y systems, we could wrap this whole \
thing in a #ifndef Q_WS_WIN block which should keep the KDE Windows team from pulling \
their hair out over the use of such UNIX-y API ;)</pre> </blockquote>
</blockquote>
<pre style="white-space: pre-wrap; white-space: -moz-pre-wrap; white-space: \
-pre-wrap; white-space: -o-pre-wrap; word-wrap: break-word;">sysconf(_SC_PAGESIZE) \
returns the page size for mmap, no need for (4*1024)</pre> <br />
<p>- Ivan</p>
<br />
<p>On May 18th, 2011, 8:53 p.m., Philipp Knechtges wrote:</p>
<table bgcolor="#fefadf" width="100%" cellspacing="0" cellpadding="8" \
style="background-image: \
url('http://git.reviewboard.kde.org/media/rb/images/review_request_box_top_bg.png'); \
background-position: left top; background-repeat: repeat-x; border: 1px black \
solid;"> <tr>
<td>
<div>Review request for kwin and Plasma.</div>
<div>By Philipp Knechtges.</div>
<p style="color: grey;"><i>Updated May 18, 2011, 8:53 p.m.</i></p>
<h1 style="color: #575012; font-size: 10pt; margin-top: 1.5em;">Description </h1>
<table width="100%" bgcolor="#ffffff" cellspacing="0" cellpadding="10" style="border: \
1px solid #b8b5a0"> <tr>
<td>
<pre style="margin: 0; padding: 0; white-space: pre-wrap; white-space: \
-moz-pre-wrap; white-space: -pre-wrap; white-space: -o-pre-wrap; word-wrap: \
break-word;">The intention of this patch is to lower the heap fragmentation in KWin \
when using the raster backend.
One can illustrate the issue with the following testcase: If one currently uses the \
raster backend and switches back to non-compositing mode one gets easily a similar \
memory usage like the following:
Situation: 14 windows, QtCurve
KWin started with compositing: 40 MB
KWin switched to non-compositing : more than 70 MB
The first guess might be, that this is due to a memleak because of not properly \
released pixmaps. But calling malloc_stats() sheds some more light on the subject \
(non-compositing mode):
Arena 0:
system bytes = 72232960
in use bytes = 8180512
Arena 1:
system bytes = 135168
in use bytes = 4784
Total (incl. mmap):
system bytes = 73080832
in use bytes = 8898000
max mmap regions = 13
max mmap bytes = 36343808
In other words, the glibc has allocated more than 70 MB memory although KWin uses \
only less than 10 MB. The problem is that glibc only resizes the heap if the heap has \
more than 128 KB of free space at the end, but many decoration pixmaps are smaller. \
Using mallopt to tune the threshold to 20 KB (i'm open for other suggestions?) \
fixes the issue. After the patch:
KWin in compositing mode: 19 MB
KWin switched to non-compositing: 13 MB
Arena 0:
system bytes = 12374016
in use bytes = 6894544
Arena 1:
system bytes = 135168
in use bytes = 4784
Total (incl. mmap):
system bytes = 13750272
in use bytes = 8140416
max mmap regions = 67
max mmap bytes = 99463168
Are there some negative effects?
The only negative effect i am aware of is, that Glibc free() is calling the sbrk \
syscall more often. But this should not be a bottleneck.</pre>
</td>
</tr>
</table>
<h1 style="color: #575012; font-size: 10pt; margin-top: 1.5em;">Testing </h1>
<table width="100%" bgcolor="#ffffff" cellspacing="0" cellpadding="10" style="border: \
1px solid #b8b5a0"> <tr>
<td>
<pre style="margin: 0; padding: 0; white-space: pre-wrap; white-space: \
-moz-pre-wrap; white-space: -pre-wrap; white-space: -o-pre-wrap; word-wrap: \
break-word;">Tested using a standard Fedora 14. Would be nice to know, whether other \
OSes have similar issues? Martin Gräßlin had some concerns about the \
portability?</pre> </td>
</tr>
</table>
<h1 style="color: #575012; font-size: 10pt; margin-top: 1.5em;">Diffs</b> </h1>
<ul style="margin-left: 3em; padding-left: 0;">
<li>kwin/composite.cpp <span style="color: grey">(9edb99d)</span></li>
<li>kwin/main.cpp <span style="color: grey">(f767f54)</span></li>
</ul>
<p><a href="http://git.reviewboard.kde.org/r/101385/diff/" style="margin-left: \
3em;">View Diff</a></p>
</td>
</tr>
</table>
</div>
</body>
</html>
_______________________________________________
kwin mailing list
kwin@kde.org
https://mail.kde.org/mailman/listinfo/kwin
[prev in list] [next in list] [prev in thread] [next in thread]
Configure |
About |
News |
Add a list |
Sponsored by KoreLogic