[prev in list] [next in list] [prev in thread] [next in thread]
List: haiku-commits
Subject: [haiku-commits] Change in haiku[master]: rpmalloc: Import and set as default.
From: Gerrit <review () review ! haiku-os ! org>
Date: 2018-12-25 22:42:35
Message-ID: gerrit.1545777752434.I33d5adacca1ec027c982e88d893a240d969d28f6 () review ! haiku-os ! org
[Download RAW message or body]
From Michael Lotz <mmlr@mlotz=2Ech>:
Michael Lotz has uploaded this change=
for review=2E ( https://review=2Ehaiku-os=2Eorg/800
Change subject: rpma=
lloc: Import and set as default=2E
=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=
=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=
=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=
=2E=2E=2E=2E=2E=2E=2E
rpmalloc: Import and set as default=2E
Hoard, the L=
GPL-licensed locking thread-caching allocator that we have
used by default =
since libroot's introduction, is showing its age=2E
It is a "pseudo-sbrk-ba=
sed" allocator (it predates our actual sbrk,
so instead it uses a single Be=
area), which has serious limitations:
as we cannot ever move the area, we =
can only resize it "in place",
and so once we hit the end of the ~1=2E5GB r=
eserved part of the address
space for the heap, we are usually out of luck =
if we need more memory=2E
On 32-bit where userspace has only 2GB of address=
space anyway, but on
64-bit where address space is not a resource worth wo=
rrying about,
this can be a serious problem for applications that want to u=
se a
lot of RAM=2E As more and more large applications get ported to Haiku,=
the time for a mmap-based allocator has come=2E
For posterity's sake, her=
e are all the possible options there were,
and why this one was selected ra=
ther than one of them, beginning
with the immediate rejects:
* nedmalloc=
=2E Unmaintained since 2014 and all benchmarks show it
underperforming v=
s=2E virtually all other allocators=2E
* bmalloc=2E Significantly worse-pe=
rforming vs=2E other options on
this list with no apparent advantages=2E=
* hoard3=2E Now GPL only, which is obviously not acceptable for
use as=
a system library=2E
* ptmalloc2=2E glibc's default allocator; underperfor=
ms vs=2E
even most of the above-listeds=2E
And now on to the honorable =
mentions:
* tcmalloc=2E This is Google's allocator; it's designed for ser=
ver
and other high-performance workloads=2E As a result, it almost
ne=
ver unmaps memory unless ordered to do so in a very explicit
way, which =
obviously is unacceptable behavior for a general-purpose
allocator=2E
=
* jemalloc=2E This is FreeBSD and NetBSD's default allocator as well
as =
finding use in Firefox and Rust=2E It is again designed for
performance,=
with significantly higher memory overhead than
other allocators, especi=
ally for small heaps; which is of course
a problem for us, as we want to=
retain our light footprint=2E
Finally this brings us to rpmalloc=2E It's =
not as well-travelled as
tcmalloc or jemalloc, but by benchmarks done by it=
self [0] and
by developers of other allocators [1], it seems to typically h=
it
the "sweet spot" of very good performance with lower (if not the lowest)=
memory use out of all the other allocators it's tested against;
even beati=
ng jemalloc in certain benchmarks for speed, too=2E
You can see a descript=
ion of the allocator's design in its README [2]=2E
[0]: https://github=2Ec=
om/rampantpixels/rpmalloc/blob/master/BENCHMARKS=2Emd
[1]: https://github=
=2Ecom/ezrosent/allocators-rs/blob/master/info/elfmalloc-performance=2Emd
[=
2]: https://github=2Ecom/rampantpixels/rpmalloc#rpmalloc---rampant-pixels-m=
emory-allocator
In general testing thus far on Haiku, it appears to be a c=
onsistent
5-10% performance boost (39=2E0s -> 37=2E9s real / 52=2E8s -> 49=
=2E7s user)
when doing the "HaikuDepot compile" benchmark=2E Memory usage b=
y
app_server, launch_daemon, and Tracker with a handful of windows
after a =
cold boot changed negligibly (launch_daemon: 444K -> 284K,
app_server: 16=
=2E70MB -> 17=2E05MB, Tracker: 7=2E50MB -> 8=2E22MB -- mostly
due to the fa=
ct that a heap object is required per-thread now=2E)
TODO: WebKit hits an =
apparently unrelated assert failure with
this allocator; probably a r=
ace condition?
TODO: test long uptimes (it will probably do better than hoa=
rd,
as it can unmap memory in a way hoard can't)
TODO: notify communi=
ty before merge, so they can keep an eye out
---
M src/system/libroot/posix=
/Jamfile
A src/system/libroot/posix/rpmalloc/Jamfile
A src/system/libroot/p=
osix/rpmalloc/rpmalloc=2Ecpp
A src/system/libroot/posix/rpmalloc/rpmalloc=
=2Eh
A src/system/libroot/posix/rpmalloc/wrapper=2Ecpp
5 files changed, 2,3=
58 insertions(+), 1 deletion(-)
git pull ssh://git=2Ehaiku-os=2Eorg/ha=
iku refs/changes/00/800/1
--
To view, visit https://review=2Ehaiku-os=2Eor=
g/800
To unsubscribe, or for help writing mail filters, visit https://revie=
w=2Ehaiku-os=2Eorg/settings
Gerrit-Project: haiku
Gerrit-Branch: master
Ge=
rrit-MessageType: newchange
Gerrit-Change-Id: I33d5adacca1ec027c982e88d893a=
240d969d28f6
Gerrit-Change-Number: 800
Gerrit-PatchSet: 1
Gerrit-Owner: Mic=
hael Lotz <mmlr@mlotz=2Ech>
Gerrit-Reviewer: waddlesplash <waddlesplash@gma=
il=2Ecom>
[Attachment #3 (text/html)]
<p>Michael Lotz has uploaded this change for <strong>review</strong>.</p><p><a \
href="https://review.haiku-os.org/800">View Change</a></p><pre style="font-family: \
monospace,monospace; white-space: pre-wrap;">rpmalloc: Import and set as \
default.<br><br>Hoard, the LGPL-licensed locking thread-caching allocator that we \
have<br>used by default since libroot's introduction, is showing its age.<br>It \
is a "pseudo-sbrk-based" allocator (it predates our actual sbrk,<br>so \
instead it uses a single Be area), which has serious limitations:<br>as we cannot \
ever move the area, we can only resize it "in place",<br>and so once we hit \
the end of the ~1.5GB reserved part of the address<br>space for the heap, we are \
usually out of luck if we need more memory.<br>On 32-bit where userspace has only 2GB \
of address space anyway, but on<br>64-bit where address space is not a resource worth \
worrying about,<br>this can be a serious problem for applications that want to use \
a<br>lot of RAM. As more and more large applications get ported to Haiku,<br>the time \
for a mmap-based allocator has come.<br><br>For posterity's sake, here are all \
the possible options there were,<br>and why this one was selected rather than one of \
them, beginning<br>with the immediate rejects:<br><br> * nedmalloc. Unmaintained \
since 2014 and all benchmarks show it<br> underperforming vs. virtually all other \
allocators.<br> * bmalloc. Significantly worse-performing vs. other options on<br> \
this list with no apparent advantages.<br> * hoard3. Now GPL only, which is obviously \
not acceptable for<br> use as a system library.<br> * ptmalloc2. glibc's \
default allocator; underperforms vs.<br> even most of the above-listeds.<br><br>And \
now on to the honorable mentions:<br><br> * tcmalloc. This is Google's allocator; \
it's designed for server<br> and other high-performance workloads. As a result, \
it almost<br> never unmaps memory unless ordered to do so in a very explicit<br> \
way, which obviously is unacceptable behavior for a general-purpose<br> \
allocator.<br><br> * jemalloc. This is FreeBSD and NetBSD's default allocator as \
well<br> as finding use in Firefox and Rust. It is again designed for<br> \
performance, with significantly higher memory overhead than<br> other allocators, \
especially for small heaps; which is of course<br> a problem for us, as we want to \
retain our light footprint.<br><br>Finally this brings us to rpmalloc. It's not \
as well-travelled as<br>tcmalloc or jemalloc, but by benchmarks done by itself [0] \
and<br>by developers of other allocators [1], it seems to typically hit<br>the \
"sweet spot" of very good performance with lower (if not the \
lowest)<br>memory use out of all the other allocators it's tested \
against;<br>even beating jemalloc in certain benchmarks for speed, too.<br><br>You \
can see a description of the allocator's design in its README [2].<br><br>[0]: \
https://github.com/rampantpixels/rpmalloc/blob/master/BENCHMARKS.md<br>[1]: \
https://github.com/ezrosent/allocators-rs/blob/master/info/elfmalloc-performance.md<br>[2]: \
https://github.com/rampantpixels/rpmalloc#rpmalloc---rampant-pixels-memory-allocator<br><br>In \
general testing thus far on Haiku, it appears to be a consistent<br>5-10% performance \
boost (39.0s -> 37.9s real / 52.8s -> 49.7s user)<br>when doing the \
"HaikuDepot compile" benchmark. Memory usage by<br>app_server, \
launch_daemon, and Tracker with a handful of windows<br>after a cold boot changed \
negligibly (launch_daemon: 444K -> 284K,<br>app_server: 16.70MB -> 17.05MB, \
Tracker: 7.50MB -> 8.22MB -- mostly<br>due to the fact that a heap object is \
required per-thread now.)<br><br>TODO: WebKit hits an apparently unrelated assert \
failure with<br> this allocator; probably a race condition?<br>TODO: test long \
uptimes (it will probably do better than hoard,<br> as it can unmap memory in a \
way hoard can't)<br>TODO: notify community before merge, so they can keep an eye \
out<br>---<br>M src/system/libroot/posix/Jamfile<br>A \
src/system/libroot/posix/rpmalloc/Jamfile<br>A \
src/system/libroot/posix/rpmalloc/rpmalloc.cpp<br>A \
src/system/libroot/posix/rpmalloc/rpmalloc.h<br>A \
src/system/libroot/posix/rpmalloc/wrapper.cpp<br>5 files changed, 2,358 \
insertions(+), 1 deletion(-)<br><br></pre><pre style="font-family: \
monospace,monospace; white-space: pre-wrap;">git pull ssh://git.haiku-os.org/haiku \
refs/changes/00/800/1</pre><p>To view, visit <a \
href="https://review.haiku-os.org/800">change 800</a>. To unsubscribe, or for help \
writing mail filters, visit <a \
href="https://review.haiku-os.org/settings">settings</a>.</p><div itemscope \
itemtype="http://schema.org/EmailMessage"><div itemscope itemprop="action" \
itemtype="http://schema.org/ViewAction"><link itemprop="url" \
href="https://review.haiku-os.org/800"/><meta itemprop="name" content="View \
Change"/></div></div>
<div style="display:none"> Gerrit-Project: haiku </div>
<div style="display:none"> Gerrit-Branch: master </div>
<div style="display:none"> Gerrit-MessageType: newchange </div>
<div style="display:none"> Gerrit-Change-Id: \
I33d5adacca1ec027c982e88d893a240d969d28f6 </div> <div style="display:none"> \
Gerrit-Change-Number: 800 </div> <div style="display:none"> Gerrit-PatchSet: 1 </div>
<div style="display:none"> Gerrit-Owner: Michael Lotz <mmlr@mlotz.ch> </div>
<div style="display:none"> Gerrit-Reviewer: waddlesplash \
<waddlesplash@gmail.com> </div>
[prev in list] [next in list] [prev in thread] [next in thread]
Configure |
About |
News |
Add a list |
Sponsored by KoreLogic