'Re: Enable EarlyOOM on Fedora KDE - Fedora 33 Self-Contained Change proposal'

[prev in list] [next in list] [prev in thread] [next in thread] 

List:       fedora-devel-list
Subject:    Re: Enable EarlyOOM on Fedora KDE - Fedora 33 Self-Contained Change proposal
From:       Simon Farnsworth <simon () farnz ! org ! uk>
Date:       2020-07-20 17:47:14
Message-ID: FFCA01C2-7D27-4321-A804-196C6F1B21BF () farnz ! org ! uk
[Download RAW message or body]

[Attachment #2 (multipart/alternative)]


> On 20 Jul 2020, at 15:40, Stephen John Smoogen <smooge@gmail.com> wrote:
> 
> On Mon, 20 Jul 2020 at 05:04, Kevin Kofler <kevin.kofler@chello.at \
> <mailto:kevin.kofler@chello.at>> wrote:
> > 
> > John M. Harris Jr wrote:
> > > Userspace isn't dead when a system is thrashing. Your software is still
> > > running. If it gets killed, you're most likely going to lose your data.
> > 
> > The thing is, there are various levels of thrashing. In some cases, the
> > system is so busy that you have no chance to bring it back to responsiveness
> > for many minutes, up to hours. (Other than hitting the Reset or Power
> > button, of course.) I have had cases where not even sshd would respond. (The
> > fact that login has been blocking on D-Bus since the introduction of
> > systemd-logind does not help either. Login timeouts are something that was
> > just never happening in the past, now they are common under heavy load.)
> > 
> > That said, I do not see how the EarlyOOM heuristic, which allows, depending
> > on the exact settings, something like 80-90% of swap to be used IN ADDITION
> > to 90+% RAM (and will only start doing anything if BOTH RAM and swap are
> > full) can prevent thrashing in any reliable way. My thrashing scenarios have
> > had much less swap than that used. (I have twice as much swap than RAM, so
> > when the EarlyOOM heuristics trigger, my programs are already trying to use
> > almost 3 times as much RAM as is actually available!)
> > 
> 
> I think the problem is that you are using way too much swap for modern
> systems. The reasons for swap being 2x to 4x real memory was a 1980's
> solution when big RAM systems had 64 MB of ram but a server might need
> 128MB for certain tasks. This was 'reasonable' because the processors
> were slow but could still walk through 128 MB of space 'pretty' fast.
> As RAM got larger this 2x became 'cargo-culted' in various
> documentation and was still reasonable while processor speed went up.
> You could still have a system with 512MB of ram and walk through 1024
> to 2048 GB of swap in similar times as the 128 MB.

Also affected by the way some UNIXes handled commit and fork (the things Linux \
heuristic overcommit was designed to avoid needing excess swap for). In the best \
case, you needed as much swap as RAM, so that a full size process (64 MB in your \
1980s system) could allocate an extra 64 MB to fork; in the worst case, things were \
using that swap and you needed some multiple just for commit accounting.

IIRC, there were also some systems where the available space for committing was set \
to the size of swap - hence the need for 2x RAM. One for matching your RAM and \
allowing you to use it, one for the space consumed during a fork.

Linux's use of overcommit means that this isn't an issue for Fedora, though.
-- 
Simon


[Attachment #5 (unknown)]

<html><head><meta http-equiv="Content-Type" content="text/html; \
charset=us-ascii"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: \
space; line-break: after-white-space;" class=""><br class=""><div><br \
class=""><blockquote type="cite" class=""><div class="">On 20 Jul 2020, at 15:40, \
Stephen John Smoogen &lt;<a href="mailto:smooge@gmail.com" \
class="">smooge@gmail.com</a>&gt; wrote:</div><br \
class="Apple-interchange-newline"><div class=""><span style="caret-color: rgb(0, 0, \
0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: \
normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: \
0px; text-transform: none; white-space: normal; word-spacing: 0px; \
-webkit-text-stroke-width: 0px; text-decoration: none; float: none; display: inline \
!important;" class="">On Mon, 20 Jul 2020 at 05:04, Kevin Kofler &lt;</span><a \
href="mailto:kevin.kofler@chello.at" style="font-family: Helvetica; font-size: 12px; \
font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: \
normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; \
white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-size-adjust: auto; \
-webkit-text-stroke-width: 0px;" class="">kevin.kofler@chello.at</a><span \
style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; \
font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: \
normal; text-align: start; text-indent: 0px; text-transform: none; white-space: \
normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; \
float: none; display: inline !important;" class="">&gt; wrote:</span><br \
style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; \
font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: \
normal; text-align: start; text-indent: 0px; text-transform: none; white-space: \
normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;" \
class=""><blockquote type="cite" style="font-family: Helvetica; font-size: 12px; \
font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: \
normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; \
white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-size-adjust: auto; \
-webkit-text-stroke-width: 0px; text-decoration: none;" class=""><br class="">John M. \
Harris Jr wrote:<br class=""><blockquote type="cite" class="">Userspace isn't dead \
when a system is thrashing. Your software is still<br class="">running. If it gets \
killed, you're most likely going to lose your data.<br class=""></blockquote><br \
class="">The thing is, there are various levels of thrashing. In some cases, the<br \
class="">system is so busy that you have no chance to bring it back to \
responsiveness<br class="">for many minutes, up to hours. (Other than hitting the \
Reset or Power<br class="">button, of course.) I have had cases where not even sshd \
would respond. (The<br class="">fact that login has been blocking on D-Bus since the \
introduction of<br class="">systemd-logind does not help either. Login timeouts are \
something that was<br class="">just never happening in the past, now they are common \
under heavy load.)<br class=""><br class="">That said, I do not see how the EarlyOOM \
heuristic, which allows, depending<br class="">on the exact settings, something like \
80-90% of swap to be used IN ADDITION<br class="">to 90+% RAM (and will only start \
doing anything if BOTH RAM and swap are<br class="">full) can prevent thrashing in \
any reliable way. My thrashing scenarios have<br class="">had much less swap than \
that used. (I have twice as much swap than RAM, so<br class="">when the EarlyOOM \
heuristics trigger, my programs are already trying to use<br class="">almost 3 times \
as much RAM as is actually available!)<br class=""><br class=""></blockquote><br \
style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; \
font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: \
normal; text-align: start; text-indent: 0px; text-transform: none; white-space: \
normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;" \
class=""><span style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: \
12px; font-style: normal; font-variant-caps: normal; font-weight: normal; \
letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; \
white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; \
text-decoration: none; float: none; display: inline !important;" class="">I think the \
problem is that you are using way too much swap for modern</span><br \
style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; \
font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: \
normal; text-align: start; text-indent: 0px; text-transform: none; white-space: \
normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;" \
class=""><span style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: \
12px; font-style: normal; font-variant-caps: normal; font-weight: normal; \
letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; \
white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; \
text-decoration: none; float: none; display: inline !important;" class="">systems. \
The reasons for swap being 2x to 4x real memory was a 1980's</span><br \
style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; \
font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: \
normal; text-align: start; text-indent: 0px; text-transform: none; white-space: \
normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;" \
class=""><span style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: \
12px; font-style: normal; font-variant-caps: normal; font-weight: normal; \
letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; \
white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; \
text-decoration: none; float: none; display: inline !important;" class="">solution \
when big RAM systems had 64 MB of ram but a server might need</span><br \
style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; \
font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: \
normal; text-align: start; text-indent: 0px; text-transform: none; white-space: \
normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;" \
class=""><span style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: \
12px; font-style: normal; font-variant-caps: normal; font-weight: normal; \
letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; \
white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; \
text-decoration: none; float: none; display: inline !important;" class="">128MB for \
certain tasks. This was 'reasonable' because the processors</span><br \
style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; \
font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: \
normal; text-align: start; text-indent: 0px; text-transform: none; white-space: \
normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;" \
class=""><span style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: \
12px; font-style: normal; font-variant-caps: normal; font-weight: normal; \
letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; \
white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; \
text-decoration: none; float: none; display: inline !important;" class="">were slow \
but could still walk through 128 MB of space 'pretty' fast.</span><br \
style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; \
font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: \
normal; text-align: start; text-indent: 0px; text-transform: none; white-space: \
normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;" \
class=""><span style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: \
12px; font-style: normal; font-variant-caps: normal; font-weight: normal; \
letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; \
white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; \
text-decoration: none; float: none; display: inline !important;" class="">As RAM got \
larger this 2x became 'cargo-culted' in various</span><br style="caret-color: rgb(0, \
0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; \
font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: \
start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: \
0px; -webkit-text-stroke-width: 0px; text-decoration: none;" class=""><span \
style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; \
font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: \
normal; text-align: start; text-indent: 0px; text-transform: none; white-space: \
normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; \
float: none; display: inline !important;" class="">documentation and was still \
reasonable while processor speed went up.</span><br style="caret-color: rgb(0, 0, 0); \
font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: \
normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: \
0px; text-transform: none; white-space: normal; word-spacing: 0px; \
-webkit-text-stroke-width: 0px; text-decoration: none;" class=""><span \
style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; \
font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: \
normal; text-align: start; text-indent: 0px; text-transform: none; white-space: \
normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; \
float: none; display: inline !important;" class="">You could still have a system with \
512MB of ram and walk through 1024</span><br style="caret-color: rgb(0, 0, 0); \
font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: \
normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: \
0px; text-transform: none; white-space: normal; word-spacing: 0px; \
-webkit-text-stroke-width: 0px; text-decoration: none;" class=""><span \
style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; \
font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: \
normal; text-align: start; text-indent: 0px; text-transform: none; white-space: \
normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; \
float: none; display: inline !important;" class="">to 2048 GB of swap in similar \
times as the 128 MB.</span><br style="caret-color: rgb(0, 0, 0); font-family: \
Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; \
font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; \
text-transform: none; white-space: normal; word-spacing: 0px; \
-webkit-text-stroke-width: 0px; text-decoration: none;" \
class=""></div></blockquote></div><br class=""><div class="">Also affected by the way \
some UNIXes handled commit and fork (the things Linux heuristic overcommit was \
designed to avoid needing excess swap for). In the best case, you needed as much swap \
as RAM, so that a full size process (64 MB in your 1980s system) could allocate an \
extra 64 MB to fork; in the worst case, things were using that swap and you needed \
some multiple just for commit accounting.</div><div class=""><br class=""></div><div \
class="">IIRC, there were also some systems where the available space for committing \
was set to the size of swap - hence the need for 2x RAM. One for matching your RAM \
and allowing you to use it, one for the space consumed during a fork.</div><div \
class=""><br class=""></div><div class="">Linux's use of overcommit means that this \
isn't an issue for Fedora, though.</div><div class="">--&nbsp;</div><div \
class="">Simon</div></body></html>


[Attachment #6 (text/plain)]

_______________________________________________
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-leave@lists.fedoraproject.org
Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org


[prev in list] [next in list] [prev in thread] [next in thread]
Configure | About | News | Add a list | Sponsored by KoreLogic