[prev in list] [next in list] [prev in thread] [next in thread]
List: kwrite-devel
Subject: Re: accumulating projects in kate lead to excessive kate startup time due to git
From: Elv1313 <elv1313 () gmail ! com>
Date: 2022-06-19 19:26:52
Message-ID: CAFnK6VXRLhrfNqSrYg93-ifRg-qS2CGhS5SSaD0vGqA5Huvohw () mail ! gmail ! com
[Download RAW message or body]
Hi,
I just want to say this bug can be a little more annoying when using sshfs.
Most of my $dayjob work is done over `sshfs` because the development
environment is just a remote VM with everything setup properly and exposes
an LSP "server" and ssh. Doing many IO on that is rather expensive (hello
git!). I don't think parallelism would make the situation better. Async is
really the best option here. As for "why sshfs and not kio". That's because
of 2 things. First, kio+yubikey = get_a_popup_for_every_io_operation. That
would be rather invasive to fix (maybe adding some middleware/proxy process
that keeps a persistent ssh connection and Kio connects to that?). Also,
there's a little limitation in Kate LSP implementation. When using Kio, it
adds `fish://totally/wrong/path` to the LSP queries and the "server"
doesn't understand it. The $dayjob provided LSP expects either macOS or
Linux home directory paths and the project name (which it assumes is
sshfs-mounted).
On Sun, 19 Jun 2022 at 12:10, Milian Wolff <mail@milianw.de> wrote:
> On Samstag, 18. Juni 2022 14:15:42 CEST Milian Wolff wrote:
> > Hey all,
> >
> > Kate took ~4s to show its main window on my beefy workstation with lots
> of
> > RAM, CPUs and speedy NVME disks. I found this quite odd and wondered
> about
> > the reason so I sat down and profiled it. Perf shows a lot of external
> git
> > processes running sequentially, which I could also replicate with strace:
>
> <snip>
>
> > b) Can we query the git status in parallel for all projects, instead of
> > serially? My machine has 12 cores and 24 threads, and the NVME disk and
> ram
> > should also allow this.
>
> Sorry, hit sent too early...
>
> You can download the perfparser file here:
>
> https://milianw.de/files/kate.slow.startup.perfparser
>
> You can open that in hotspot and then go to the off-CPU time flame graph.
> Basically all of that comes from _really_ slow memory allocations, which
> is a
> first for me. It seems like my system is suffering from some extreme
> slowdowns
> in `int_malloc` - but only in kate. Other applications don't show this
> behavior, and I'm unsure where this comes from... See the excessively slow
> calls to rwsem_down_read_slowpath from _int_malloc, even in the main
> thread.
> If you look at the main thread e.g. there we see ~1s off cpu time from
> _int_realloc by _FcConfigParse::FcStrBufData alone!
>
> I'll try to continue to figure this out
> --
> Milian Wolff
> mail@milianw.de
> http://milianw.de
>
>
>
[Attachment #3 (text/html)]
<div dir="ltr"><div>Hi,</div><div><br></div><div>I just want to say this bug can be a \
little more annoying when using sshfs. Most of my $dayjob work is done over `sshfs` \
because the development environment is just a remote VM with everything setup \
properly and exposes an LSP "server" and ssh. Doing many IO on that is \
rather expensive (hello git!). I don't think parallelism would make the situation \
better. Async is really the best option here. As for "why sshfs and not \
kio". That's because of 2 things. First, kio+yubikey = \
get_a_popup_for_every_io_operation. That would be rather invasive to fix (maybe \
adding some middleware/proxy process that keeps a persistent ssh connection and Kio \
connects to that?). Also, there's a little limitation in Kate LSP implementation. \
When using Kio, it adds `fish://totally/wrong/path` to the LSP queries and the \
"server" doesn't understand it. The $dayjob provided LSP expects either \
macOS or Linux home directory paths and the project name (which it assumes is \
sshfs-mounted).</div></div><br><div class="gmail_quote"><div dir="ltr" \
class="gmail_attr">On Sun, 19 Jun 2022 at 12:10, Milian Wolff <<a \
href="mailto:mail@milianw.de">mail@milianw.de</a>> wrote:<br></div><blockquote \
class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid \
rgb(204,204,204);padding-left:1ex">On Samstag, 18. Juni 2022 14:15:42 CEST Milian \
Wolff wrote:<br> > Hey all,<br>
> <br>
> Kate took ~4s to show its main window on my beefy workstation with lots of<br>
> RAM, CPUs and speedy NVME disks. I found this quite odd and wondered about<br>
> the reason so I sat down and profiled it. Perf shows a lot of external git<br>
> processes running sequentially, which I could also replicate with strace:<br>
<br>
<snip><br>
<br>
> b) Can we query the git status in parallel for all projects, instead of<br>
> serially? My machine has 12 cores and 24 threads, and the NVME disk and ram<br>
> should also allow this.<br>
<br>
Sorry, hit sent too early...<br>
<br>
You can download the perfparser file here:<br>
<br>
<a href="https://milianw.de/files/kate.slow.startup.perfparser" rel="noreferrer" \
target="_blank">https://milianw.de/files/kate.slow.startup.perfparser</a><br> <br>
You can open that in hotspot and then go to the off-CPU time flame graph. <br>
Basically all of that comes from _really_ slow memory allocations, which is a <br>
first for me. It seems like my system is suffering from some extreme slowdowns <br>
in `int_malloc` - but only in kate. Other applications don't show this <br>
behavior, and I'm unsure where this comes from... See the excessively slow <br>
calls to rwsem_down_read_slowpath from _int_malloc, even in the main thread. <br>
If you look at the main thread e.g. there we see ~1s off cpu time from <br>
_int_realloc by _FcConfigParse::FcStrBufData alone!<br>
<br>
I'll try to continue to figure this out<br>
-- <br>
Milian Wolff<br>
<a href="mailto:mail@milianw.de" target="_blank">mail@milianw.de</a><br>
<a href="http://milianw.de" rel="noreferrer" \
target="_blank">http://milianw.de</a><br> <br>
<br>
</blockquote></div>
[prev in list] [next in list] [prev in thread] [next in thread]
Configure |
About |
News |
Add a list |
Sponsored by KoreLogic