[prev in list] [next in list] [prev in thread] [next in thread]
List: kwrite-devel
Subject: Re: accumulating projects in kate lead to excessive kate startup time due to git
From: Dominik Haumann <dhaumann () gmail ! com>
Date: 2022-06-23 12:00:40
Message-ID: CALi_srCfRFmBqXVeN2=C44VkeWCLMYS+677rdA-jXpqZ8E4J9A () mail ! gmail ! com
[Download RAW message or body]
With respect to using sshfs: the only thing I could imagine is to use
KMountPoint::probablySlow() and then exclude projects that are located on a
network drive.
See:
https://api.kde.org/frameworks/kio/html/classKMountPoint.html#a8f99e2505a979ed8e589af0ea068a355
We'd need an additional checkbox for disabling auto-reloading of remote
projects.
This would be an addition to
https://invent.kde.org/utilities/kate/-/merge_requests/672
Greetings
Dominik
On Sun, Jun 19, 2022 at 9:27 PM Elv1313 <elv1313@gmail.com> wrote:
> Hi,
>
> I just want to say this bug can be a little more annoying when using
> sshfs. Most of my $dayjob work is done over `sshfs` because the development
> environment is just a remote VM with everything setup properly and exposes
> an LSP "server" and ssh. Doing many IO on that is rather expensive (hello
> git!). I don't think parallelism would make the situation better. Async is
> really the best option here. As for "why sshfs and not kio". That's because
> of 2 things. First, kio+yubikey = get_a_popup_for_every_io_operation. That
> would be rather invasive to fix (maybe adding some middleware/proxy process
> that keeps a persistent ssh connection and Kio connects to that?). Also,
> there's a little limitation in Kate LSP implementation. When using Kio, it
> adds `fish://totally/wrong/path` to the LSP queries and the "server"
> doesn't understand it. The $dayjob provided LSP expects either macOS or
> Linux home directory paths and the project name (which it assumes is
> sshfs-mounted).
>
> On Sun, 19 Jun 2022 at 12:10, Milian Wolff <mail@milianw.de> wrote:
>
>> On Samstag, 18. Juni 2022 14:15:42 CEST Milian Wolff wrote:
>> > Hey all,
>> >
>> > Kate took ~4s to show its main window on my beefy workstation with lots
>> of
>> > RAM, CPUs and speedy NVME disks. I found this quite odd and wondered
>> about
>> > the reason so I sat down and profiled it. Perf shows a lot of external
>> git
>> > processes running sequentially, which I could also replicate with
>> strace:
>>
>> <snip>
>>
>> > b) Can we query the git status in parallel for all projects, instead of
>> > serially? My machine has 12 cores and 24 threads, and the NVME disk and
>> ram
>> > should also allow this.
>>
>> Sorry, hit sent too early...
>>
>> You can download the perfparser file here:
>>
>> https://milianw.de/files/kate.slow.startup.perfparser
>>
>> You can open that in hotspot and then go to the off-CPU time flame graph.
>> Basically all of that comes from _really_ slow memory allocations, which
>> is a
>> first for me. It seems like my system is suffering from some extreme
>> slowdowns
>> in `int_malloc` - but only in kate. Other applications don't show this
>> behavior, and I'm unsure where this comes from... See the excessively
>> slow
>> calls to rwsem_down_read_slowpath from _int_malloc, even in the main
>> thread.
>> If you look at the main thread e.g. there we see ~1s off cpu time from
>> _int_realloc by _FcConfigParse::FcStrBufData alone!
>>
>> I'll try to continue to figure this out
>> --
>> Milian Wolff
>> mail@milianw.de
>> http://milianw.de
>>
>>
>>
[Attachment #3 (text/html)]
<div dir="ltr"><div>With respect to using sshfs: the only thing I could imagine is to \
use KMountPoint::probablySlow() and then exclude projects that are located on a \
network drive.</div><div><br></div><div>See: <a \
href="https://api.kde.org/frameworks/kio/html/classKMountPoint.html#a8f99e2505a979ed8e \
589af0ea068a355">https://api.kde.org/frameworks/kio/html/classKMountPoint.html#a8f99e2505a979ed8e589af0ea068a355</a></div><div><br></div><div>We'd \
need an additional checkbox for disabling auto-reloading of remote \
projects.</div><div>This would be an addition to <a \
href="https://invent.kde.org/utilities/kate/-/merge_requests/672">https://invent.kde.o \
rg/utilities/kate/-/merge_requests/672</a></div><div><br></div><div><br></div><div>Greetings</div><div>Dominik</div><div><br></div></div><br><div \
class="gmail_quote"><div dir="ltr" class="gmail_attr">On Sun, Jun 19, 2022 at 9:27 PM \
Elv1313 <<a href="mailto:elv1313@gmail.com">elv1313@gmail.com</a>> \
wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px \
0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div \
dir="ltr"><div>Hi,</div><div><br></div><div>I just want to say this bug can be a \
little more annoying when using sshfs. Most of my $dayjob work is done over `sshfs` \
because the development environment is just a remote VM with everything setup \
properly and exposes an LSP "server" and ssh. Doing many IO on that is \
rather expensive (hello git!). I don't think parallelism would make the situation \
better. Async is really the best option here. As for "why sshfs and not \
kio". That's because of 2 things. First, kio+yubikey = \
get_a_popup_for_every_io_operation. That would be rather invasive to fix (maybe \
adding some middleware/proxy process that keeps a persistent ssh connection and Kio \
connects to that?). Also, there's a little limitation in Kate LSP implementation. \
When using Kio, it adds `fish://totally/wrong/path` to the LSP queries and the \
"server" doesn't understand it. The $dayjob provided LSP expects either \
macOS or Linux home directory paths and the project name (which it assumes is \
sshfs-mounted).</div></div><br><div class="gmail_quote"><div dir="ltr" \
class="gmail_attr">On Sun, 19 Jun 2022 at 12:10, Milian Wolff <<a \
href="mailto:mail@milianw.de" target="_blank">mail@milianw.de</a>> \
wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px \
0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">On Samstag, 18. Juni \
2022 14:15:42 CEST Milian Wolff wrote:<br> > Hey all,<br>
> <br>
> Kate took ~4s to show its main window on my beefy workstation with lots of<br>
> RAM, CPUs and speedy NVME disks. I found this quite odd and wondered about<br>
> the reason so I sat down and profiled it. Perf shows a lot of external git<br>
> processes running sequentially, which I could also replicate with strace:<br>
<br>
<snip><br>
<br>
> b) Can we query the git status in parallel for all projects, instead of<br>
> serially? My machine has 12 cores and 24 threads, and the NVME disk and ram<br>
> should also allow this.<br>
<br>
Sorry, hit sent too early...<br>
<br>
You can download the perfparser file here:<br>
<br>
<a href="https://milianw.de/files/kate.slow.startup.perfparser" rel="noreferrer" \
target="_blank">https://milianw.de/files/kate.slow.startup.perfparser</a><br> <br>
You can open that in hotspot and then go to the off-CPU time flame graph. <br>
Basically all of that comes from _really_ slow memory allocations, which is a <br>
first for me. It seems like my system is suffering from some extreme slowdowns <br>
in `int_malloc` - but only in kate. Other applications don't show this <br>
behavior, and I'm unsure where this comes from... See the excessively slow <br>
calls to rwsem_down_read_slowpath from _int_malloc, even in the main thread. <br>
If you look at the main thread e.g. there we see ~1s off cpu time from <br>
_int_realloc by _FcConfigParse::FcStrBufData alone!<br>
<br>
I'll try to continue to figure this out<br>
-- <br>
Milian Wolff<br>
<a href="mailto:mail@milianw.de" target="_blank">mail@milianw.de</a><br>
<a href="http://milianw.de" rel="noreferrer" \
target="_blank">http://milianw.de</a><br> <br>
<br>
</blockquote></div>
</blockquote></div>
[prev in list] [next in list] [prev in thread] [next in thread]
Configure |
About |
News |
Add a list |
Sponsored by KoreLogic