From kwrite-devel Thu Jun 23 12:00:40 2022 From: Dominik Haumann Date: Thu, 23 Jun 2022 12:00:40 +0000 To: kwrite-devel Subject: Re: accumulating projects in kate lead to excessive kate startup time due to git Message-Id: X-MARC-Message: https://marc.info/?l=kwrite-devel&m=165600391624580 MIME-Version: 1 Content-Type: multipart/mixed; boundary="--00000000000086c22e05e21c349a" --00000000000086c22e05e21c349a Content-Type: text/plain; charset="UTF-8" With respect to using sshfs: the only thing I could imagine is to use KMountPoint::probablySlow() and then exclude projects that are located on a network drive. See: https://api.kde.org/frameworks/kio/html/classKMountPoint.html#a8f99e2505a979ed8e589af0ea068a355 We'd need an additional checkbox for disabling auto-reloading of remote projects. This would be an addition to https://invent.kde.org/utilities/kate/-/merge_requests/672 Greetings Dominik On Sun, Jun 19, 2022 at 9:27 PM Elv1313 wrote: > Hi, > > I just want to say this bug can be a little more annoying when using > sshfs. Most of my $dayjob work is done over `sshfs` because the development > environment is just a remote VM with everything setup properly and exposes > an LSP "server" and ssh. Doing many IO on that is rather expensive (hello > git!). I don't think parallelism would make the situation better. Async is > really the best option here. As for "why sshfs and not kio". That's because > of 2 things. First, kio+yubikey = get_a_popup_for_every_io_operation. That > would be rather invasive to fix (maybe adding some middleware/proxy process > that keeps a persistent ssh connection and Kio connects to that?). Also, > there's a little limitation in Kate LSP implementation. When using Kio, it > adds `fish://totally/wrong/path` to the LSP queries and the "server" > doesn't understand it. The $dayjob provided LSP expects either macOS or > Linux home directory paths and the project name (which it assumes is > sshfs-mounted). > > On Sun, 19 Jun 2022 at 12:10, Milian Wolff wrote: > >> On Samstag, 18. Juni 2022 14:15:42 CEST Milian Wolff wrote: >> > Hey all, >> > >> > Kate took ~4s to show its main window on my beefy workstation with lots >> of >> > RAM, CPUs and speedy NVME disks. I found this quite odd and wondered >> about >> > the reason so I sat down and profiled it. Perf shows a lot of external >> git >> > processes running sequentially, which I could also replicate with >> strace: >> >> >> >> > b) Can we query the git status in parallel for all projects, instead of >> > serially? My machine has 12 cores and 24 threads, and the NVME disk and >> ram >> > should also allow this. >> >> Sorry, hit sent too early... >> >> You can download the perfparser file here: >> >> https://milianw.de/files/kate.slow.startup.perfparser >> >> You can open that in hotspot and then go to the off-CPU time flame graph. >> Basically all of that comes from _really_ slow memory allocations, which >> is a >> first for me. It seems like my system is suffering from some extreme >> slowdowns >> in `int_malloc` - but only in kate. Other applications don't show this >> behavior, and I'm unsure where this comes from... See the excessively >> slow >> calls to rwsem_down_read_slowpath from _int_malloc, even in the main >> thread. >> If you look at the main thread e.g. there we see ~1s off cpu time from >> _int_realloc by _FcConfigParse::FcStrBufData alone! >> >> I'll try to continue to figure this out >> -- >> Milian Wolff >> mail@milianw.de >> http://milianw.de >> >> >> --00000000000086c22e05e21c349a Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
With respect to using sshfs: the only thing I could i= magine is to use KMountPoint::probablySlow() and then exclude projects that= are located on a network drive.


We&#= 39;d need an additional checkbox for disabling auto-reloading of remote pro= jects.


Greet= ings
Dominik


On Sun, Jun 19, 2022 at 9:27 PM = Elv1313 <elv1313@gmail.com> = wrote:
Hi,

I just want to say this bug can = be a little more annoying when using sshfs. Most of my $dayjob work is done= over `sshfs` because the development environment is just a remote VM with = everything setup properly and exposes an LSP "server" and ssh. Do= ing many IO on that is rather expensive (hello git!). I don't think par= allelism would make the situation better. Async is really the best option h= ere. As for "why sshfs and not kio". That's because of 2 thin= gs. First, kio+yubikey =3D get_a_popup_for_every_io_operation. That would b= e rather invasive to fix (maybe adding some middleware/proxy process that k= eeps a persistent ssh connection and Kio connects to that?). Also, there= 9;s a little limitation in Kate LSP implementation. When using Kio, it adds= `fish://totally/wrong/path` to the LSP queries and the "server" = doesn't understand it. The $dayjob provided LSP expects either macOS or= Linux home directory paths and the project name (which it assumes is sshfs= -mounted).

On Sun, 19 Jun 2022 at 12:10, Milian Wolff <mail@milianw.de> wrote:
=
On Samstag, 18. Jun= i 2022 14:15:42 CEST Milian Wolff wrote:
> Hey all,
>
> Kate took ~4s to show its main window on my beefy workstation with lot= s of
> RAM, CPUs and speedy NVME disks. I found this quite odd and wondered a= bout
> the reason so I sat down and profiled it. Perf shows a lot of external= git
> processes running sequentially, which I could also replicate with stra= ce:

<snip>

> b) Can we query the git status in parallel for all projects, instead o= f
> serially? My machine has 12 cores and 24 threads, and the NVME disk an= d ram
> should also allow this.

Sorry, hit sent too early...

You can download the perfparser file here:

https://milianw.de/files/kate.slow.startup.perf= parser

You can open that in hotspot and then go to the off-CPU time flame graph. <= br> Basically all of that comes from _really_ slow memory allocations, which is= a
first for me. It seems like my system is suffering from some extreme slowdo= wns
in `int_malloc` - but only in kate. Other applications don't show this =
behavior, and I'm unsure where this comes from... See the excessively s= low
calls to rwsem_down_read_slowpath from _int_malloc, even in the main thread= .
If you look at the main thread e.g. there we see ~1s off cpu time from
_int_realloc by _FcConfigParse::FcStrBufData alone!

I'll try to continue to figure this out
--
Milian Wolff
mail@milianw.de http://m= ilianw.de


--00000000000086c22e05e21c349a--