--nextPart2101211.VLH7GnMWUR Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="UTF-8"; protected-headers="v1" From: Milian Wolff To: kde-core-devel@kde.org Cc: Aleix Pol Subject: Re: RFC: an improved ki18n_install Date: Tue, 07 Mar 2023 09:21:07 +0100 Message-ID: <6158128.MhkbZ0Pkbq@agathemoarbauer> MIME-Version: 1.0 On Dienstag, 7. M=C3=A4rz 2023 02:51:06 CET Aleix Pol wrote: > On Mon, Mar 6, 2023 at 8:31=E2=80=AFPM Milian Wolff wro= te: > > Hey all, > >=20 > > for context please see [1] and [2]. > >=20 > > [1]: https://bugs.kde.org/show_bug.cgi?id=3D460245 > > [2]: > > https://discourse.cmake.org/t/feature-request-add-custom-command-all/76= 09 > >=20 > > The gist is that me and Igor are annoyed by `ki18n_install` abusing > > `add_custom_target`, which makes the build always dirty. I.e. every time > > we > > run `ninja` we will, at the very least, run the two mo & ts generation > > targets. For kdevelop, these do non-trivial amounts of work that takes > > time > > even on a beefy laptop: > >=20 > > ``` > > $ ninja && time ninja > > [2/2] Generating mo... > > [2/2] Generating mo... > >=20 > > real 0m1,780s > > user 0m5,742s > > sys 0m2,327s > > ``` > >=20 > > I would like to fix this, but first want to get feedback from the KDE > > developers at large. > >=20 > > First of all: Are all projects using ki18n_install in the way we do? Why > > is > > no-one else complaining about this so far? Are your po files so small t= hat > > this doesn't take a significant amount of time? Or are there potentially > > just so few of them in your project? KDevelop is heavily plugin based > > which means we have tons of po files. 2464 *.po files to be exact... > >=20 > > Secondly: does anyone know how to best approach a solution to this issu= e? > > The problem is that I don't see an easy way to solve it: While Kyle > > Edwards was nice enough to show me a way to tell CMake to not make the > > custom target always-dirty, `k18n_install` suffers from another design > > deficiency: It uses globbing internally and has an undefined amount of > > inputs and outputs, which basically makes it impossible for us to > > leverage `add_custom_command(OUTPUT` here, or? > >=20 > > As such, it seems like one would need a different approach to this prob= lem > > anyways. Globbing is a known-evil in cmake land, and I would personally > > prefer to have the inputs explicitly listed, just like we do for source > > files etc. Because we have many languages though, listing all possible > > *.po or *.ts files is cumbersome. Instead, what about we maintain the > > list of known languages in the ki18n framework? Then we could have > > something like > >=20 > > ``` > > ki18n_target( > >=20 > > # the target for which we are handling messages > > TARGET KF5I18n > > # the translation domain, added as compile_options and to find files > > TRANSLATION_DOMAIN ki18n5 > > # the root po folder location, to look for the .po/.js files > > PO_FOLDER ${KI18n_SOURCE_DIR}/po > >=20 > > ) > > ``` > >=20 > > Internally, this would do what is currently done manually: > >=20 > > ``` > > target_compile_options(KF5I18n PRIVATE -DTRANSLATION_DOMAIN=3D\"ki18n5\= ") > > ``` > >=20 > > Then it would iterate over the list of known languages and try to find = the > > .po or .js file. For every match, we would add_custom_target to generate > > the corresponding .mo/.ts file and add the reverse dependency. > >=20 > > What do others say to this approach? > >=20 > > Thanks > >=20 > > -- > > Milian Wolff > > http://milianw.de >=20 > +1 to getting the problem solved. I've also been bothered by this and > thought about looking into this so thanks for stepping up! (also I'm > pretty sure this code is originally mine from a time when we didn't > generate translations on every single build ^^'). >=20 > Having ninja/make do this dependency generation can end up being more > work than worth it because then we need to have a static list and then > we need to re-run it when the po files change and it's all a mess. The only mess that I can see is having to maintain the static list. Otherwi= se,=20 we would just piggy back on the underlying buildsystem to drive the generat= ion=20 for us. Meaning, we would get separate targets for every .po/.js, leading t= o=20 proper parallelization too. And only new/changed files would get re-generat= ed=20 as needed. Anyhow, reaping those benefits is going to be hard due to the static list. = As=20 Albert said in his email, my proposal has no advantages over just using glo= b.=20 As such, we have basically three options: a) status quo: + trivial to setup + will always correctly track changes to the .po/.js files - serial instead of parallel generation of .mo/.ts files - always dirty ALL target b) glob with separate targets: + trivial to setup + parallelized generation of .mo/.ts files + clean ALL target - detecting new .po/.js files requires manual re-run of cmake (changes to= =20 existing .po/.js files will be detected automatically) c) static list: + parallelized generation of .mo/.ts files + clean ALL target - hard to setup, to maintain the static list I think c) would only be an option paired with some scripting. I.e. ideally= we=20 would let scripty maintain the static list for us when it syncs translation= s? If we do that, then I think it would be the best solution. What do others=20 think? > How > about changing build-pofiles/build-tsfiles to only execute the process > if the origin file is newer than the target? > > I've put together a MR to explain what I meant (and I guess out of > guilt for having produced this :D) > https://invent.kde.org/frameworks/ki18n/-/merge_requests/81 Thanks, that sounds like a great improvement that should get in one way or= =20 another. Can you also backport that to KF5 (or will there be no more releas= es=20 of that?) Thanks =2D-=20 Milian Wolff http://milianw.de --nextPart2101211.VLH7GnMWUR Content-Type: application/pgp-signature; name="signature.asc" Content-Description: This is a digitally signed message part. Content-Transfer-Encoding: 7Bit -----BEGIN PGP SIGNATURE----- iQIzBAABCAAdFiEEezawi1aUvUGg3A1+8zYW/HGdOX8FAmQG8/MACgkQ8zYW/HGd OX8tIxAAyU3UfHaIRfDJlxq3oLhHLT05ylD5Lw7SYZxH36gLWCqBQLqq+yQZt2Rj B6DaNhBfFGyJFAzeY/v1DpUaQiARlJEaepULdC4Wch/j7db/MjyVOWHmUp3z71Y/ vMqovCTM3w1CaXGh84HRKucWBzsLNCTcC83Bhx5nEA7izI+5DCD7TvQMRkulL4uj yBU+0cHTk3ioe9hlqWXU1kowB2LBoh9I25pCjtpHXzsgkYv9jFAhQVlDZrtQGM3N BJ9/hOe9nyMUFlVF8JMYBHfi9PPlk51Nyj9S2QOH4hMd9BeQs2l8ebHwV+zBx7O8 MfYqHcEdFyatMLC+Znj/66qzGQJW9Aj5i3aqm60JilqPWLV8G7g1yjSG9Yq7ahq4 8Biu3cUpRS1G9EJKxlNqKT85gc69/nZzSl/ZYAd1D0LtmMSRbzm0vlPYZmrgnRPC MEbXwQJXg4ExLLHZECMOa+BIYZD6Ss4zvu73rxKtzLPL7y2FltsZXqprM/LCREgD N4lgb6yW8gOp1VO5CDj/zSVOaudGURQoe86eHTsIae7OtIGnCGvCfSwzz65EEdgo HR+2jnhfdDfoSljZLtOjia511thZBneBFPrSPcrewVKeKrzH2eHJuUuPXvL1Ju1C Fv3JgjaUU34IZr4n4nUjZ0jpZ2p8qT9d5YlEPb0c5Sm8O0+0H4s= =Wxg8 -----END PGP SIGNATURE----- --nextPart2101211.VLH7GnMWUR--