[prev in list] [next in list] [prev in thread] [next in thread]
List: kde-i18n-doc
Subject: Re: Review Request 7106: AppStream support for l10n-script (example)
From: "Albert Astals Cid" <aacid () kde ! org>
Date: 2014-04-23 18:30:58
Message-ID: 20140423183058.3535.4074 () probe ! kde ! org
[Download RAW message or body]
> On April 22, 2014, 1:15 p.m., Burkhard Lück wrote:
> > @ Matthias Klumpp:
> >
> > xmlstarlet and itstool are interesting tools, I was not aware of them
> >
> > Some questions:
> >
> > # We have only one global file as-metainfo.its for all *.appdata.xml / \
> > *.metainfo.xml files in the kde repos?
> > # Naming scheme appdata will be \
> > <replaceable>applicationname</replaceable>.appdata.xml ? Therefore we will have \
> > unique file names?
> > # Naming scheme metainfo will be <replaceable>foo</replaceable>.metainfo.xml ?
> > What is foo here?
> > Do we have unique names for *.metainfo.xml files?
> >
> > @ Albert:
> >
> > These scripts require packages named xmlstarlet and itstool on debian based \
> > systems and itstool >= 1.2.0 with the option -j ("join mode" for multilingual XML \
> > formats) Are these reqired packages available on the l10n server?
>
> Matthias Klumpp wrote:
> For 1): Yes, it is exactly the same scheme :-) See Question 4.6 on \
> http://techbase.kde.org/MetaInfo (it's still a discussion if this is useful or \
> confusing ^^) 2): Yes, filenames will be unique. The naming should follow the name \
> of the .desktop-file referenced in <id/> 3): Yes, you should have unique names, but \
> naming is arbitrary. The name just has to be unique in the software world. There is \
> some pressure on upstream developers from distributors, since the recommendation is \
> to drop components with the same IDs and consider that as bug.
> Itstool is available in Debian since wheezy (7.0), but it only wants Python and \
> libxml2, so it should be trivial to install on older Debian versions (you can use \
> the Jessie package). Xmlstarlet as been around for a very long time already and \
> should even be available in pre-Squeeze releases of Debian :-) Hope that helps!
>
>
> Burkhard Lück wrote:
> Apparently I can not update the diff, so my patch inline, based on Matthias' diff.
> Not tested so far, my itstool is < 1.2.0 without "join mode" for multilingual XML
>
> Index: update_translations
> ===================================================================
> --- update_translations (Revision 1385092)
> +++ update_translations (Arbeitskopie)
> @@ -204,6 +204,11 @@
> L10NDIR=$BASEDIR/$transmod \
> bash $BASEDIR/$transmod/scripts/extract-xml.sh)
>
> + ( SUBMODULE=$templatename \
> + ASMETAINFOITS=$BASEDIR/$transmod/scripts/as-metainfo.its \
> + L10NDIR=$BASEDIR/$transmod \
> + bash $BASEDIR/$transmod/scripts/extract-metainfo.sh)
> +
> ( XGETTEXT=`which xgettext` \
> MSGCAT=`which msgcat` \
> EXTRACTRC="perl $extractrc --ignore-no-input" \
> Index: as-metainfo.its
> ===================================================================
> --- as-metainfo.its (Revision 0)
> +++ as-metainfo.its (Revision 0)
> @@ -0,0 +1,8 @@
> +<its:rules
> + xmlns:its="http://www.w3.org/2005/11/its"
> + version="1.0">
> + <its:translateRule translate="no" selector="/component"/>
> + <its:translateRule translate="yes"
> + selector="/component/name | /component/summary |
> + /component/description"/>
> +</its:rules>
> Index: extract_metainfo.sh
> ===================================================================
> --- extract_metainfo.sh (Revision 0)
> +++ extract_metainfo.sh (Revision 0)
> @@ -0,0 +1,31 @@
> +#!/bin/bash
> +
> +podir=${podir:-$PWD/po}
> +metainfo_file_list=$(find . -name '*.appdata.xml' -or -name '*.metainfo.xml')
> +
> +for metainfo_file in $metainfo_file_list; do
> +
> + # first, strip translation from project metadata file
> + tmpxmlfile=$(mktemp)
> + xmlstarlet ed -d "//*[@xml:lang]" $metainfo_file > $tmpxmlfile
> +
> + # get rid of the dirname
> + dataname=$(basename $metainfo_file)
> +
> + # create pot file foo.[appdata|metadata].xml -> foo.[appdata|metadata].xml.pot
> + itstool -i $ASMETAINFOITS -o $podir/$dataname.pot $tmpfile
> +
> + # find po files in l10n module dir
> + for pofile in $(ls $L10NDIR/*/messages/$SUBMODULE/$dataname.po); do
> + # generate mo files:
> + $tmpmofile=$(mktemp)
> + msgfmt $pofile -o $tmpmofile
> +
> + # recreate file, using the untranslated temporary data and the translation
> + itstool -j $tmpxmlfile -o $metainfo_file $tmpmofile
> + rm $tmpmofile
> + done
> +
> + # cleanup
> + rm $tmpxmlfile
> +done
>
> Eigenschaftsänderungen: extract_metainfo.sh
> ___________________________________________________________________
> Hinzugefügt: svn:executable
> + *
>
Only the person that has created the review can update it
- Albert
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://svn.reviewboard.kde.org/r/7106/#review11088
-----------------------------------------------------------
On April 22, 2014, 1:16 p.m., Matthias Klumpp wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://svn.reviewboard.kde.org/r/7106/
> -----------------------------------------------------------
>
> (Updated April 22, 2014, 1:16 p.m.)
>
>
> Review request for Localization and Translation (l10n) and Albert Astals Cid.
>
>
> Repository: kde
>
>
> Description
> -------
>
> Hi!
> It's very difficult to extend the l10n-script if you don't know it's internal \
> workings, and if all comments are stripped out, so in order to get AppStream \
> support into the l10n-script, I created two small bash scripts to translate \
> AppStream upstream metadata:
> extract_metainfo.sh: This script generates .pot files from metadata
> XML files found in a project. Just cd to the directory you want the
> resulting pot files to be stored in, then call the script and point it
> at the project's source tree you want to be searched for the XML
> files.
>
> merge_metainfo.sh: The script merges translation back into the XML
> files. Simply cd to a directory which stores .mo files with the
> translated data and call the script with the project's source dir as
> first parameter.
>
> These scripts illustrate how translation of the XML can be done in
> KDE. I would kindly ask the l10n-script developers to take a look at
> it any maybe integrate it at the proper places, because they know the
> code way better than I do.
> The scripts can also be found at:
> http://people.freedesktop.org/~mak/appstream/kde-asmetadata-l10n.tar.gz
> (I also included a demo XML file there)
>
> If you have questions, please let me know!
> Cheers,
> Matthias
>
>
> Diffs
> -----
>
> trunk/l10n-kde4/scripts/extract_metainfo.sh PRE-CREATION
> trunk/l10n-kde4/scripts/its/as-metainfo.its PRE-CREATION
> trunk/l10n-kde4/scripts/merge_metainfo.sh PRE-CREATION
>
> Diff: https://svn.reviewboard.kde.org/r/7106/diff/
>
>
> Testing
> -------
>
> * Finds AppStream metadata in project source directories
> * .pot extraction works
> * Merging translation to untranslated files works
> * Merging new translation into already translated metadata files works (without \
> duplicating entries)
> * Resulting files are valid AppStream metadata files (compliance-check passed)
>
>
> Thanks,
>
> Matthias Klumpp
>
>
[Attachment #3 (text/html)]
<html>
<body>
<div style="font-family: Verdana, Arial, Helvetica, Sans-Serif;">
<table bgcolor="#f9f3c9" width="100%" cellpadding="8" style="border: 1px #c9c399 \
solid;"> <tr>
<td>
This is an automatically generated e-mail. To reply, visit:
<a href="https://svn.reviewboard.kde.org/r/7106/">https://svn.reviewboard.kde.org/r/7106/</a>
</td>
</tr>
</table>
<br />
<blockquote style="margin-left: 1em; border-left: 2px solid #d0d0d0; padding-left: \
10px;"> <p style="margin-top: 0;">On April 22nd, 2014, 1:15 p.m. UTC, <b>Burkhard \
Lück</b> wrote:</p> <blockquote style="margin-left: 1em; border-left: 2px solid \
#d0d0d0; padding-left: 10px;"> <pre style="white-space: pre-wrap; white-space: \
-moz-pre-wrap; white-space: -pre-wrap; white-space: -o-pre-wrap; word-wrap: \
break-word;">@ Matthias Klumpp:
xmlstarlet and itstool are interesting tools, I was not aware of them
Some questions:
# We have only one global file as-metainfo.its for all *.appdata.xml / *.metainfo.xml \
files in the kde repos?
# Naming scheme appdata will be \
<replaceable>applicationname</replaceable>.appdata.xml ? Therefore we \
will have unique file names?
# Naming scheme metainfo will be \
<replaceable>foo</replaceable>.metainfo.xml ? What is foo here?
Do we have unique names for *.metainfo.xml files?
@ Albert:
These scripts require packages named xmlstarlet and itstool on debian based systems
and itstool >= 1.2.0 with the option -j ("join mode" for multilingual XML formats)
Are these reqired packages available on the l10n server?</pre>
</blockquote>
<p>On April 22nd, 2014, 7:37 p.m. UTC, <b>Matthias Klumpp</b> wrote:</p>
<blockquote style="margin-left: 1em; border-left: 2px solid #d0d0d0; padding-left: \
10px;"> <pre style="white-space: pre-wrap; white-space: -moz-pre-wrap; white-space: \
-pre-wrap; white-space: -o-pre-wrap; word-wrap: break-word;">For 1): Yes, it is \
exactly the same scheme :-) See Question 4.6 on http://techbase.kde.org/MetaInfo \
(it's still a discussion if this is useful or confusing ^^) 2): Yes, filenames \
will be unique. The naming should follow the name of the .desktop-file referenced in \
<id/> 3): Yes, you should have unique names, but naming is arbitrary. The name \
just has to be unique in the software world. There is some pressure on upstream \
developers from distributors, since the recommendation is to drop components with the \
same IDs and consider that as bug.
Itstool is available in Debian since wheezy (7.0), but it only wants Python and \
libxml2, so it should be trivial to install on older Debian versions (you can use the \
Jessie package). Xmlstarlet as been around for a very long time already and should \
even be available in pre-Squeeze releases of Debian :-) Hope that helps!
</pre>
</blockquote>
<p>On April 23rd, 2014, 6:22 p.m. UTC, <b>Burkhard Lück</b> wrote:</p>
<blockquote style="margin-left: 1em; border-left: 2px solid #d0d0d0; padding-left: \
10px;"> <pre style="white-space: pre-wrap; white-space: -moz-pre-wrap; white-space: \
-pre-wrap; white-space: -o-pre-wrap; word-wrap: break-word;">Apparently I can not \
update the diff, so my patch inline, based on Matthias' diff. Not tested so far, \
my itstool is < 1.2.0 without "join mode" for multilingual XML
Index: update_translations
===================================================================
--- update_translations (Revision 1385092)
+++ update_translations (Arbeitskopie)
@@ -204,6 +204,11 @@
L10NDIR=$BASEDIR/$transmod \
bash $BASEDIR/$transmod/scripts/extract-xml.sh)
+ ( SUBMODULE=$templatename \
+ ASMETAINFOITS=$BASEDIR/$transmod/scripts/as-metainfo.its \
+ L10NDIR=$BASEDIR/$transmod \
+ bash $BASEDIR/$transmod/scripts/extract-metainfo.sh)
+
( XGETTEXT=`which xgettext` \
MSGCAT=`which msgcat` \
EXTRACTRC="perl $extractrc --ignore-no-input" \
Index: as-metainfo.its
===================================================================
--- as-metainfo.its (Revision 0)
+++ as-metainfo.its (Revision 0)
@@ -0,0 +1,8 @@
+<its:rules
+ xmlns:its="http://www.w3.org/2005/11/its"
+ version="1.0">
+ <its:translateRule translate="no" \
selector="/component"/> + <its:translateRule \
translate="yes" + selector="/component/name | /component/summary |
+ /component/description"/>
+</its:rules>
Index: extract_metainfo.sh
===================================================================
--- extract_metainfo.sh (Revision 0)
+++ extract_metainfo.sh (Revision 0)
@@ -0,0 +1,31 @@
+#!/bin/bash
+
+podir=${podir:-$PWD/po}
+metainfo_file_list=$(find . -name '*.appdata.xml' -or -name \
'*.metainfo.xml') +
+for metainfo_file in $metainfo_file_list; do
+
+ # first, strip translation from project metadata file
+ tmpxmlfile=$(mktemp)
+ xmlstarlet ed -d "//*[@xml:lang]" $metainfo_file > $tmpxmlfile
+
+ # get rid of the dirname
+ dataname=$(basename $metainfo_file)
+
+ # create pot file foo.[appdata|metadata].xml -> foo.[appdata|metadata].xml.pot
+ itstool -i $ASMETAINFOITS -o $podir/$dataname.pot $tmpfile
+
+ # find po files in l10n module dir
+ for pofile in $(ls $L10NDIR/*/messages/$SUBMODULE/$dataname.po); do
+ # generate mo files:
+ $tmpmofile=$(mktemp)
+ msgfmt $pofile -o $tmpmofile
+
+ # recreate file, using the untranslated temporary data and the translation
+ itstool -j $tmpxmlfile -o $metainfo_file $tmpmofile
+ rm $tmpmofile
+ done
+
+ # cleanup
+ rm $tmpxmlfile
+done
Eigenschaftsänderungen: extract_metainfo.sh
___________________________________________________________________
Hinzugefügt: svn:executable
+ *
</pre>
</blockquote>
</blockquote>
<pre style="white-space: pre-wrap; white-space: -moz-pre-wrap; white-space: \
-pre-wrap; white-space: -o-pre-wrap; word-wrap: break-word;">Only the person that has \
created the review can update it</pre> <br />
<p>- Albert</p>
<br />
<p>On April 22nd, 2014, 1:16 p.m. UTC, Matthias Klumpp wrote:</p>
<table bgcolor="#fefadf" width="100%" cellspacing="0" cellpadding="8" \
style="background-image: \
url('https://svn.reviewboard.kde.org/static/rb/images/review_request_box_top_bg.ab6f3b1072c9.png'); \
background-position: left top; background-repeat: repeat-x; border: 1px black \
solid;"> <tr>
<td>
<div>Review request for Localization and Translation (l10n) and Albert Astals \
Cid.</div> <div>By Matthias Klumpp.</div>
<p style="color: grey;"><i>Updated April 22, 2014, 1:16 p.m.</i></p>
<div style="margin-top: 1.5em;">
<b style="color: #575012; font-size: 10pt;">Repository: </b>
kde
</div>
<h1 style="color: #575012; font-size: 10pt; margin-top: 1.5em;">Description </h1>
<table width="100%" bgcolor="#ffffff" cellspacing="0" cellpadding="10" \
style="border: 1px solid #b8b5a0"> <tr>
<td>
<pre style="margin: 0; padding: 0; white-space: pre-wrap; white-space: \
-moz-pre-wrap; white-space: -pre-wrap; white-space: -o-pre-wrap; word-wrap: \
break-word;">Hi! It's very difficult to extend the l10n-script if you don't \
know it's internal workings, and if all comments are stripped out, so in order to \
get AppStream support into the l10n-script, I created two small bash scripts to \
translate AppStream upstream metadata:
extract_metainfo.sh: This script generates .pot files from metadata
XML files found in a project. Just cd to the directory you want the
resulting pot files to be stored in, then call the script and point it
at the project's source tree you want to be searched for the XML
files.
merge_metainfo.sh: The script merges translation back into the XML
files. Simply cd to a directory which stores .mo files with the
translated data and call the script with the project's source dir as
first parameter.
These scripts illustrate how translation of the XML can be done in
KDE. I would kindly ask the l10n-script developers to take a look at
it any maybe integrate it at the proper places, because they know the
code way better than I do.
The scripts can also be found at:
http://people.freedesktop.org/~mak/appstream/kde-asmetadata-l10n.tar.gz
(I also included a demo XML file there)
If you have questions, please let me know!
Cheers,
Matthias
</pre>
</td>
</tr>
</table>
<h1 style="color: #575012; font-size: 10pt; margin-top: 1.5em;">Testing </h1>
<table width="100%" bgcolor="#ffffff" cellspacing="0" cellpadding="10" style="border: \
1px solid #b8b5a0"> <tr>
<td>
<pre style="margin: 0; padding: 0; white-space: pre-wrap; white-space: \
-moz-pre-wrap; white-space: -pre-wrap; white-space: -o-pre-wrap; word-wrap: \
break-word;">* Finds AppStream metadata in project source directories
* .pot extraction works
* Merging translation to untranslated files works
* Merging new translation into already translated metadata files works (without \
duplicating entries)
* Resulting files are valid AppStream metadata files (compliance-check passed)
</pre>
</td>
</tr>
</table>
<h1 style="color: #575012; font-size: 10pt; margin-top: 1.5em;">Diffs</b> </h1>
<ul style="margin-left: 3em; padding-left: 0;">
<li>trunk/l10n-kde4/scripts/extract_metainfo.sh <span style="color: \
grey">(PRE-CREATION)</span></li>
<li>trunk/l10n-kde4/scripts/its/as-metainfo.its <span style="color: \
grey">(PRE-CREATION)</span></li>
<li>trunk/l10n-kde4/scripts/merge_metainfo.sh <span style="color: \
grey">(PRE-CREATION)</span></li>
</ul>
<p><a href="https://svn.reviewboard.kde.org/r/7106/diff/" style="margin-left: \
3em;">View Diff</a></p>
</td>
</tr>
</table>
</div>
</body>
</html>
[prev in list] [next in list] [prev in thread] [next in thread]
Configure |
About |
News |
Add a list |
Sponsored by KoreLogic