From kde-bugs-dist  Wed Jan 31 21:34:16 2001
From: Michael =?iso-8859-1?q?H=E4ckel?= <Michael () Haeckel ! Net>
Date: Wed, 31 Jan 2001 21:34:16 +0000
To: kde-bugs-dist
Subject: Bug#19440: KMail and Japanese.
X-MARC-Message: https://marc.info/?l=kde-bugs-dist&m=98097701219711
MIME-Version: 1
Content-Type: multipart/mixed; boundary="--------------Boundary-00=_4LQ16TS23Y6SD1HSH5SV"


--------------Boundary-00=_4LQ16TS23Y6SD1HSH5SV
Content-Type: text/plain;
  charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable
Subject: 

On Wednesday, 31. January 2001 05:13, toyohiro@ksmplus.com wrote:
>
> $cd SOURCES/kdelibs/kdecore
> $diff -u charsets.config.orig charsets.config
> --- charsets.config.orig        Tue Jan 30 11:20:22 2001
> +++ charsets.config     Tue Jan 30 11:22:39 2001
> @@ -68,6 +68,13 @@
>  x-cp-1257=3Dcp 1257
>  tis620=3Diso 8859-11
>  tis-620=3Diso 8859-11
> +jisx0201.1976-0=3Deucjp
> +jisx0208.1983-0=3Deucjp
> +jisx0208.1990-0=3Deucjp
> +jisx0208.1997-0=3Deucjp
> +jisx0212.1990-0=3Deucjp
> +jisx0213.2000-1=3Deucjp
> +jisx0213.2000-2=3Deucjp
>  windows-874=3Diso 8859-11
>  windows874=3Diso 8859-11
>  x-windows-874=3Diso 8859-11

Thanks for your detailed description. I think charsets in kdelibs is the =
job=20
of Lars Knoll. I posted it on kde-core-devel.
Unfortunately I just was not even with your changes able to get any Japan=
ese=20
characters displayed although I thought I have a font for it installed=20
(jis-fixed and jis-gothic). Could you tell me where I can get a Japanese =
font,=20
or are these the wrong ones?
Ususally I use a unicode font, but that has only squares where the Japane=
se=20
characters should be.

> diff -ur kmail.orig/kmmsgbase.cpp kmail/kmmsgbase.cpp
> --- kmail.orig/kmmsgbase.cpp    Wed Jan  3 00:30:16 2001
> +++ kmail/kmmsgbase.cpp Tue Jan 30 12:04:37 2001
> @@ -372,6 +372,9 @@
>         // decode base64 text
>         cstr =3D decodeBase64(str);
>        }
> +#if QT_VERSION < 224
> +      if(QString(charset).lower() =3D=3D "shift_jis" ||
> +                              QString(charset).lower() =3D=3D "shift-j=
is" )
> +          charset =3D"sjis"; // toyo
> +#endif
>        QTextCodec *codec =3D codecForName(charset);
>        if (!codec) codec =3D codecForName(KGlobal::locale()->charset())=
;
>        if (codec) str =3D codec->toUnicode(cstr);

I did these things a bit different, because I already wrote=20
KMMsgBase::codecForName for a similar problem. To change it there should =
fix=20
a few more problems.

> diff -ur kmail.orig/kmreaderwin.cpp kmail/kmreaderwin.cpp
> --- kmail.orig/kmreaderwin.cpp  Fri Jan 26 06:46:28 2001
> +++ kmail/kmreaderwin.cpp       Tue Jan 30 13:00:58 2001
> @@ -904,6 +907,12 @@
>      } // if (!pgpMessage) then the message only looked similar to a pg=
p
> message else htmlStr =3D mCodec->toUnicode(quotedHTML(aStr));
>    }
> +//  Mail header charset(iso-2022-jp) is using all most E-mail system i=
n
> Japan. +//  ISO-2022-JP code consists of ESC(0x1b) character and 7Bit
> character which +//  used from '!' character to  '~' character.
> +//  JIS7 is header charset of iso-2022-jp.  toyo
> +  else if( QString(mCodec->name()) =3D=3D "JIS7" )
> +    htmlStr +=3D quotedHTML(mCodec->toUnicode(aStr));
>    else htmlStr +=3D mCodec->toUnicode(quotedHTML(aStr));
>    mViewer->write(htmlStr);
>  }

Well, it's a hack, but it works. This ESC definitely makes also problems =
at a=20
few other places and I expected the charset problem mainly fixed.
Japanese really seems to need some special treatment.
Why can't you simply use utf-8 :-)

> (C) Please get the addressbook file and save to
>     $HOME/.kde/share/apps/addressbook.
>
>     addressbook file is in http://www.ksmplus.com/~toyohiro/kde2.1
>
>     Please execute kmail and push AddressBook button.
>     Fig-1  http://www.ksmplus.com/~toyohiro/kde2.1/addressbook.gif
>
>     Please Composer button -> click "To:" field .
>     and press Control-Key + Left-arrow-Key and display the contents
>     of addressbook.  The messages did not display in Japanese.
>     Fig-2  http://www.ksmplus.com/~toyohiro/kde2.1/composer1.gif
>
>     Please apply below patch and re-compile and execuete kmail again.
>     Please Composer button -> click "To:" field .
>     and press Control-Key + Left-arrow-Key and display the contents
>     of addressbook.  The messages displayed in Japanese.
>     Fig-3  http://www.ksmplus.com/~toyohiro/kde2.1/composer2.gif
>
> diff -ur kmail.orig/kmcomposewin.cpp kmail/kmcomposewin.cpp
> --- kmail.orig/kmcomposewin.cpp Sat Jan 27 04:31:57 2001
> +++ kmail/kmcomposewin.cpp      Tue Jan 30 12:14:33 2001
> @@ -2427,7 +2427,8 @@
>
>    n=3D0;
>    if (!KMAddrBookExternal::useKAB())
> -    for (QString a=3Dadb.first(); a; a=3Dadb.next())
> +    for (QString a=3DQString::fromLocal8Bit(adb.first())
> +                      ; a; a=3DQString::fromLocal8Bit(adb.next()))
>        {
>         if (QString(a).find(s,0,false) >=3D 0)
>           {

Thanks for this patch, but the internal addressbook is obsolote. KAB or=20
abbrowser are recommended. Maybe we definitely should remove it someday.

Some time ago I committed a patch for internal addressbook to support=20
non-latin characters. I didn't test it vary well myself before. I just tr=
ied=20
it again. Well, it seems that even with your patch it does not work=20
correctely. If I exit KMail and start it again the non-latin characters a=
re=20
converted to question marks.

Do you think, it also works, if I apply the attached patch (to KMail). Al=
so=20
headers (Subject and address fields) should work with iso-2022-jp, at lea=
st=20
if you don't make them too long.

Regards,
Michael H=E4ckel
--------------Boundary-00=_4LQ16TS23Y6SD1HSH5SV
Content-Type: text/plain;
  charset="iso-8859-1";
  name="japanese.diff"
Content-Transfer-Encoding: base64
Content-Disposition: attachment; filename="japanese.diff"

SW5kZXg6IGttbWVzc2FnZS5jcHAKPT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09
PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PQpSQ1MgZmlsZTogL2hvbWUva2RlL2tkZW5l
dHdvcmsva21haWwva21tZXNzYWdlLmNwcCx2CnJldHJpZXZpbmcgcmV2aXNpb24gMS4xNjkKZGlm
ZiAtdSAtMyAtcCAtcjEuMTY5IGttbWVzc2FnZS5jcHAKLS0tIGttbWVzc2FnZS5jcHAJMjAwMS8w
MS8yMyAwODoxMTozMAkxLjE2OQorKysga21tZXNzYWdlLmNwcAkyMDAxLzAxLzMxIDIxOjE1OjAz
CkBAIC0yNTUsNyArMjU1LDEwIEBAIHZvaWQgS01NZXNzYWdlOjpmcm9tU3RyaW5nKGNvbnN0IFFT
dHJpbmcKICAgcmVzdWx0UG9zID0gKGNoYXIqKXJlc3VsdC5kYXRhKCk7CiAgIGlmIChzdHJQb3Mp
IGZvciAoOyAoY2g9KnN0clBvcykhPSdcMCc7IHN0clBvcysrKQogICB7Ci0gICAgaWYgKChjaD49
JyAnIHx8IGNoPT0nXHQnIHx8IGNoPT0nXG4nIHx8IGNoPD0nXDAnKQorLy8gIE1haWwgaGVhZGVy
IGNoYXJzZXQoaXNvLTIwMjItanApIGlzIHVzaW5nIGFsbCBtb3N0IEUtbWFpbCBzeXN0ZW0gaW4g
SmFwYW4uCisvLyAgSVNPLTIwMjItSlAgY29kZSBjb25zaXN0cyBvZiBFU0MoMHgxYikgY2hhcmFj
dGVyIGFuZCA3Qml0IGNoYXJhY3RlciB3aGljaAorLy8gIHVzZWQgZnJvbSAnIScgY2hhcmFjdGVy
IHRvICAnficgY2hhcmFjdGVyLiAgdG95bworICAgIGlmICgoY2g+PScgJyB8fCBjaD09J1x0JyB8
fCBjaD09J1xuJyB8fCBjaDw9J1wwJyB8fCBjaCA9PSAweDFiKQogICAgICAgICYmICEoY2g9PSc+
JyAmJiBhU3RyLm1pZChzdHJQb3MtYVN0ci5kYXRhKCktMSw2KT09IlxuPkZyb20iKSkKICAgICAg
ICpyZXN1bHRQb3MrKyA9IGNoOwogICB9CkluZGV4OiBrbW1zZ2Jhc2UuY3BwCj09PT09PT09PT09
PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT0K
UkNTIGZpbGU6IC9ob21lL2tkZS9rZGVuZXR3b3JrL2ttYWlsL2ttbXNnYmFzZS5jcHAsdgpyZXRy
aWV2aW5nIHJldmlzaW9uIDEuNjQKZGlmZiAtdSAtMyAtcCAtcjEuNjQga21tc2diYXNlLmNwcAot
LS0ga21tc2diYXNlLmNwcAkyMDAxLzAxLzAyIDE1OjMwOjE2CTEuNjQKKysrIGttbXNnYmFzZS5j
cHAJMjAwMS8wMS8zMSAyMToxNTowNApAQCAtMjg0LDYgKzI4NCw4IEBAIFFTdHJpbmcgS01Nc2dC
YXNlOjpza2lwS2V5d29yZChjb25zdCBRU3QKIFFUZXh0Q29kZWMqIEtNTXNnQmFzZTo6Y29kZWNG
b3JOYW1lKGNvbnN0IFFTdHJpbmcmIF9zdHIpCiB7CiAgIGlmIChfc3RyLmlzRW1wdHkoKSkgcmV0
dXJuIE5VTEw7CisgIGlmIChfc3RyLmxvd2VyKCkgPT0gInNoaWZ0X2ppcyIgfHwgX3N0ci5sb3dl
cigpID09ICJzaGlmdC1qaXMiKQorICAgIHJldHVybiBRVGV4dENvZGVjOjpjb2RlY0Zvck5hbWUo
InNqaXMiKTsKICAgcmV0dXJuIFFUZXh0Q29kZWM6OmNvZGVjRm9yTmFtZShfc3RyLmxvd2VyKCku
cmVwbGFjZSgKICAgICBRUmVnRXhwKCJ3aW5kb3dzIiksICJjcCIpICk7CiB9CkBAIC00MDIsNyAr
NDA0LDcgQEAgY29uc3QgUVN0cmluZyBLTU1zZ0Jhc2U6OmRlY29kZVJGQzIwNDdTdAogCiAKIC8v
LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t
LS0tLS0tLS0tLS0tLS0tLS0tLS0KLWNvbnN0IGNoYXIgZXNwZWNpYWxzWzE3XSA9ICIoKTw+QCw7
OlwiL1tdPy49ICI7Citjb25zdCBjaGFyIGVzcGVjaWFsc1sxOF0gPSAiKCk8PkAsOzpcIi9bXT8u
PSBcMDMzIjsKIAogY29uc3QgUVN0cmluZyBLTU1zZ0Jhc2U6OmVuY29kZVJGQzIwNDdTdHJpbmco
Y29uc3QgUVN0cmluZyYgX3N0ciwKICAgY29uc3QgUVN0cmluZyYgY2hhcnNldCkKQEAgLTQyOCw3
ICs0MzAsNyBAQCBjb25zdCBRU3RyaW5nIEtNTXNnQmFzZTo6ZW5jb2RlUkZDMjA0N1N0CiAgICAg
d2hpbGUgKGNyIDwgbGF0aW5MZW4pCiAgICAgewogICAgICAgaWYgKGxhdGluW2NyXSA9PSAzMikg
c3RhcnQgPSBjciArIDE7Ci0gICAgICBpZiAobGF0aW5bY3JdIDwgMCkgYnJlYWs7CisgICAgICBp
ZiAobGF0aW5bY3JdIDwgMzIpIGJyZWFrOwogICAgICAgY3IrKzsKICAgICB9CiAgICAgaWYgKGNy
IDwgbGF0aW5MZW4pCkBAIC00MzgsNyArNDQwLDcgQEAgY29uc3QgUVN0cmluZyBLTU1zZ0Jhc2U6
OmVuY29kZVJGQzIwNDdTdAogICAgICAgd2hpbGUgKGNyIDwgbGF0aW5MZW4pCiAgICAgICB7CiAg
ICAgICAgIC8qIFRoZSBlbmNvZGVkIHdvcmQgbXVzdCBiZSBsaW1pdGVkIHRvIDc1IGNoYXJhY3Rl
ciAqLwotICAgICAgICBmb3IgKGkgPSAwOyBpIDwgMTY7IGkrKykgaWYgKGxhdGluW2NyXSA9PSBl
c3BlY2lhbHNbaV0pIG51bVF1b3RlcysrOworICAgICAgICBmb3IgKGkgPSAwOyBpIDwgMTc7IGkr
KykgaWYgKGxhdGluW2NyXSA9PSBlc3BlY2lhbHNbaV0pIG51bVF1b3RlcysrOwogICAgICAgICBp
ZiAobGF0aW5bY3JdIDwgMCkgbnVtUXVvdGVzKys7CiAgICAgICAgIC8qIFN0b3AgYWZ0ZXIgNTgg
PSA3NSAtIDE3IGNoYXJhY3RlcnMgb3IgYXQgIjx1c2VyQGhvc3QuLi4iICovCiAgICAgICAgIGlm
IChjciAtIHN0YXJ0ICsgMiAqIG51bVF1b3RlcyA+PSA1OCB8fCBsYXRpbltjcl0gPT0gNjApIGJy
ZWFrOwpAQCAtNDU2LDcgKzQ1OCw3IEBAIGNvbnN0IFFTdHJpbmcgS01Nc2dCYXNlOjplbmNvZGVS
RkMyMDQ3U3QKICAgICAgIHdoaWxlIChwb3MgPCBzdG9wKQogICAgICAgewogICAgICAgICBudW1R
dW90ZXMgPSAwOwotICAgICAgICBmb3IgKGkgPSAwOyBpIDwgMTY7IGkrKykgaWYgKGxhdGluW3Bv
c10gPT0gZXNwZWNpYWxzW2ldKSBudW1RdW90ZXMgPSAxOworICAgICAgICBmb3IgKGkgPSAwOyBp
IDwgMTc7IGkrKykgaWYgKGxhdGluW3Bvc10gPT0gZXNwZWNpYWxzW2ldKSBudW1RdW90ZXMgPSAx
OwogICAgICAgICBpZiAobGF0aW5bcG9zXSA8IDApIG51bVF1b3RlcyA9IDE7CiAgICAgICAgIGlm
IChudW1RdW90ZXMpCiAgICAgICAgIHsKQEAgLTUwMSw3ICs1MDMsNyBAQCBjb25zdCBRU3RyaW5n
IEtNTXNnQmFzZTo6ZW5jb2RlUkZDMjIzMVN0CiAgIGJvb2wgcXVvdGU7CiAgIHdoaWxlICgqbCkK
ICAgewotICAgIGlmICgqbCA8IDApIGJyZWFrOworICAgIGlmICgqbCA8IDMyKSBicmVhazsKICAg
ICBsKys7CiAgIH0KICAgaWYgKCEqbCkgcmV0dXJuIGxhdGluOwpAQCAtNTExLDcgKzUxMyw3IEBA
IGNvbnN0IFFTdHJpbmcgS01Nc2dCYXNlOjplbmNvZGVSRkMyMjMxU3QKICAgd2hpbGUgKCpsKQog
ICB7CiAgICAgcXVvdGUgPSAqbCA8IDA7Ci0gICAgZm9yIChpID0gMDsgaSA8IDE2OyBpKyspIGlm
ICgqbCA9PSBlc3BlY2lhbHNbaV0pIHF1b3RlID0gdHJ1ZTsKKyAgICBmb3IgKGkgPSAwOyBpIDwg
MTc7IGkrKykgaWYgKCpsID09IGVzcGVjaWFsc1tpXSkgcXVvdGUgPSB0cnVlOwogICAgIGlmIChx
dW90ZSkKICAgICB7CiAgICAgICByZXN1bHQgKz0gIiUiOwpJbmRleDoga21yZWFkZXJ3aW4uY3Bw
Cj09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09
PT09PT09PT09PT0KUkNTIGZpbGU6IC9ob21lL2tkZS9rZGVuZXR3b3JrL2ttYWlsL2ttcmVhZGVy
d2luLmNwcCx2CnJldHJpZXZpbmcgcmV2aXNpb24gMS4yMzYKZGlmZiAtdSAtMyAtcCAtcjEuMjM2
IGttcmVhZGVyd2luLmNwcAotLS0ga21yZWFkZXJ3aW4uY3BwCTIwMDEvMDEvMjUgMjE6NDY6MjgJ
MS4yMzYKKysrIGttcmVhZGVyd2luLmNwcAkyMDAxLzAxLzMxIDIxOjE1OjA4CkBAIC05MDQsNiAr
OTA0LDEyIEBAIHZvaWQgS01SZWFkZXJXaW46OndyaXRlQm9keVN0cihjb25zdCBRQ1MKICAgICB9
IC8vIGlmICghcGdwTWVzc2FnZSkgdGhlbiB0aGUgbWVzc2FnZSBvbmx5IGxvb2tlZCBzaW1pbGFy
IHRvIGEgcGdwIG1lc3NhZ2UKICAgICBlbHNlIGh0bWxTdHIgPSBtQ29kZWMtPnRvVW5pY29kZShx
dW90ZWRIVE1MKGFTdHIpKTsKICAgfQorLy8gIE1haWwgaGVhZGVyIGNoYXJzZXQoaXNvLTIwMjIt
anApIGlzIHVzaW5nIGFsbCBtb3N0IEUtbWFpbCBzeXN0ZW0gaW4gSmFwYW4uCisvLyAgSVNPLTIw
MjItSlAgY29kZSBjb25zaXN0cyBvZiBFU0MoMHgxYikgY2hhcmFjdGVyIGFuZCA3Qml0IGNoYXJh
Y3RlciB3aGljaAorLy8gIHVzZWQgZnJvbSAnIScgY2hhcmFjdGVyIHRvICAnficgY2hhcmFjdGVy
LgorLy8gIEpJUzcgaXMgaGVhZGVyIGNoYXJzZXQgb2YgaXNvLTIwMjItanAuICB0b3lvCisgIGVs
c2UgaWYoIFFTdHJpbmcobUNvZGVjLT5uYW1lKCkpID09ICJKSVM3IiApCisgICAgaHRtbFN0ciAr
PSBxdW90ZWRIVE1MKG1Db2RlYy0+dG9Vbmljb2RlKGFTdHIpKTsKICAgZWxzZSBodG1sU3RyICs9
IG1Db2RlYy0+dG9Vbmljb2RlKHF1b3RlZEhUTUwoYVN0cikpOwogICBtVmlld2VyLT53cml0ZSho
dG1sU3RyKTsKIH0K

--------------Boundary-00=_4LQ16TS23Y6SD1HSH5SV--