From kde-kimageshop Tue Feb 07 14:43:19 2012 From: Dmitry Kazakov Date: Tue, 07 Feb 2012 14:43:19 +0000 To: kde-kimageshop Subject: Re: Lossless image compression Message-Id: X-MARC-Message: https://marc.info/?l=kde-kimageshop&m=132862601411563 MIME-Version: 1 Content-Type: multipart/mixed; boundary="--===============3880085489882481440==" --===============3880085489882481440== Content-Type: multipart/alternative; boundary=14dae9399b5bd56a6904b860cca6 --14dae9399b5bd56a6904b860cca6 Content-Type: text/plain; charset=UTF-8 On Tue, Feb 7, 2012 at 5:01 PM, Boudewijn Rempt wrote: > On Tue, 7 Feb 2012, Dmitry Kazakov wrote: > > Compression >> For the actual compression of the tile data we use LZF algorithm. I did >> some research on the compression times of LZO and LZF [0], [1]. It >> turned out that LZF has better times. It takes twice faster to compress >> the image with LZF while keeping almost the same compression rate. >> One more thing I found then is that it is faster to "linearize" colors >> before compression and then compress the image than compress the >> image directly (RGBARGBA -> RRGGBBAA). It boosts both the speed and the >> compression rate. The time of the "linearization" itself is >> negligible in comparison with the boost it give, so now we do it this way. >> >> > Well, actually when Sven wanted to know why loading and saving files is so > slow, it turns out that the linearization is the biggest bottleneck we > currently have! But that might depend a lot on cache sizes and so on. In > general, we want to avoid missing the cache when compressing a tile. > Probably, I'm a bit outdated. I've just run the test on my new machine. The results are the following: Compression LZF w/o linearization: 247 memcpy's. Ratio: 0.938 Compression LZF with linearization: 213 memcpy's. Ratio: 0.769 Decompression LZF w/o linearization: 78 memcpy's. Decompression LZF with linearization: 112 memcpy's. It means that for compression the linearization still makes it faster, but for decompression it makes the things worse on newer cpu. Here are the callgrind graphs for compression and decompression: [0], [1] [0] - http://dimula73.narod.ru/callgrind.out.9540_compression_two_pass [1] - http://dimula73.narod.ru/callgrind.out.9513_decompression_two_pass -- Dmitry Kazakov --14dae9399b5bd56a6904b860cca6 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable

On Tue, Feb 7, 2012 at 5:01 PM, Boudewij= n Rempt <boud@vald= yas.org> wrote:
On Tue, 7 Feb 2012, Dmitry Kazakov wrote:

Compression
For the actual compression of the tile data we use LZF algorithm. I did som= e research on the compression times of LZO and LZF [0], [1]. It
turned out that LZF has better times. It takes twice faster to compress the= image with LZF while keeping almost the same compression rate.
One more thing I found then is that it is faster to "linearize" c= olors before compression and then compress the image than compress the
image directly (RGBARGBA -> RRGGBBAA). It boosts both the speed and the = compression rate. The time of the "linearization" itself is
negligible in comparison with the boost it give, so now we do it this way.<= br>

Well, actually when Sven wanted to know why loading and saving files is so = slow, it turns out that the linearization is the biggest bottleneck we curr= ently have! But that might depend a lot on cache sizes and so on. In
general, we want to avoid missing the cache when compressing a tile.

Probably, I'm a bit outdated.

I've just run the test on= my new machine.
The results are the following:

Compression LZF w/o=C2=A0 linearization: 247 memcpy's. Rat= io: 0.938
Compression LZF with linearization: = 213 memcpy's. Ratio: 0.769

Decompres= sion LZF w/o=C2=A0 linearization:=C2=A0 78 memcpy's.
Decompression LZF with li= nearization: 112 memcpy's.

It means that for compression the linearization still makes it faster, = but for decompression it makes the things worse on newer cpu. Here are the = callgrind graphs for compression and decompression: [0], [1]

[0] - <= a href=3D"http://dimula73.narod.ru/callgrind.out.9540_compression_two_pass"= >http://dimula73.narod.ru/callgrind.out.9540_compression_two_pass
[1] - http://dimula73.narod.ru/callgrind.out.9513_decompression_two_pas= s



--
Dmitry Kazakov
--14dae9399b5bd56a6904b860cca6-- --===============3880085489882481440== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ kimageshop mailing list kimageshop@kde.org https://mail.kde.org/mailman/listinfo/kimageshop --===============3880085489882481440==--