[prev in list] [next in list] [prev in thread] [next in thread]
List: kde-kimageshop
Subject: Re: [Kde-imaging] Lossless image compression
From: Dmitry Kazakov <dimula73 () gmail ! com>
Date: 2012-02-09 19:50:11
Message-ID: CAEkBSfXCkBEk1+MiwqZnVxOCO4HHTtHJme6D2EGuOxvP6cgXEQ () mail ! gmail ! com
[Download RAW message or body]
[Attachment #2 (multipart/alternative)]
> > tile and the size of a pixel. The algorithm is split into two
> > classes: KisTileCompressor2 [2] and KisLzfCompression [3]. The
> > benchmarks are placed in [4].
>
> Link [3] and [4] are identical.
>
Sorry, new ones:
KisLzfCompression:
http://quickgit.kde.org/index.php?p=calligra.git&a=blob&h=13910830b29418a814 \
07078619a0c47e8d794d63&hb=194ee7c39975e0be1ce37bb9394456290b326019&f=krita%2Fimage%2Ftiles3%2Fswap%2Fkis_lzf_compression.cpp
KisTileCompressor2:
http://quickgit.kde.org/index.php?p=calligra.git&a=blob&h=c46843999c159f8855 \
3e1846e7d4681e375a7e3c&hb=194ee7c39975e0be1ce37bb9394456290b326019&f=krita%2Fimage%2Ftiles3%2Fswap%2Fkis_tile_compressor_2.cpp
Do you still have that 640x441 RGBA image? Which memcpy
> implementation? Also, why is memcpy of a single tile three times as
> fast as memcpy of the complete image? Did you iterate multiple times
> and measure cache performance? :)
>
I measured memcpy of the whole operation. So the less the image the less
the time. I didn't normalized that on the amount of data. And I used
QBENCHMARK which performes many cycles, but it doesn't flush the cache on
every cycle (I guess).
I've just prepared a standalone version of the test:
http://dimula73.narod.ru/compression_lzf_test.tar.gz
You can switch between two source files by changing TEST_FILE definition at
the top of kis_compression_tests.cpp
Probably, there are some flaws in the methodology of testing, if you find
one don't hesitate to point it up ;)
From the compression rate, it looks like the compression is actually
> only done for the alpha channel, whereas the RGB data is nearly
> uncompressed.
Hm... I'm afraid you are right =( I never thought why it was 75% only.
> I am not sure how often the data actually gets written
> to disk.
> If it gets written to the disk, having a better compression rate will
> result in the speedup, not a faster compression method.
I cannot get the exact timings, because actual swapping is done by the OS.
I can either measure how often the data is written/read to/from
memory-mapped file (this is a kind of "virtual" swapping) or measure how
often 16MiB memory window moves (which flushes the file). But the latter
parameter is quite fuzzy.
> If you are
> mostly doing in-memory operations, having the fastest method available
> will be beneficial, but with the marginal improvement you get, I would
> not bother compressing the RGB data at all, only the alpha.
>
Well, yes. Though in this case we won't be able to compress the tiles which
are fully filled with one color.
> IZ compression is symmetrical, i.e. needs the same time for
> compression and decompression. From looking at your numbers,
> compression speed for IZ should be similar to LZF, but decompression
> will be much slower. I can try to extract the LZF code and make a
> comparison on my machine, if you have representative test data (Krita
> isn't really for photos, which I used for all my benchmarks).
>
I've written the link to a standalone test above. You need to inherit your
algorithm from KisAbstractCompression and add it to KisCompressionTests.
You can check it with both source images.
--
Dmitry Kazakov
[Attachment #5 (text/html)]
<br><div class="gmail_quote"><blockquote class="gmail_quote" \
style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div \
class="im"> > tile and the size of a pixel. The algorithm is split into \
two<br> > classes: KisTileCompressor2 [2] and KisLzfCompression [3]. \
The<br> > benchmarks are placed in [4].<br>
<br>
</div>Link [3] and [4] are identical.<br></blockquote><div \
class="im"><br>Sorry, new ones:<br><br>KisLzfCompression:<br><a \
href="http://quickgit.kde.org/index.php?p=calligra.git&a=blob&h=1391 \
0830b29418a81407078619a0c47e8d794d63&hb=194ee7c39975e0be1ce37bb939445629 \
0b326019&f=krita%2Fimage%2Ftiles3%2Fswap%2Fkis_lzf_compression.cpp">http \
://quickgit.kde.org/index.php?p=calligra.git&a=blob&h=13910830b29418 \
a81407078619a0c47e8d794d63&hb=194ee7c39975e0be1ce37bb9394456290b326019&f=krita%2Fimage%2Ftiles3%2Fswap%2Fkis_lzf_compression.cpp</a><br>
<br>KisTileCompressor2:<br><a \
href="http://quickgit.kde.org/index.php?p=calligra.git&a=blob&h=c468 \
43999c159f88553e1846e7d4681e375a7e3c&hb=194ee7c39975e0be1ce37bb939445629 \
0b326019&f=krita%2Fimage%2Ftiles3%2Fswap%2Fkis_tile_compressor_2.cpp">ht \
tp://quickgit.kde.org/index.php?p=calligra.git&a=blob&h=c46843999c15 \
9f88553e1846e7d4681e375a7e3c&hb=194ee7c39975e0be1ce37bb9394456290b326019&f=krita%2Fimage%2Ftiles3%2Fswap%2Fkis_tile_compressor_2.cpp</a><br>
<br>
<br>
</div><blockquote class="gmail_quote" style="margin:0pt 0pt 0pt \
0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">Do you still \
have that 640x441 RGBA image? Which memcpy<br> implementation? Also, why is \
memcpy of a single tile three times as<br> fast as memcpy of the complete \
image? Did you iterate multiple times<br> and measure cache performance? \
:)<br></blockquote><div><br>I measured memcpy of the whole operation. So \
the less the image the less the time. I didn't normalized that on the \
amount of data. And I used QBENCHMARK which performes many cycles, but it \
doesn't flush the cache on every cycle (I guess).<br> <br>I've just \
prepared a standalone version of the test: <a \
href="http://dimula73.narod.ru/compression_lzf_test.tar.gz">http://dimula73.narod.ru/compression_lzf_test.tar.gz</a><br><br>You \
can switch between two source files by changing TEST_FILE definition at the \
top of kis_compression_tests.cpp<br> <br>Probably, there are some flaws in \
the methodology of testing, if you find one don't hesitate to point it \
up ;)<br><br><br></div><blockquote class="gmail_quote" style="margin:0pt \
0pt 0pt 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
From the compression rate, it looks like the compression is actually<br>
only done for the alpha channel, whereas the RGB data is nearly<br>
uncompressed. </blockquote><div><br>Hm... I'm afraid you are right =( I \
never thought why it was 75% only.<br><br> </div><blockquote \
class="gmail_quote" style="margin:0pt 0pt 0pt 0.8ex;border-left:1px solid \
rgb(204,204,204);padding-left:1ex"> I am not sure how often the data \
actually gets written<br> to disk.<br>
If it gets written to the disk, having a better compression rate will<br>
result in the speedup, not a faster compression method. \
</blockquote><div><br>I cannot get the exact timings, because actual \
swapping is done by the OS. I can either measure how often the data is \
written/read to/from memory-mapped file (this is a kind of \
"virtual" swapping) or measure how often 16MiB memory window \
moves (which flushes the file). But the latter parameter is quite \
fuzzy.<br> <br> </div><blockquote class="gmail_quote" style="margin:0pt \
0pt 0pt 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">If \
you are<br> mostly doing in-memory operations, having the fastest method \
available<br> will be beneficial, but with the marginal improvement you \
get, I would<br> not bother compressing the RGB data at all, only the \
alpha.<br></blockquote><div><br>Well, yes. Though in this case we won't \
be able to compress the tiles which are fully filled with one color.<br> \
</div><blockquote class="gmail_quote" style="margin:0pt 0pt 0pt \
0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
IZ compression is symmetrical, i.e. needs the same time for<br>
compression and decompression. From looking at your numbers,<br>
compression speed for IZ should be similar to LZF, but decompression<br>
will be much slower. I can try to extract the LZF code and make a<br>
comparison on my machine, if you have representative test data (Krita<br>
isn't really for photos, which I used for all my benchmarks).<br>
</blockquote></div><br>I've written the link to a standalone test \
above. You need to inherit your algorithm from KisAbstractCompression and \
add it to KisCompressionTests. You can check it with both source images.<br \
clear="all"> <br>-- <br>Dmitry Kazakov<br>
_______________________________________________
kimageshop mailing list
kimageshop@kde.org
https://mail.kde.org/mailman/listinfo/kimageshop
[prev in list] [next in list] [prev in thread] [next in thread]
Configure |
About |
News |
Add a list |
Sponsored by KoreLogic