[prev in list] [next in list] [prev in thread] [next in thread] 

List:       openjdk-core-libs-dev
Subject:    Code Review Request for # 6975829 : Perf. of gzip in existing JDKs is too slower than in 1.3.1
From:       xueming.shen () oracle ! com (Xueming Shen)
Date:       2010-09-30 22:29:17
Message-ID: 4CA50F3D.8090509 () oracle ! com
[Download RAW message or body]

  Martin, Alan

Please help review the change for #6975829

http://cr.openjdk.java.net/~sherman/6975829/webrev/ 
<http://cr.openjdk.java.net/%7Esherman/6975829/webrev/>

This is again a problem triggered by the change we putback years ago for
6206933: GZipOutputStream/InputStream goes critical(calls 
JNI_Get*Critical) and causes slowness

in which we switch from using the JNI_Get*Critical to access the input bits
to using GetByteArrayregion(...) (which invloves array copy)

The test case attached in the bug report exaggeratedly use file size of the
.gz file as the buf size param for the GZIPInputStream() constructor.

         File f = new File(TESTFILE);
         DataInputStream dis = new DataInputStream(new 
GZIPInputStream(new FileInputStream(f), (int)f.length()))

and then start to read in the data in very small chunk via dis, which is 
a disaster.

While it is possible to workaround the issue easily by
a) to use a smaller buf size, such as the defualt 512
b) wrap the GZIPInputStream with a BufferedInputStream

It still appears the memory/buffer handling code in Inflater.c can be 
changed to improve
the performance in use cases similar to above mentioned scenario. The 
proposed change
is to copy no more than the size of the "user's buffer". It's an 
de-compression, so you can't
consume more bytes than you can write out in most cases. The attached 
test case shows
the performance of gzip-ing the un-compressed rt.jar by using different 
buffer size.

Again, we still need to address the "fundamental performance" issue 
caused by moving
away from JNI_Get*Critical for #6206933. But that will be a separate 
project.

-Sherman


-----------------------------------------------------------------------------
     private static int BUFSIZE = 1024 * 1024;

     public static void main(String[] args) throws Exception {

         File f = new File(args[0]);

         //gzip it into memory, yeah we have enough memoery:-)
         ByteArrayOutputStream baos = new 
ByteArrayOutputStream((int)f.length() / 2);
         GZIPOutputStream gout = new GZIPOutputStream(baos);
         FileInputStream in = new FileInputStream(args[0]);
         byte[] copyBuf = new byte[BUFSIZE];
         int n;
         while ((n = in.read(copyBuf)) != -1) {
             gout.write(copyBuf, 0, n);
         }
         in.close();
         gout.close();

         byte[] gsrc = baos.toByteArray();

         for (int i = 1; i <= 128; i <<= 1 ) {
             copyBuf = new byte[BUFSIZE / i];
             long time = System.currentTimeMillis();
             for (int j = 0; j < 20; j++) {
                 ByteArrayInputStream bais = new ByteArrayInputStream(gsrc);
                 GZIPInputStream gin = new GZIPInputStream(bais, BUFSIZE);
                 while ((gin.read(copyBuf)) != -1) {}
                 gin.close();
             }
             System.out.printf( "buf[1/%d]: %d ms%n", i, 
(System.currentTimeMillis() - time)/ 20);
         }
     }
------------------------------------------------------------------------

(1) latest jdk7
sherman at sherman-linux:/home/sherman/TL/dis$  
/net/koori/onestop/jdk/7/latest/binaries/linux-i586/bin/java Inf /tmp/rt.jar
buf[1/1]: 579 ms
buf[1/2]: 640 ms
buf[1/4]: 613 ms
buf[1/8]: 683 ms
buf[1/16]: 873 ms
buf[1/32]: 1247 ms
buf[1/64]: 1958 ms
buf[1/128]: 3379 ms

(2) with proposed fix
sherman at sherman-linux:/home/sherman/TL/dis$ java Inf 
/tmp/rt.jarbuf[1/1]: 514 ms
buf[1/2]: 546 ms
buf[1/4]: 460 ms
buf[1/8]: 413 ms
buf[1/16]: 426 ms
buf[1/32]: 448 ms
buf[1/64]: 436 ms
buf[1/128]: 448 ms


[prev in list] [next in list] [prev in thread] [next in thread]