[prev in list] [next in list] [prev in thread] [next in thread]
List: openjdk-serviceability-dev
Subject: JDK-8005604, instance size/metaspace data in HPROF
From: Vladimir Sitnikov <sitnikov.vladimir () gmail ! com>
Date: 2014-02-18 19:57:46
Message-ID: CAB=Je-HFxP63UH95aMGqJey7=DqGEva25tm2sQVKTHgs6yu-kA () mail ! gmail ! com
[Download RAW message or body]
Seems like HPROF does not include information on non-heap memory.
It would be nice to have accurate class size (e.g. with @Contended), space
wastage, permgen/metaspace (e.g. constant pool size or whatever is in
C-heap), code cache blobs in the heap dump.
Alexey Shipil£v raised similar question regarding instance size CR 8005604:
HPROF should report the actual instance size [1], [2], however it seems no
progress is made.
Creating brand-new format does not look like an easy solution, and it is
not clear how new features will be added later. It is not clear why new
format would be better in terms of efforts to update tooling.
What do you think if we keep HPROF format completely intact and dump
additional information via synthetic classes and objects?
For instance, to dump precise instance size of java class, we include
"vm.native.synthetic.ClassLayout { int instanceSize; int spaceLoss; }"
class in the generated HPROF file. For each java class we dump "instance"
of this ClassInfo.
This will reveal true instance size, space loss (e.g. alignment,
@contended, etc) to the heap analyzer.
The resulting HPROF will be fully compatible with existing analyzers, so no
significant harm is made.
To associate java.lang.Class and its vm.native metadata, a new synthetic
static reference from java.lang.Class to vm.native.synthetic.ClassInfo is
added.
Another approach is to reference from vm.native to java.lang.Class, however
it will be harder to analyze (more clicks, more complex queries)
Pros:
1) We can dump real instance size or any new information while keeping
HPROF format intact
2) Current tools will parse and display the dump just fine. With moderate
effort tools can be improved to use this "metadata" for calculations (and
proper accounting of "used heap size")
3) Query language of memory analyzers (e.g. map-reduce-javascript in
VirtualVM, OQL/SQL in Eclipse MAT, etc) can leverage this new data. The
approach of synthetic classes does not require to update analyzer for that.
4) More details (e.g. code cache blobs or whatever) can be added in the
similar way to the dump while still keeping forward and backward
compatibility in terms of file format
Cons:
1) I am not sure if iteration over permgen/metaspace/codecache is safe in
terms of crashes (or whatever). It is not good a good idea to crash when
writing HPROF dump (especially during OutOfMemoryError), however I believe
at least basic information (e.g. instance size) should be available in a
safe way.
2) Existing analyzers will charge "heap size" for synthetic
classes/objects. I do not think it is a big deal, however, if we dump
permgen/metaspace/codecache, then the amount of synthetic objects might be
significant enough to confuse non-expecting engineers
1: https://bugs.openjdk.java.net/browse/JDK-8005604
2:
http://mail.openjdk.java.net/pipermail/serviceability-dev/2012-December/007852.html
--
Regards,
Vladimir Sitnikov
[Attachment #3 (text/html)]
<div dir="ltr"><div>Seems like HPROF does not include information on non-heap \
memory.</div><div>It would be nice to have accurate class size (e.g. with \
@Contended), space wastage, permgen/metaspace (e.g. constant pool size or whatever is \
in C-heap), code cache blobs in the heap dump.</div> <div><br></div><div>Alexey \
Shipil£v raised similar question regarding instance size CR 8005604: HPROF should \
report the actual instance sizeš[1], [2], however it seems no progress is \
made.</div><div><br></div><div>Creating brand-new format does not look like an easy \
solution, and it is not clear how new features will be added later. It is not clear \
why new format would be better in terms of efforts to update tooling.</div> \
<div><br></div><div>What do you think if we keep HPROF format completely intact and \
dump additional information via synthetic classes and \
objects?</div><div><br></div><div>For instance, to dump precise instance size of java \
class, we include "vm.native.synthetic.ClassLayout { int instanceSize; int \
spaceLoss; }" class in the generatedšHPROFšfile. For each java class we dump \
"instance" of this ClassInfo.</div> <div>This will reveal true instance \
size, space loss (e.g. alignment, @contended, etc) to the heap \
analyzer.<br></div><div>The resulting HPROF will be fully compatible with existing \
analyzers, so no significant harm is made.</div> <div><br></div><div>To associate \
java.lang.Class and its vm.nativešmetadata, a new synthetic static reference from \
java.lang.Class to vm.native.synthetic.ClassInfo is added.</div><div>Another approach \
is to reference from vm.nativešto java.lang.Class, however it will be harder to \
analyze (more clicks, more complex queries)</div> \
<div><br></div><div>Pros:</div><div>1) We can dump real instance size or any new \
information while keeping HPROF format intact</div><div>2) Current tools will parse \
and display the dump just fine. With moderate effort tools can be improved to use \
this "metadata" for calculations (and proper accounting of "used heap \
size")</div> <div>3) Query language of memory analyzers (e.g. \
map-reduce-javascript in VirtualVM, OQL/SQL in Eclipse MAT, etc) can leverage this \
new data. The approach of synthetic classes does not require to update analyzer for \
that.</div> <div>4) More details (e.g. code cache blobs or whatever) can be added in \
the similar way to the dump while still keeping forward and backward compatibility in \
terms of file format</div><div><br></div><div>Cons:</div><div> 1) I am not sure if \
iteration over permgen/metaspace/codecache is safe in terms of crashes (or whatever). \
It is not good a good idea to crash when writingšHPROFšdump (especially during \
OutOfMemoryError), however I believe at least basic information (e.g. instance size) \
should bešavailable in a safe way.</div> <div>2) Existing analyzers will charge \
"heap size" for synthetic classes/objects. I do not think it is a big deal, \
however, if we dump permgen/metaspace/codecache, then the amount of synthetic objects \
might be significant enough to confuse non-expecting engineers</div> \
<div><br></div><div><br></div><div>1:š<a \
href="https://bugs.openjdk.java.net/browse/JDK-8005604">https://bugs.openjdk.java.net/browse/JDK-8005604</a></div><div></div><div>2: \
<a href="http://mail.openjdk.java.net/pipermail/serviceability-dev/2012-December/00785 \
2.html">http://mail.openjdk.java.net/pipermail/serviceability-dev/2012-December/007852.html</a></div>
<div><br></div><div>-- <br>Regards,<div>Vladimir Sitnikov</div>
</div></div>
[prev in list] [next in list] [prev in thread] [next in thread]
Configure |
About |
News |
Add a list |
Sponsored by KoreLogic