[prev in list] [next in list] [prev in thread] [next in thread]
List: openjdk-serviceability-dev
Subject: Re: hprof format question
From: Kirk Pepperdine <kirk.pepperdine () gmail ! com>
Date: 2018-10-31 18:33:37
Message-ID: 405F2C86-3494-494D-91EA-B96579557732 () gmail ! com
[Download RAW message or body]
Hi Simon,
I've also started a small project to try and solve the we need to look at very large \
heap problem. My solution is to load the data into Neo4J. You can find the project on \
my GitHub account.
So, I believe I've taken the same tactic in just abandoning the segment for the \
moment. It would be useful to sort that out but I listed it as a future…
Kind regards,
Kirk
L
> On Oct 31, 2018, at 4:07 AM, Simon Roberts <simon@dancingcloudservices.com> wrote:
>
> Hi all, I'm hoping this is the correct list for a question on the hprof file format \
> (1.0.2)?
> I found this information: \
> http://hg.openjdk.java.net/jdk6/jdk6/jdk/raw-file/tip/src/share/demo/jvmti/hprof/manual.html \
> <http://hg.openjdk.java.net/jdk6/jdk6/jdk/raw-file/tip/src/share/demo/jvmti/hprof/manual.html>
>
> and have been working on a small project to read these files. (Yes, I know that \
> NetBeans/VisualVM and Eclipse both have such libraries, and a number of other tools \
> have been derived from those, but so far as I can tell, they all are fundamentally \
> built on the notion of fully decoding everything, and creating memory \
> representations of the entire heap. I want to pull out only certain pieces of \
> information--specifically object counts by class--from a large, ~20Gb, dump file, \
> and those tools just give up the ghost on my systems.)
> Anyway, my code reads the file pretty well so far, except that the file I want to \
> analyze seems to contradict the specifications of the document mentioned above. \
> Specifically, after processing about five HEAP_DUMP_SEGMENTS with around 1.5 \
> million sub-records in each, I come across some ROOT_JNI_LOCAL records. The first \
> 15 follow the format specified in the above document (one 8 byte "ID" and two four \
> byte values.) But the 16th omits the two four byte records (well, it might simply \
> have more, but visual analysis shows that after the 8 byte ID, I have a new block \
> tag, and a believable structure. I've actually noticed that several of the record \
> types defined in this "group" seem to diverge from the paper I mentioned.
> My solution is that if my parser trips, it abandons that HEAP_DUMP_SEGMENT from \
> that point forward. It doesn't seem to matter much, since I was looking for object \
> data, and it appears that all of that has already been handled. However, clearly \
> this is not ideal!
> Is there any more detailed, newer, better, information? Or anything else I should \
> know to pursue this tool (or indeed a simple object frequency by classname result) \
> from an hprof 1.0.2 format file?
> (And yes, I'm pursuing a putative memory leak :)
>
> Thanks for any input (including "dude, this is the wrong list!")
> Cheers,
> Simon
>
>
>
> --
> Simon Roberts
> (303) 249 3613
>
>
>
> --
> Simon Roberts
> (303) 249 3613
>
[Attachment #3 (unknown)]
<html><head><meta http-equiv="Content-Type" content="text/html \
charset=utf-8"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; \
-webkit-line-break: after-white-space;" class="">Hi Simon,<div class=""><br \
class=""></div><div class="">I've also started a small project to try and solve the \
we need to look at very large heap problem. My solution is to load the data into \
Neo4J. You can find the project on my GitHub account.</div><div class=""><br \
class=""></div><div class="">So, I believe I've taken the same tactic in just \
abandoning the segment for the moment. It would be useful to sort that out but I \
listed it as a future…</div><div class=""><br class=""></div><div class="">Kind \
regards,</div><div class="">Kirk</div><div class="">L<br class=""><div><blockquote \
type="cite" class=""><div class="">On Oct 31, 2018, at 4:07 AM, Simon Roberts <<a \
href="mailto:simon@dancingcloudservices.com" \
class="">simon@dancingcloudservices.com</a>> wrote:</div><br \
class="Apple-interchange-newline"><div class=""><div dir="ltr" class=""><div \
class="gmail_quote"><div dir="ltr" class=""><div dir="ltr" class="">Hi all, I'm \
hoping this is the correct list for a question on the hprof file format (1.0.2)?<div \
class=""><br class=""></div><div class="">I found this information: <a \
href="http://hg.openjdk.java.net/jdk6/jdk6/jdk/raw-file/tip/src/share/demo/jvmti/hprof/manual.html" \
target="_blank" class="">http://hg.openjdk.java.net/jdk6/jdk6/jdk/raw-file/tip/src/share/demo/jvmti/hprof/manual.html</a><br \
clear="all" class=""><div class=""><br class=""></div><div class="">and have been \
working on a small project to read these files. (Yes, I know that NetBeans/VisualVM \
and Eclipse both have such libraries, and a number of other tools have been derived \
from those, but so far as I can tell, they all are fundamentally built on the notion \
of fully decoding everything, and creating memory representations of the entire heap. \
I want to pull out only certain pieces of information--specifically object counts by \
class--from a large, ~20Gb, dump file, and those tools just give up the ghost on my \
systems.)</div><div class=""><br class=""></div><div class="">Anyway, my code reads \
the file pretty well so far, except that the file I want to analyze seems to \
contradict the specifications of the document mentioned above. Specifically, after \
processing about five HEAP_DUMP_SEGMENTS with around 1.5 million sub-records in each, \
I come across some ROOT_JNI_LOCAL records. The first 15 follow the format specified \
in the above document (one 8 byte "ID" and two four byte values.) But the 16th omits \
the two four byte records (well, it might simply have more, but visual analysis shows \
that after the 8 byte ID, I have a new block tag, and a believable structure. I've \
actually noticed that several of the record types defined in this "group" seem to \
diverge from the paper I mentioned.</div><div class=""><br class=""></div><div \
class="">My solution is that if my parser trips, it abandons that HEAP_DUMP_SEGMENT \
from that point forward. It doesn't seem to matter much, since I was looking for \
object data, and it appears that all of that has already been handled. However, \
clearly this is not ideal!</div><div class=""><br class=""></div><div class="">Is \
there any more detailed, newer, better, information? Or anything else I should know \
to pursue this tool (or indeed a simple object frequency by classname result) from an \
hprof 1.0.2 format file?</div><div class=""><br class=""></div><div class="">(And \
yes, I'm pursuing a putative memory leak :)</div><div class=""><br \
class=""></div><div class="">Thanks for any input (including "dude, this is the wrong \
list!")</div><div class="">Cheers,</div><div class="">Simon</div><div class=""><br \
class=""></div><div class=""><br class=""></div><div class=""><br class=""></div>-- \
<br class=""><div dir="ltr" class="m_-7838466769366283700gmail_signature">Simon \
Roberts<div class="">(303) 249 3613</div><div class=""><br \
class=""></div></div></div></div></div> </div><br clear="all" class=""><div \
class=""><br class=""></div>-- <br class=""><div dir="ltr" class="gmail_signature" \
data-smartmail="gmail_signature">Simon Roberts<div class="">(303) 249 3613</div><div \
class=""><br class=""></div></div></div> </div></blockquote></div><br \
class=""></div></body></html>
[prev in list] [next in list] [prev in thread] [next in thread]
Configure |
About |
News |
Add a list |
Sponsored by KoreLogic