[prev in list] [next in list] [prev in thread] [next in thread]
List: openjdk-serviceability-dev
Subject: Re: Low-Overhead Heap Profiling
From: JC Beyler <jcbeyler () google ! com>
Date: 2017-04-21 21:34:23
Message-ID: CAF9BGBz81YYVUsMLw9sOMR9fH7JQmyb8d_OZ9aJ47EOAqwWhEw () mail ! gmail ! com
[Download RAW message or body]
Hi all,
I've added size information to the allocation sampling system. This allows
the callback to remember the size of each sampled allocation.
http://cr.openjdk.java.net/~rasbold/8171119/webrev.01/
The new webrev.01 also adds the actual heap monitoring sampling system in
files:
http://cr.openjdk.java.net/~rasbold/8171119/webrev.01/src/share/vm/runtime/heapMonitoring.cpp.patch
and
http://cr.openjdk.java.net/~rasbold/8171119/webrev.01/src/share/vm/runtime/heapMonitoring.hpp.patch
My next step is to add the GC part to the webrev, which will allow users to
determine what objects are live and what are garbage.
Thanks for your attention and let me know if there are any questions!
Have a wonderful Friday!
Jc
On Mon, Apr 17, 2017 at 12:37 PM, JC Beyler <jcbeyler@google.com> wrote:
> Hi all,
>
> I worked on getting a few numbers for overhead and accuracy for my
> feature. I'm unsure if here is the right place to provide the full data, so
> I am just summarizing here for now.
>
> - Overhead of the feature
>
> Using the Dacapo benchmark (http://dacapobench.org/). My initial results
> are that sampling provides 2.4% with a 512k sampling, 512k being our
> default setting.
>
> - Note: this was without the tradesoap, tradebeans and tomcat benchmarks
> since they did not work with my JDK9 (issue between Dacapo and JDK9 it
> seems)
> - I want to rerun next week to ensure number stability
>
> - Accuracy of the feature
>
> I wrote a small microbenchmark that allocates from two different
> stacktraces at a given ratio. For example, 10% of stacktrace S1 and 90%
> from stacktrace S2. The microbenchmark was run 20 times, I averaged the
> results and looked for accuracy. It seems that statistically it is sound
> since if I allocated10% S1 and 90% S2, with a sampling rate of 512k, I
> obtained 9.61% S1 and 90.49% S2.
>
> Let me know if there are any questions on the numbers and if you'd like to
> see some more data.
>
> Note: this was done using our internal JDK8 implementation since the
> webrev provided by http://cr.openjdk.java.net/
> ~rasbold/heapz/webrev.00/index.html does not yet contain the whole
> implementation and therefore would have been misleading.
>
> Thanks,
> Jc
>
>
> On Tue, Apr 4, 2017 at 3:55 PM, JC Beyler <jcbeyler@google.com> wrote:
>
>> Hi all,
>>
>> To move the discussion forward, with Chuck Rasbold's help to make a
>> webrev, we pushed this:
>> http://cr.openjdk.java.net/~rasbold/heapz/webrev.00/index.html
>> 415 lines changed: 399 ins; 13 del; 3 mod; 51122 unchg
>>
>> This is not a final change that does the whole proposition from the JBS
>> entry: https://bugs.openjdk.java.net/browse/JDK-8177374; what it does
>> show is parts of the implementation that is proposed and hopefully can
>> start the conversation going as I work through the details.
>>
>> For example, the changes to C2 are done here for the allocations:
>> http://cr.openjdk.java.net/~rasbold/heapz/webrev.00/src/share/vm/opto/
>> macro.cpp.patch
>>
>> Hopefully this all makes sense and thank you for all your future comments!
>> Jc
>>
>>
>> On Tue, Dec 13, 2016 at 1:11 PM, JC Beyler <jcbeyler@google.com> wrote:
>>
>>> Hello all,
>>>
>>> This is a follow-up from Jeremy's initial email from last year:
>>> http://mail.openjdk.java.net/pipermail/serviceability-dev/20
>>> 15-June/017543.html
>>>
>>> I've gone ahead and started working on preparing this and Jeremy and I
>>> went down the route of actually writing it up in JEP form:
>>> https://bugs.openjdk.java.net/browse/JDK-8171119
>>>
>>> I think original conversation that happened last year in that thread
>>> still holds true:
>>>
>>> - We have a patch at Google that we think others might be interested in
>>> - It provides a means to understand where the allocation hotspots
>>> are at a very low overhead
>>> - Since it is at a low overhead, we can leave it on by default
>>>
>>> So I come to the mailing list with Jeremy's initial question:
>>> "I thought I would ask if there is any interest / if I should write a
>>> JEP / if I should just forget it."
>>>
>>> A year ago, it seemed some thought it was a good idea, is this still
>>> true?
>>>
>>> Thanks,
>>> Jc
>>>
>>>
>>>
>>
>
[Attachment #3 (text/html)]
<div dir="ltr">Hi all,<div><br></div><div>I've added size information to the \
allocation sampling system. This allows the callback to remember the size of each \
sampled allocation.</div><div><a \
href="http://cr.openjdk.java.net/~rasbold/8171119/webrev.01/">http://cr.openjdk.java.net/~rasbold/8171119/webrev.01/</a><br></div><div><br></div><div>The \
new webrev.01 also adds the actual heap monitoring sampling system in \
files:</div><div><a href="http://cr.openjdk.java.net/~rasbold/8171119/webrev.01/src/sh \
are/vm/runtime/heapMonitoring.cpp.patch">http://cr.openjdk.java.net/~rasbold/8171119/w \
ebrev.01/src/share/vm/runtime/heapMonitoring.cpp.patch</a><br></div><div>and</div><div><a \
href="http://cr.openjdk.java.net/~rasbold/8171119/webrev.01/src/share/vm/runtime/heapM \
onitoring.hpp.patch">http://cr.openjdk.java.net/~rasbold/8171119/webrev.01/src/share/vm/runtime/heapMonitoring.hpp.patch</a><br></div><div><br></div><div>My \
next step is to add the GC part to the webrev, which will allow users to determine \
what objects are live and what are garbage. </div><div><br></div><div>Thanks for \
your attention and let me know if there are any \
questions!</div><div><br></div><div>Have a wonderful \
Friday!</div><div>Jc</div></div><div class="gmail_extra"><br><div \
class="gmail_quote">On Mon, Apr 17, 2017 at 12:37 PM, JC Beyler <span \
dir="ltr"><<a href="mailto:jcbeyler@google.com" \
target="_blank">jcbeyler@google.com</a>></span> wrote:<br><blockquote \
class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc \
solid;padding-left:1ex"><div dir="ltr"><span class="m_-6615616309373040614gmail-im" \
style="font-size:12.8px"><span style="font-size:12.8px">Hi all,</span><div \
style="font-size:12.8px"><br></div><div style="font-size:12.8px">I worked on getting \
a few numbers for overhead and accuracy for my feature. I'm unsure if here is the \
right place to provide the full data, so I am just summarizing here for \
now.</div><div style="font-size:12.8px"><br></div><div style="font-size:12.8px"><span \
style="font-size:12.8px">- Overhead of the feature</span><br></div><div \
style="font-size:12.8px"><br></div></span><div style="font-size:12.8px">Using the \
Dacapo benchmark (<a href="http://dacapobench.org/" \
target="_blank">http://dacapobench.org/</a>). My initial results are that sampling \
provides 2.4% with a 512k sampling, 512k being our default setting.</div><span \
class="m_-6615616309373040614gmail-im" style="font-size:12.8px"><div \
style="font-size:12.8px"><br></div><div style="font-size:12.8px">- Note: this was \
without the tradesoap, tradebeans and tomcat benchmarks since they did not work with \
my JDK9 (issue between Dacapo and JDK9 it seems)</div><div style="font-size:12.8px">- \
I want to rerun next week to ensure number stability</div><div \
style="font-size:12.8px"><br></div><div style="font-size:12.8px">- Accuracy of the \
feature</div><div style="font-size:12.8px"><br></div></span><div \
style="font-size:12.8px">I wrote a small microbenchmark that allocates from two \
different stacktraces at a given ratio. For example, 10% of stacktrace S1 and 90% \
from stacktrace S2. The microbenchmark was run 20 times, I averaged the results and \
looked for accuracy. I<span style="font-size:12.8px">t seems that statistically it is \
sound since if I allocated10% S1 and 90% S2, with a sampling rate of 512k, I obtained \
9.61% S1 and 90.49% S2.</span></div><div class="gmail_extra"><div \
style="font-size:12.8px"><br \
class="m_-6615616309373040614gmail-Apple-interchange-newline">Let me know if there \
are any questions on the numbers and if you'd like to see some more \
data.</div><div><br></div></div><div class="gmail_extra"><div \
style="font-size:12.8px">Note: this was done using our internal JDK8 implementation \
since the webrev provided by <a \
href="http://cr.openjdk.java.net/~rasbold/heapz/webrev.00/index.html" \
target="_blank">http://cr.openjdk.java.net/<wbr>~rasbold/heapz/webrev.00/index<wbr>.html</a> \
does not yet contain the whole implementation and therefore would have been \
misleading.</div><div style="font-size:12.8px"><br></div><div \
style="font-size:12.8px">Thanks,</div><div style="font-size:12.8px">Jc</div><div \
style="font-size:12.8px"><br></div></div><div><div class="h5"><div \
class="gmail_extra"><br><div class="gmail_quote">On Tue, Apr 4, 2017 at 3:55 PM, JC \
Beyler <span dir="ltr"><<a href="mailto:jcbeyler@google.com" \
target="_blank">jcbeyler@google.com</a>></span> wrote:<br><blockquote \
class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid \
rgb(204,204,204);padding-left:1ex"><div dir="ltr">Hi all,<div><br></div><div>To move \
the discussion forward, with Chuck Rasbold's help to make a webrev, we pushed \
this:</div><div><a href="http://cr.openjdk.java.net/~rasbold/heapz/webrev.00/index.html" \
target="_blank">http://cr.openjdk.java.net/~ra<wbr>sbold/heapz/webrev.00/index.ht<wbr>ml</a><br></div><div><span \
style="color:rgb(0,0,0);font-family:"times new \
roman";font-size:12.8px;background-color:rgb(238,238,238)">415 lines changed: \
399 ins; 13 del; 3 mod; 51122 unchg</span><br></div><div><span \
style="color:rgb(0,0,0);font-family:"times new \
roman";font-size:12.8px;background-color:rgb(238,238,238)"><br></span></div><div>This \
is not a final change that does the whole proposition from the JBS entry: <a \
href="https://bugs.openjdk.java.net/browse/JDK-8177374" \
target="_blank">https://bugs.openjdk.java.net/<wbr>browse/JDK-8177374</a>; what it \
does show is parts of the implementation that is proposed and hopefully can start the \
conversation going as I work through the details.</div><div><br></div><div>For \
example, the changes to C2 are done here for the allocations: <a \
href="http://cr.openjdk.java.net/~rasbold/heapz/webrev.00/src/share/vm/opto/macro.cpp.patch" \
target="_blank">http://cr.openjdk<wbr>.java.net/~rasbold/heapz/<wbr>webrev.00/src/share/vm/opto/<wbr>macro.cpp.patch</a></div><div><br></div><div>Hopefully \
this all makes sense and thank you for all your future comments!</div><span \
class="m_-6615616309373040614gmail-HOEnZb"><font \
color="#888888"><div>Jc</div><div><br></div></font></span></div><div \
class="m_-6615616309373040614gmail-HOEnZb"><div \
class="m_-6615616309373040614gmail-h5"><div class="gmail_extra"><br><div \
class="gmail_quote">On Tue, Dec 13, 2016 at 1:11 PM, JC Beyler <span dir="ltr"><<a \
href="mailto:jcbeyler@google.com" target="_blank">jcbeyler@google.com</a>></span> \
wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px \
0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">Hello \
all,<div><br></div><div>This is a follow-up from Jeremy's initial email from last \
year:</div><div><a href="http://mail.openjdk.java.net/pipermail/serviceability-dev/2015-June/017543.html" \
target="_blank">http://mail.openjdk.java.net/p<wbr>ipermail/serviceability-dev/20<wbr>15-June/017543.html</a><br></div><div><br></div><div>I've \
gone ahead and started working on preparing this and Jeremy and I went down the route \
of actually writing it up in JEP form:</div><div><span \
style="color:rgb(17,85,204);font-size:12.8px;text-decoration:underline"><a \
href="https://bugs.openjdk.java.net/" \
target="_blank">https://bugs.openjdk.java.net/</a></span><span \
style="color:rgb(17,85,204);font-size:12.8px;text-decoration:underline"><wbr>browse/JDK-8171119</span><br></div><div><span \
style="color:rgb(17,85,204);font-size:12.8px;text-decoration:underline"><br></span></div><div>I \
think original conversation that happened last year in that thread still holds \
true:</div><div><br></div><div> - We have a patch at Google that we think others \
might be interested in</div><div> - It provides a means to understand where the \
allocation hotspots are at a very low overhead</div><div> - Since it is at a low \
overhead, we can leave it on by default</div><div><br></div><div>So I come to the \
mailing list with Jeremy's initial question: </div><div>"<span \
style="color:rgb(0,0,0)">I thought I would ask if there is any </span><span \
style="color:rgb(0,0,0)">interest / if I should write a JEP / if I should just forget \
it."</span></div><div><span style="color:rgb(0,0,0)"><br></span></div><div><font \
color="#000000">A year ago, it seemed some thought it was a good idea, is this still \
true?</font></div><div><font color="#000000"><br></font></div><div><font \
color="#000000">Thanks,</font></div><div><font \
color="#000000">Jc</font></div><div><br></div><div><br></div></div> \
</blockquote></div><br></div> \
</div></div></blockquote></div><br></div></div></div></div> \
</blockquote></div><br></div>
[prev in list] [next in list] [prev in thread] [next in thread]
Configure |
About |
News |
Add a list |
Sponsored by KoreLogic