[prev in list] [next in list] [prev in thread] [next in thread]
List: openjdk-serviceability-dev
Subject: Re: GcInfo: longest pause duration?
From: Roman Kennke <rkennke () redhat ! com>
Date: 2017-10-24 15:08:52
Message-ID: 383488f8-8012-d082-c541-6ab4117798a3 () redhat ! com
[Download RAW message or body]
Hi Paul,
> + hotspot-gc-dev.
>
> I'd say that hsperfdata is the right place to put detailed
> collector-specific data, and JMX the place to put anything that might
> be generalized.
>
> How far does anyone want to go with this? I took a look at Azul's
> GC-related MXBean docs
>
> http://docs.azul.com/zing/Zing_UserGuide/Zing_UserGuide/ZingMXBeans.htm
>
> http://docs.azul.com/zing/ZingMXBeans_javadoc/
>
> which has good ideas in it, e.g., PauseDetails and a data collection
> interval concept. My proposed getMaxPauseDuration() seems to be the
> same as the max of Azul's
> GarbageCollectorMXBean.getLastGCDetails.getCollectorMaxPauseDetails().
>
> A JEP might be in order. Opinions?
>
I like the idea of publishing GC pause info via JMX. Yes I think it
makes sense to draft up a JEP about it.
Roman
> Thanks,
>
> Paul
>
> *From: *Jeremy Manson <jeremymanson@google.com>
> *Date: *Monday, October 23, 2017 at 11:50 AM
> *To: *"Hohensee, Paul" <hohensee@amazon.com>, Derek Thomson
> <dthomson@google.com>, Wessam Hassanein <wessam@google.com>, Man Cao
> <manc@google.com>
> *Cc: *"serviceability-dev@openjdk.java.net"
> <serviceability-dev@openjdk.java.net>
> *Subject: *Re: GcInfo: longest pause duration?
>
> Yes, this information is useful.
>
> We've done a couple of things to get at it, which folks might find
> interesting.
>
> 1) We extended the hsperfdata to include a large number of stats for
> CMS. Parsing hsperfdata is ridiculously easy, and hasn't changed in
> years, so we just parse it directly and report the values to
> monitoring tools. I'm too lazy to retype what we export, so are the
> relevant CMS variables from our code; what they do should be
> relatively obvious:
>
> _concurrent_mode_failure_count =
> PerfDataManager::create_counter(NULL_NS,
> "concurrent-mode-failure-count",
> PerfData::U_Events, CHECK);
> _concurrent_mode_failure_due_to_fragmentation_count =
> PerfDataManager::create_counter(NULL_NS,
> "concurrent-mode-failure-due-to-fragmentation-count",
> PerfData::U_Events, CHECK);
> _concurrent_mode_failure_time =
> PerfDataManager::create_counter(NULL_NS,
> "concurrent-mode-failure-time",
> PerfData::U_Ticks, CHECK);
> _cms_initial_mark_count =
> PerfDataManager::create_counter(NULL_NS,
> "*cms-initial-mark-count*",
> PerfData::U_Events, CHECK);
> _cms_initial_mark_time =
> PerfDataManager::create_counter(NULL_NS, "cms-initial-mark-time",
> PerfData::U_Ticks, CHECK);
> _cms_remark_count =
> PerfDataManager::create_counter(NULL_NS, "cms-remark-count",
> PerfData::U_Events, CHECK);
> _cms_remark_time =
> PerfDataManager::create_counter(NULL_NS, "cms-remark-time",
> PerfData::U_Ticks, CHECK);
> _cms_full_gc_count =
> PerfDataManager::create_counter(NULL_NS, "cms-full-gc-count",
> PerfData::U_Events, CHECK);
> _cms_full_gc_time =
> PerfDataManager::create_counter(NULL_NS, "cms-full-gc-time",
> PerfData::U_Ticks, CHECK);
> _cms_compacting_full_gc_count =
> PerfDataManager::create_counter(NULL_NS,
> "cms-compacting-full-gc-count",
> PerfData::U_Events, CHECK);
> _cms_compacting_full_gc_time =
> PerfDataManager::create_counter(NULL_NS,
> "cms-compacting-full-gc-time",
> PerfData::U_Ticks, CHECK);
> _cms_noncompacting_full_gc_count =
> PerfDataManager::create_counter(NULL_NS,
> "cms-noncompacting-full-gc-count",
> PerfData::U_Events, CHECK);
> _cms_noncompacting_full_gc_time =
> PerfDataManager::create_counter(NULL_NS,
> "cms-noncompacting-full-gc-time",
> PerfData::U_Ticks, CHECK);
> _cms_conc_mark_count =
> PerfDataManager::create_counter(NULL_NS, "cms-conc-mark-count",
> PerfData::U_Events, CHECK);
> _cms_conc_mark_time =
> PerfDataManager::create_counter(NULL_NS, "cms-conc-mark-time",
> PerfData::U_Ticks, CHECK);
> _cms_conc_preclean_count =
> PerfDataManager::create_counter(NULL_NS,
> "cms-conc-preclean-count",
> PerfData::U_Events, CHECK);
> _cms_conc_preclean_time =
> PerfDataManager::create_counter(NULL_NS, "cms-conc-preclean-time",
> PerfData::U_Ticks, CHECK);
> _cms_conc_abortable_preclean_count =
> PerfDataManager::create_counter(NULL_NS,
> "cms-conc-abortable-preclean-count",
> PerfData::U_Events, CHECK);
> _cms_conc_abortable_preclean_time =
> PerfDataManager::create_counter(NULL_NS,
> "cms-conc-abortable-preclean-time",
> PerfData::U_Ticks, CHECK);
> _cms_conc_sweep_count =
> PerfDataManager::create_counter(NULL_NS, "cms-conc-sweep-count",
> PerfData::U_Events, CHECK);
> _cms_conc_sweep_time =
> PerfDataManager::create_counter(NULL_NS, "cms-conc-sweep-time",
> PerfData::U_Ticks, CHECK);
> _cms_conc_reset_count =
> PerfDataManager::create_counter(NULL_NS, "cms-conc-reset-count",
> PerfData::U_Events, CHECK);
> _cms_conc_reset_time =
> PerfDataManager::create_counter(NULL_NS, "cms-conc-reset-time",
> PerfData::U_Ticks, CHECK);
> _cms_collection_count =
> PerfDataManager::create_counter(NULL_NS, "cms-collection-count",
> PerfData::U_Events, CHECK);
> _cms_collection_time =
> PerfDataManager::create_counter(NULL_NS, "cms-collection-time",
> PerfData::U_Ticks, CHECK);
> _cms_ref_proc_count =
> PerfDataManager::create_counter(NULL_NS, "cms-ref-proc-count",
> PerfData::U_Events, CHECK);
> _cms_ref_proc_time =
> PerfDataManager::create_counter(NULL_NS, "cms-ref-proc-time",
> PerfData::U_Ticks, CHECK);
> _yg_alloc_bytes =
> PerfDataManager::create_variable(NULL_NS,
> "jvm-gc-cms-young-gen-alloc-bytes",
> PerfData::U_None, CHECK);
> _yg_promo_bytes =
> PerfDataManager::create_variable(NULL_NS,
> "jvm-gc-cms-young-gen-promo-bytes",
> PerfData::U_None, CHECK);
> _og_direct_alloc_bytes =
> PerfDataManager::create_variable(NULL_NS,
> "jvm-gc-cms-old-gen-directalloc-bytes",
> PerfData::U_None, CHECK);
> _og_alloc_bytes =
> PerfDataManager::create_variable(NULL_NS,
> "jvm-gc-cms-old-gen-alloc-bytes",
> PerfData::U_None, CHECK);
> _total_alloc_bytes =
> PerfDataManager::create_variable(NULL_NS,
> "jvm-gc-cms-total-alloc-bytes",
> PerfData::U_None, CHECK);
>
> 2) We also implemented an API built on JFR's handling of GC events.
> We use it to export the GC history to monitoring tools in a format
> that doesn't involve parsing logs that can change.
>
> (Prior to JFR, we had a bespoke implementation where we instrumented
> everywhere a log happened, but we fortunately don't have to do that
> anymore.)
>
> We'd love both of these things to be something that was more generally
> usable.
>
> Jeremy
>
> On Mon, Oct 23, 2017 at 9:31 AM, Hohensee, Paul <hohensee@amazon.com
> <mailto:hohensee@amazon.com>> wrote:
>
> I'd like to solicit opinions on reporting GC pause duration
> (stopped-world pause time) via JMX. This info would be useful in
> figuring out whether or not GC pause times are factors in failing
> to meet response time SLAs. The info is of course available
> directly from GC logs, but parsing logs is fraught and JMX doesn't
> seem to report the equivalent info.
>
> GcInfo
>
> https://docs.oracle.com/javase/9/docs/api/com/sun/management/GcInfo.html
>
> has a getDuration() method which works fine for the non-concurrent
> collectors (since they're STW), but for CMS and G1 it appears to
> report the duration of an entire concurrent cycle, which isn't
> what I want. The number of STW pauses during a concurrent cycle
> varies by collector, so ideally there would be a method that
> reports cause (as a string) and duration for each STW pause. If
> that's too much, perhaps the minimum might be a
> getMaxPauseDuration() method that reports the maximum pause
> duration of all the STW pauses that happen during a concurrent cycle.
>
> Relatedly, the full compacting GCs that happen as a result of CMS
> and G1 concurrent mode failure aren't reported separately from
> concurrent cycles. It would be useful to differentiate these from
> "ConcurrentMarkSweep" and "G1 Old Generation". Perhaps add
> collector types to CMS and G1, vis. "MarkSweepCompact" (which
> already exists and is literally what's executed by CMS) and a new
> "G1 MarkSweepCompact" collector for G1.
>
> If there's a consensus that something should be done about either
> of these issues, I'd be happy to file RFE(s) and do the work.
>
> Thanks,
>
> Paul
>
[Attachment #3 (text/html)]
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
</head>
<body text="#000000" bgcolor="#FFFFFF">
<div class="moz-cite-prefix">Hi Paul,<br>
<br>
</div>
<blockquote type="cite"
cite="mid:DC434A34-F403-4009-9F68-906D05CCF248@amazon.com">
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="Title" content="">
<meta name="Keywords" content="">
<meta name="Generator" content="Microsoft Word 15 (filtered
medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:"Courier New";
panose-1:2 7 3 9 2 2 5 2 4 4;}
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
pre
{mso-style-priority:99;
mso-style-link:"HTML Preformatted Char";
margin:0in;
margin-bottom:.0001pt;
font-size:10.0pt;
font-family:"Courier New",serif;}
span.HTMLPreformattedChar
{mso-style-name:"HTML Preformatted Char";
mso-style-priority:99;
mso-style-link:"HTML Preformatted";
font-family:"Courier",serif;}
span.gmail-stx-line
{mso-style-name:gmail-stx-line;}
span.gmail-stx-string
{mso-style-name:gmail-stx-string;}
span.gmail-chl
{mso-style-name:gmail-chl;}
span.hoenzb
{mso-style-name:hoenzb;}
span.EmailStyle23
{mso-style-type:personal-reply;
font-family:"Calibri",sans-serif;
color:windowtext;}
span.msoIns
{mso-style-type:export-only;
mso-style-name:"";
text-decoration:underline;
color:teal;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style>
<div class="WordSection1">
<p class="MsoNormal">+ hotspot-gc-dev.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">I'd say that hsperfdata is the right place
to put detailed collector-specific data, and JMX the place to
put anything that might be generalized.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">How far does anyone want to go with this? I
took a look at Azul's GC-related MXBean docs<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><a
href="http://docs.azul.com/zing/Zing_UserGuide/Zing_UserGuide/ZingMXBeans.htm"
moz-do-not-send="true">http://docs.azul.com/zing/Zing_UserGuide/Zing_UserGuide/ZingMXBeans.htm</a><o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><a
href="http://docs.azul.com/zing/ZingMXBeans_javadoc/"
moz-do-not-send="true">http://docs.azul.com/zing/ZingMXBeans_javadoc/</a><o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">which has good ideas in it, e.g.,
PauseDetails and a data collection interval concept. My
proposed getMaxPauseDuration() seems to be the same as the max
of Azul's
GarbageCollectorMXBean.getLastGCDetails.getCollectorMaxPauseDetails().<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">A JEP might be in order. Opinions?<o:p></o:p></p>
</div>
</blockquote>
I like the idea of publishing GC pause info via JMX. Yes I think it
makes sense to draft up a JEP about it.<br>
<br>
Roman<br>
<br>
<br>
<br>
<blockquote type="cite"
cite="mid:DC434A34-F403-4009-9F68-906D05CCF248@amazon.com">
<div class="WordSection1">
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Thanks,<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Paul<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<div style="border:none;border-top:solid #B5C4DF
1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><b><span
style="font-size:12.0pt;color:black">From: </span></b><span
style="font-size:12.0pt;color:black">Jeremy Manson
<a class="moz-txt-link-rfc2396E" \
href="mailto:jeremymanson@google.com"><jeremymanson@google.com></a><br> \
<b>Date: </b>Monday, October 23, 2017 at 11:50 AM<br>
<b>To: </b>"Hohensee, Paul" <a class="moz-txt-link-rfc2396E" \
href="mailto:hohensee@amazon.com"><hohensee@amazon.com></a>,
Derek Thomson <a class="moz-txt-link-rfc2396E" \
href="mailto:dthomson@google.com"><dthomson@google.com></a>, \
Wessam
Hassanein <a class="moz-txt-link-rfc2396E" \
href="mailto:wessam@google.com"><wessam@google.com></a>, Man \
Cao
<a class="moz-txt-link-rfc2396E" \
href="mailto:manc@google.com"><manc@google.com></a><br>
<b>Cc: </b><a class="moz-txt-link-rfc2396E" \
href="mailto:serviceability-dev@openjdk.java.net">"serviceability-dev@openjdk.java.net"</a>
<a class="moz-txt-link-rfc2396E" \
href="mailto:serviceability-dev@openjdk.java.net"><serviceability-dev@openjdk.java.net></a><br>
<b>Subject: </b>Re: GcInfo: longest pause \
duration?<o:p></o:p></span></p> </div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">Yes, this information is useful. <o:p></o:p></p>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">We've done a couple of things to get at
it, which folks might find interesting.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">1) We extended the hsperfdata to
include a large number of stats for CMS. Parsing
hsperfdata is ridiculously easy, and hasn't changed in
years, so we just parse it directly and report the values
to monitoring tools. I'm too lazy to retype what we
export, so are the relevant CMS variables from our code;
what they do should be relatively obvious:<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<pre style="outline:none" id="gmail-c1"><span \
class="gmail-stx-line"><span style="font-size:13.5pt;color:black"> \
_concurrent_mode_failure_count =<o:p></o:p></span></span></pre> <pre><span \
class="gmail-stx-line"><span style="font-size:13.5pt;color:black"> \
PerfDataManager::create_counter(NULL_NS, </span></span><span \
class="gmail-stx-string"><span \
style="font-size:13.5pt;color:#008800">"concurrent-mode-failure-count"</span></span><span \
class="gmail-stx-line"><span \
style="font-size:13.5pt;color:black">,<o:p></o:p></span></span></pre> <pre><span \
class="gmail-stx-line"><span style="font-size:13.5pt;color:black"> \
PerfData::U_Events, CHECK);<o:p></o:p></span></span></pre> <pre><span \
class="gmail-stx-line"><span style="font-size:13.5pt;color:black"> \
_concurrent_mode_failure_due_to_fragmentation_count =<o:p></o:p></span></span></pre> \
<pre><span class="gmail-stx-line"><span style="font-size:13.5pt;color:black"> \
PerfDataManager::create_counter(NULL_NS, </span></span><span \
class="gmail-stx-string"><span \
style="font-size:13.5pt;color:#008800">"concurrent-mode-failure-due-to-fragmentation-count"</span></span><span \
class="gmail-stx-line"><span \
style="font-size:13.5pt;color:black">,<o:p></o:p></span></span></pre> <pre><span \
class="gmail-stx-line"><span style="font-size:13.5pt;color:black"> \
PerfData::U_Events, CHECK);<o:p></o:p></span></span></pre>
<pre><span class="gmail-stx-line"><span \
style="font-size:13.5pt;color:black"> _concurrent_mode_failure_time \
=<o:p></o:p></span></span></pre> <pre><span class="gmail-stx-line"><span \
style="font-size:13.5pt;color:black"> \
PerfDataManager::create_counter(NULL_NS, </span></span><span \
class="gmail-stx-string"><span \
style="font-size:13.5pt;color:#008800">"concurrent-mode-failure-time"</span></span><span \
class="gmail-stx-line"><span \
style="font-size:13.5pt;color:black">,<o:p></o:p></span></span></pre> <pre><span \
class="gmail-stx-line"><span style="font-size:13.5pt;color:black"> \
PerfData::U_Ticks, CHECK);<o:p></o:p></span></span></pre>
<pre><span class="gmail-stx-line"><span \
style="font-size:13.5pt;color:black"> _cms_initial_mark_count \
=<o:p></o:p></span></span></pre> <pre><span class="gmail-stx-line"><span \
style="font-size:13.5pt;color:black"> \
PerfDataManager::create_counter(NULL_NS, </span></span><span \
class="gmail-stx-string"><span style="font-size:13.5pt;color:#008800">"<b><span \
style="background:white">cms-initial-mark-count</span></b>"</span></span><span \
class="gmail-stx-line"><span \
style="font-size:13.5pt;color:black">,<o:p></o:p></span></span></pre> <pre><span \
class="gmail-stx-line"><span style="font-size:13.5pt;color:black"> \
PerfData::U_Events, CHECK);<o:p></o:p></span></span></pre>
<pre><span class="gmail-stx-line"><span \
style="font-size:13.5pt;color:black"> _cms_initial_mark_time \
=<o:p></o:p></span></span></pre> <pre><span class="gmail-stx-line"><span \
style="font-size:13.5pt;color:black"> \
PerfDataManager::create_counter(NULL_NS, </span></span><span \
class="gmail-stx-string"><span \
style="font-size:13.5pt;color:#008800">"cms-initial-mark-time"</span></span><span \
class="gmail-stx-line"><span \
style="font-size:13.5pt;color:black">,<o:p></o:p></span></span></pre> <pre><span \
class="gmail-stx-line"><span style="font-size:13.5pt;color:black"> \
PerfData::U_Ticks, CHECK);<o:p></o:p></span></span></pre>
<pre><span class="gmail-stx-line"><span \
style="font-size:13.5pt;color:black"> _cms_remark_count \
=<o:p></o:p></span></span></pre> <pre><span class="gmail-stx-line"><span \
style="font-size:13.5pt;color:black"> \
PerfDataManager::create_counter(NULL_NS, </span></span><span \
class="gmail-stx-string"><span \
style="font-size:13.5pt;color:#008800">"cms-remark-count"</span></span><span \
class="gmail-stx-line"><span \
style="font-size:13.5pt;color:black">,<o:p></o:p></span></span></pre> <pre><span \
class="gmail-stx-line"><span style="font-size:13.5pt;color:black"> \
PerfData::U_Events, CHECK);<o:p></o:p></span></span></pre>
<pre><span class="gmail-stx-line"><span \
style="font-size:13.5pt;color:black"> _cms_remark_time \
=<o:p></o:p></span></span></pre> <pre><span class="gmail-stx-line"><span \
style="font-size:13.5pt;color:black"> \
PerfDataManager::create_counter(NULL_NS, </span></span><span \
class="gmail-stx-string"><span \
style="font-size:13.5pt;color:#008800">"cms-remark-time"</span></span><span \
class="gmail-stx-line"><span \
style="font-size:13.5pt;color:black">,<o:p></o:p></span></span></pre> <pre><span \
class="gmail-stx-line"><span style="font-size:13.5pt;color:black"> \
PerfData::U_Ticks, CHECK);<o:p></o:p></span></span></pre>
<pre><span class="gmail-stx-line"><span \
style="font-size:13.5pt;color:black"> _cms_full_gc_count \
=<o:p></o:p></span></span></pre> <pre><span class="gmail-stx-line"><span \
style="font-size:13.5pt;color:black"> \
PerfDataManager::create_counter(NULL_NS, </span></span><span \
class="gmail-stx-string"><span \
style="font-size:13.5pt;color:#008800">"cms-full-gc-count"</span></span><span \
class="gmail-stx-line"><span \
style="font-size:13.5pt;color:black">,<o:p></o:p></span></span></pre> <pre><span \
class="gmail-stx-line"><span style="font-size:13.5pt;color:black"> \
PerfData::U_Events, CHECK);<o:p></o:p></span></span></pre>
<pre><span class="gmail-stx-line"><span \
style="font-size:13.5pt;color:black"> _cms_full_gc_time \
=<o:p></o:p></span></span></pre> <pre><span class="gmail-stx-line"><span \
style="font-size:13.5pt;color:black"> \
PerfDataManager::create_counter(NULL_NS, </span></span><span \
class="gmail-stx-string"><span \
style="font-size:13.5pt;color:#008800">"cms-full-gc-time"</span></span><span \
class="gmail-stx-line"><span \
style="font-size:13.5pt;color:black">,<o:p></o:p></span></span></pre> <pre><span \
class="gmail-stx-line"><span style="font-size:13.5pt;color:black"> \
PerfData::U_Ticks, CHECK);<o:p></o:p></span></span></pre>
<pre><span class="gmail-stx-line"><span \
style="font-size:13.5pt;color:black"> _cms_compacting_full_gc_count \
=<o:p></o:p></span></span></pre> <pre><span class="gmail-stx-line"><span \
style="font-size:13.5pt;color:black"> \
PerfDataManager::create_counter(NULL_NS, </span></span><span \
class="gmail-stx-string"><span \
style="font-size:13.5pt;color:#008800">"cms-compacting-full-gc-count"</span></span><span \
class="gmail-stx-line"><span \
style="font-size:13.5pt;color:black">,<o:p></o:p></span></span></pre> <pre><span \
class="gmail-stx-line"><span style="font-size:13.5pt;color:black"> \
PerfData::U_Events, CHECK);<o:p></o:p></span></span></pre>
<pre><span class="gmail-stx-line"><span \
style="font-size:13.5pt;color:black"> _cms_compacting_full_gc_time \
=<o:p></o:p></span></span></pre> <pre><span class="gmail-stx-line"><span \
style="font-size:13.5pt;color:black"> \
PerfDataManager::create_counter(NULL_NS, </span></span><span \
class="gmail-stx-string"><span \
style="font-size:13.5pt;color:#008800">"cms-compacting-full-gc-time"</span></span><span \
class="gmail-stx-line"><span \
style="font-size:13.5pt;color:black">,<o:p></o:p></span></span></pre> <pre><span \
class="gmail-stx-line"><span style="font-size:13.5pt;color:black"> \
PerfData::U_Ticks, CHECK);<o:p></o:p></span></span></pre>
<pre><span class="gmail-stx-line"><span \
style="font-size:13.5pt;color:black"> _cms_noncompacting_full_gc_count \
=<o:p></o:p></span></span></pre> <pre><span class="gmail-stx-line"><span \
style="font-size:13.5pt;color:black"> \
PerfDataManager::create_counter(NULL_NS, </span></span><span \
class="gmail-stx-string"><span \
style="font-size:13.5pt;color:#008800">"cms-noncompacting-full-gc-count"</span></span><span \
class="gmail-stx-line"><span \
style="font-size:13.5pt;color:black">,<o:p></o:p></span></span></pre> <pre><span \
class="gmail-stx-line"><span style="font-size:13.5pt;color:black"> \
PerfData::U_Events, CHECK);<o:p></o:p></span></span></pre>
<pre><span class="gmail-stx-line"><span \
style="font-size:13.5pt;color:black"> _cms_noncompacting_full_gc_time \
=<o:p></o:p></span></span></pre> <pre><span class="gmail-stx-line"><span \
style="font-size:13.5pt;color:black"> \
PerfDataManager::create_counter(NULL_NS, </span></span><span \
class="gmail-stx-string"><span \
style="font-size:13.5pt;color:#008800">"cms-noncompacting-full-gc-time"</span></span><span \
class="gmail-stx-line"><span \
style="font-size:13.5pt;color:black">,<o:p></o:p></span></span></pre> <pre><span \
class="gmail-stx-line"><span style="font-size:13.5pt;color:black"> \
PerfData::U_Ticks, CHECK);<o:p></o:p></span></span></pre>
<pre><span class="gmail-stx-line"><span \
style="font-size:13.5pt;color:black"> _cms_conc_mark_count \
=<o:p></o:p></span></span></pre> <pre><span class="gmail-stx-line"><span \
style="font-size:13.5pt;color:black"> \
PerfDataManager::create_counter(NULL_NS, </span></span><span \
class="gmail-stx-string"><span \
style="font-size:13.5pt;color:#008800">"cms-conc-mark-count"</span></span><span \
class="gmail-stx-line"><span \
style="font-size:13.5pt;color:black">,<o:p></o:p></span></span></pre> <pre><span \
class="gmail-stx-line"><span style="font-size:13.5pt;color:black"> \
PerfData::U_Events, CHECK);<o:p></o:p></span></span></pre>
<pre><span class="gmail-stx-line"><span \
style="font-size:13.5pt;color:black"> _cms_conc_mark_time \
=<o:p></o:p></span></span></pre> <pre><span class="gmail-stx-line"><span \
style="font-size:13.5pt;color:black"> \
PerfDataManager::create_counter(NULL_NS, </span></span><span \
class="gmail-stx-string"><span \
style="font-size:13.5pt;color:#008800">"cms-conc-mark-time"</span></span><span \
class="gmail-stx-line"><span \
style="font-size:13.5pt;color:black">,<o:p></o:p></span></span></pre> <pre><span \
class="gmail-stx-line"><span style="font-size:13.5pt;color:black"> \
PerfData::U_Ticks, CHECK);<o:p></o:p></span></span></pre>
<pre><span class="gmail-stx-line"><span \
style="font-size:13.5pt;color:black"> _cms_conc_preclean_count \
=<o:p></o:p></span></span></pre> <pre><span class="gmail-stx-line"><span \
style="font-size:13.5pt;color:black"> \
PerfDataManager::create_counter(NULL_NS, </span></span><span \
class="gmail-stx-string"><span \
style="font-size:13.5pt;color:#008800">"cms-conc-preclean-count"</span></span><span \
class="gmail-stx-line"><span \
style="font-size:13.5pt;color:black">,<o:p></o:p></span></span></pre> <pre><span \
class="gmail-stx-line"><span style="font-size:13.5pt;color:black"> \
PerfData::U_Events, CHECK);<o:p></o:p></span></span></pre>
<pre><span class="gmail-stx-line"><span \
style="font-size:13.5pt;color:black"> _cms_conc_preclean_time \
=<o:p></o:p></span></span></pre> <pre><span class="gmail-stx-line"><span \
style="font-size:13.5pt;color:black"> \
PerfDataManager::create_counter(NULL_NS, </span></span><span \
class="gmail-stx-string"><span \
style="font-size:13.5pt;color:#008800">"cms-conc-preclean-time"</span></span><span \
class="gmail-stx-line"><span \
style="font-size:13.5pt;color:black">,<o:p></o:p></span></span></pre> <pre><span \
class="gmail-stx-line"><span style="font-size:13.5pt;color:black"> \
PerfData::U_Ticks, CHECK);<o:p></o:p></span></span></pre>
<pre><span class="gmail-stx-line"><span \
style="font-size:13.5pt;color:black"> _cms_conc_abortable_preclean_count \
=<o:p></o:p></span></span></pre> <pre><span class="gmail-stx-line"><span \
style="font-size:13.5pt;color:black"> \
PerfDataManager::create_counter(NULL_NS, </span></span><span \
class="gmail-stx-string"><span \
style="font-size:13.5pt;color:#008800">"cms-conc-abortable-preclean-count"</span></span><span \
class="gmail-stx-line"><span \
style="font-size:13.5pt;color:black">,<o:p></o:p></span></span></pre> <pre><span \
class="gmail-stx-line"><span style="font-size:13.5pt;color:black"> \
PerfData::U_Events, CHECK);<o:p></o:p></span></span></pre>
<pre><span class="gmail-stx-line"><span \
style="font-size:13.5pt;color:black"> _cms_conc_abortable_preclean_time \
=<o:p></o:p></span></span></pre> <pre><span class="gmail-stx-line"><span \
style="font-size:13.5pt;color:black"> \
PerfDataManager::create_counter(NULL_NS, </span></span><span \
class="gmail-stx-string"><span \
style="font-size:13.5pt;color:#008800">"cms-conc-abortable-preclean-time"</span></span><span \
class="gmail-stx-line"><span \
style="font-size:13.5pt;color:black">,<o:p></o:p></span></span></pre> <pre><span \
class="gmail-stx-line"><span style="font-size:13.5pt;color:black"> \
PerfData::U_Ticks, CHECK);<o:p></o:p></span></span></pre>
<pre><span class="gmail-stx-line"><span \
style="font-size:13.5pt;color:black"> _cms_conc_sweep_count \
=<o:p></o:p></span></span></pre> <pre><span class="gmail-stx-line"><span \
style="font-size:13.5pt;color:black"> \
PerfDataManager::create_counter(NULL_NS, </span></span><span \
class="gmail-stx-string"><span \
style="font-size:13.5pt;color:#008800">"cms-conc-sweep-count"</span></span><span \
class="gmail-stx-line"><span \
style="font-size:13.5pt;color:black">,<o:p></o:p></span></span></pre> <pre><span \
class="gmail-stx-line"><span style="font-size:13.5pt;color:black"> \
PerfData::U_Events, CHECK);<o:p></o:p></span></span></pre>
<pre><span class="gmail-stx-line"><span \
style="font-size:13.5pt;color:black"> _cms_conc_sweep_time \
=<o:p></o:p></span></span></pre> <pre><span class="gmail-stx-line"><span \
style="font-size:13.5pt;color:black"> \
PerfDataManager::create_counter(NULL_NS, </span></span><span \
class="gmail-stx-string"><span \
style="font-size:13.5pt;color:#008800">"cms-conc-sweep-time"</span></span><span \
class="gmail-stx-line"><span \
style="font-size:13.5pt;color:black">,<o:p></o:p></span></span></pre> <pre><span \
class="gmail-stx-line"><span style="font-size:13.5pt;color:black"> \
PerfData::U_Ticks, CHECK);<o:p></o:p></span></span></pre>
<pre><span class="gmail-stx-line"><span \
style="font-size:13.5pt;color:black"> _cms_conc_reset_count \
=<o:p></o:p></span></span></pre> <pre><span class="gmail-stx-line"><span \
style="font-size:13.5pt;color:black"> \
PerfDataManager::create_counter(NULL_NS, </span></span><span \
class="gmail-stx-string"><span \
style="font-size:13.5pt;color:#008800">"cms-conc-reset-count"</span></span><span \
class="gmail-stx-line"><span \
style="font-size:13.5pt;color:black">,<o:p></o:p></span></span></pre> <pre><span \
class="gmail-stx-line"><span style="font-size:13.5pt;color:black"> \
PerfData::U_Events, CHECK);<o:p></o:p></span></span></pre>
<pre><span class="gmail-stx-line"><span \
style="font-size:13.5pt;color:black"> _cms_conc_reset_time \
=<o:p></o:p></span></span></pre> <pre><span class="gmail-stx-line"><span \
style="font-size:13.5pt;color:black"> \
PerfDataManager::create_counter(NULL_NS, </span></span><span \
class="gmail-stx-string"><span \
style="font-size:13.5pt;color:#008800">"cms-conc-reset-time"</span></span><span \
class="gmail-stx-line"><span \
style="font-size:13.5pt;color:black">,<o:p></o:p></span></span></pre> <pre><span \
class="gmail-stx-line"><span style="font-size:13.5pt;color:black"> \
PerfData::U_Ticks, CHECK);<o:p></o:p></span></span></pre>
<pre><span class="gmail-stx-line"><span \
style="font-size:13.5pt;color:black"> _cms_collection_count \
=<o:p></o:p></span></span></pre> <pre><span class="gmail-stx-line"><span \
style="font-size:13.5pt;color:black"> \
PerfDataManager::create_counter(NULL_NS, </span></span><span \
class="gmail-stx-string"><span \
style="font-size:13.5pt;color:#008800">"cms-collection-count"</span></span><span \
class="gmail-stx-line"><span \
style="font-size:13.5pt;color:black">,<o:p></o:p></span></span></pre> <pre><span \
class="gmail-stx-line"><span style="font-size:13.5pt;color:black"> \
PerfData::U_Events, CHECK);<o:p></o:p></span></span></pre>
<pre><span class="gmail-stx-line"><span \
style="font-size:13.5pt;color:black"> _cms_collection_time \
=<o:p></o:p></span></span></pre> <pre><span class="gmail-stx-line"><span \
style="font-size:13.5pt;color:black"> \
PerfDataManager::create_counter(NULL_NS, </span></span><span \
class="gmail-stx-string"><span \
style="font-size:13.5pt;color:#008800">"cms-collection-time"</span></span><span \
class="gmail-stx-line"><span \
style="font-size:13.5pt;color:black">,<o:p></o:p></span></span></pre> <pre><span \
class="gmail-stx-line"><span style="font-size:13.5pt;color:black"> \
PerfData::U_Ticks, CHECK);<o:p></o:p></span></span></pre>
<pre><span class="gmail-stx-line"><span \
style="font-size:13.5pt;color:black"> _cms_ref_proc_count \
=<o:p></o:p></span></span></pre> <pre><span class="gmail-stx-line"><span \
style="font-size:13.5pt;color:black"> \
PerfDataManager::create_counter(NULL_NS, </span></span><span \
class="gmail-stx-string"><span \
style="font-size:13.5pt;color:#008800">"cms-ref-proc-count"</span></span><span \
class="gmail-stx-line"><span \
style="font-size:13.5pt;color:black">,<o:p></o:p></span></span></pre> <pre><span \
class="gmail-stx-line"><span style="font-size:13.5pt;color:black"> \
PerfData::U_Events, CHECK);<o:p></o:p></span></span></pre>
<pre><span class="gmail-stx-line"><span \
style="font-size:13.5pt;color:black"> _cms_ref_proc_time \
=<o:p></o:p></span></span></pre> <pre><span class="gmail-stx-line"><span \
style="font-size:13.5pt;color:black"> \
PerfDataManager::create_counter(NULL_NS, </span></span><span \
class="gmail-stx-string"><span \
style="font-size:13.5pt;color:#008800">"cms-ref-proc-time"</span></span><span \
class="gmail-stx-line"><span \
style="font-size:13.5pt;color:black">,<o:p></o:p></span></span></pre> <pre><span \
class="gmail-stx-line"><span style="font-size:13.5pt;color:black"> \
PerfData::U_Ticks, CHECK);<o:p></o:p></span></span></pre>
<pre><span class="gmail-stx-line"><span \
style="font-size:13.5pt;color:black"><o:p> \
</o:p></span></span></pre>
<pre><span class="gmail-stx-line"><span \
style="font-size:13.5pt;color:black"> _yg_alloc_bytes \
=<o:p></o:p></span></span></pre> <pre><span class="gmail-stx-line"><span \
style="font-size:13.5pt;color:black"> \
PerfDataManager::create_variable(NULL_NS, </span></span><span \
class="gmail-stx-string"><span \
style="font-size:13.5pt;color:#008800">"jvm-gc-cms-young-gen-alloc-bytes"</span></span><span \
class="gmail-stx-line"><span \
style="font-size:13.5pt;color:black">,<o:p></o:p></span></span></pre> <pre><span \
class="gmail-stx-line"><span style="font-size:13.5pt;color:black"> \
PerfData::U_None, CHECK);<o:p></o:p></span></span></pre>
<pre><span class="gmail-stx-line"><span \
style="font-size:13.5pt;color:black"> _yg_promo_bytes \
=<o:p></o:p></span></span></pre> <pre><span class="gmail-stx-line"><span \
style="font-size:13.5pt;color:black"> \
PerfDataManager::create_variable(NULL_NS, </span></span><span \
class="gmail-stx-string"><span \
style="font-size:13.5pt;color:#008800">"jvm-gc-cms-young-gen-promo-bytes"</span></span><span \
class="gmail-stx-line"><span \
style="font-size:13.5pt;color:black">,<o:p></o:p></span></span></pre> <pre><span \
class="gmail-stx-line"><span style="font-size:13.5pt;color:black"> \
PerfData::U_None, CHECK);<o:p></o:p></span></span></pre>
<pre><span class="gmail-stx-line"><span \
style="font-size:13.5pt;color:black"> _og_direct_alloc_bytes \
=<o:p></o:p></span></span></pre> <pre><span class="gmail-stx-line"><span \
style="font-size:13.5pt;color:black"> \
PerfDataManager::create_variable(NULL_NS, </span></span><span \
class="gmail-stx-string"><span \
style="font-size:13.5pt;color:#008800">"jvm-gc-cms-old-gen-directalloc-bytes"</span></span><span \
class="gmail-stx-line"><span \
style="font-size:13.5pt;color:black">,<o:p></o:p></span></span></pre> <pre><span \
class="gmail-stx-line"><span style="font-size:13.5pt;color:black"> \
PerfData::U_None, CHECK);<o:p></o:p></span></span></pre>
<pre><span class="gmail-stx-line"><span \
style="font-size:13.5pt;color:black"> _og_alloc_bytes \
=<o:p></o:p></span></span></pre> <pre><span class="gmail-stx-line"><span \
style="font-size:13.5pt;color:black"> \
PerfDataManager::create_variable(NULL_NS, </span></span><span \
class="gmail-stx-string"><span \
style="font-size:13.5pt;color:#008800">"jvm-gc-cms-old-gen-alloc-bytes"</span></span><span \
class="gmail-stx-line"><span \
style="font-size:13.5pt;color:black">,<o:p></o:p></span></span></pre> <pre><span \
class="gmail-stx-line"><span style="font-size:13.5pt;color:black"> \
PerfData::U_None, CHECK);<o:p></o:p></span></span></pre>
<pre><span class="gmail-stx-line"><span \
style="font-size:13.5pt;color:black"> _total_alloc_bytes \
=<o:p></o:p></span></span></pre> <pre><span class="gmail-stx-line"><span \
style="font-size:13.5pt;color:black"> \
PerfDataManager::create_variable(NULL_NS, </span></span><span \
class="gmail-stx-string"><span \
style="font-size:13.5pt;color:#008800">"jvm-gc-cms-total-alloc-bytes"</span></span><span \
class="gmail-stx-line"><span \
style="font-size:13.5pt;color:black">,<o:p></o:p></span></span></pre> <pre><span \
class="gmail-stx-line"><span style="font-size:13.5pt;color:black"> \
PerfData::U_None, CHECK);</span></span><span \
style="font-size:13.5pt;color:black"><o:p></o:p></span></pre> </div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">2) We also implemented an API built on
JFR's handling of GC events. We use it to export the GC
history to monitoring tools in a format that doesn't
involve parsing logs that can change.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">(Prior to JFR, we had a bespoke
implementation where we instrumented everywhere a log
happened, but we fortunately don't have to do that
anymore.)<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">We'd love both of these things to be
something that was more generally usable.<o:p></o:p></p>
</div>
<div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">Jeremy<o:p></o:p></p>
</div>
</div>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<p class="MsoNormal">On Mon, Oct 23, 2017 at 9:31 AM,
Hohensee, Paul <<a href="mailto:hohensee@amazon.com"
target="_blank" moz-do-not-send="true">hohensee@amazon.com</a>>
wrote:<o:p></o:p></p>
<blockquote style="border:none;border-left:solid #CCCCCC
1.0pt;padding:0in 0in 0in
6.0pt;margin-left:4.8pt;margin-right:0in">
<div>
<div>
<p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">I'd
like to solicit opinions on reporting GC pause
duration (stopped-world pause time) via JMX. This
info would be useful in figuring out whether or not
GC pause times are factors in failing to meet
response time SLAs. The info is of course available
directly from GC logs, but parsing logs is fraught
and JMX doesn't seem to report the equivalent \
info.<o:p></o:p></p> <p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> \
<o:p></o:p></p> <p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">GcInfo<o:p></o:p></p>
<p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> \
<o:p></o:p></p> <p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><a
href="https://docs.oracle.com/javase/9/docs/api/com/sun/management/GcInfo.html"
target="_blank" \
moz-do-not-send="true">https://docs.oracle.com/javase/9/docs/api/com/sun/management/GcInfo.html</a><o:p></o:p></p>
<p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> \
<o:p></o:p></p> <p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">has
a getDuration() method which works fine for the
non-concurrent collectors (since they're STW), but
for CMS and G1 it appears to report the duration of
an entire concurrent cycle, which isn't what I want.
The number of STW pauses during a concurrent cycle
varies by collector, so ideally there would be a
method that reports cause (as a string) and duration
for each STW pause. If that's too much, perhaps the
minimum might be a getMaxPauseDuration() method that
reports the maximum pause duration of all the STW
pauses that happen during a concurrent cycle.<o:p></o:p></p>
<p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> \
<o:p></o:p></p> <p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">Relatedly,
the full compacting GCs that happen as a result of
CMS and G1 concurrent mode failure aren't reported
separately from concurrent cycles. It would be
useful to differentiate these from
"ConcurrentMarkSweep" and "G1 Old Generation".
Perhaps add collector types to CMS and G1, vis.
"MarkSweepCompact" (which already exists and is
literally what's executed by CMS) and a new "G1
MarkSweepCompact" collector for G1.<o:p></o:p></p>
<p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> \
<o:p></o:p></p> <p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">If
there's a consensus that something should be done
about either of these issues, I'd be happy to file
RFE(s) and do the work.<o:p></o:p></p>
<p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> \
<o:p></o:p></p> <p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">Thanks,<o:p></o:p></p>
<p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span
style="color:#888888"> <o:p></o:p></span></p>
<p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span
style="color:#888888">Paul<o:p></o:p></span></p>
<p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span
style="color:#888888"> <o:p></o:p></span></p>
</div>
</div>
</blockquote>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
</div>
</blockquote>
<p><br>
</p>
</body>
</html>
[prev in list] [next in list] [prev in thread] [next in thread]
Configure |
About |
News |
Add a list |
Sponsored by KoreLogic