[prev in list] [next in list] [prev in thread] [next in thread] 

List:       openjdk-serviceability-dev
Subject:    Re: RFR: JDK-8306441: Two-stage Segmented Heap Dump [v14]
From:       Alex Menkov <amenkov () openjdk ! org>
Date:       2023-06-30 22:25:55
Message-ID: RtMypSvwWhPRnCNctiIzKsrLdZ8ZWvqhWudrc8S_65o=.badd5d5f-0112-4134-9c99-de047226e276 () github ! com
[Download RAW message or body]

On Wed, 28 Jun 2023 08:04:21 GMT, Yi Yang <yyang@openjdk.org> wrote:

> > ### Motivation and proposal
> > Hi, heap dump brings about pauses for application's execution(STW), this is a \
> > well-known pain. JDK-8252842 have added parallel support to heapdump in an \
> > attempt to alleviate this issue. However, all concurrent threads competitively \
> > write heap data to the same file, and more memory is required to maintain the \
> > concurrent buffer queue. In experiments, we did not feel a significant \
> > performance improvement from that. 
> > The minor-pause solution, which is presented in this PR, is a two-stage segmented \
> > heap dump: 
> > 1. Stage One(STW): Concurrent threads directly write data to multiple heap files.
> > 2. Stage Two(Non-STW): Merge multiple heap files into one complete heap dump \
> > file. 
> > Now concurrent worker threads are not required to maintain a buffer queue, which \
> > would result in more memory overhead, nor do they need to compete for locks. The \
> > changes in the overall design are as follows: 
> > ![image](https://github.com/openjdk/jdk/assets/5010047/86d2717d-c621-446f-98c2-cce761ec8db5)
> >  <p align="center">Fig1. Before</p>
> > 
> > ![image](https://github.com/openjdk/jdk/assets/5010047/c912c516-83b6-4343-a2e8-5d5bdb99cb87)
> >  <p align="center">Fig2. After this patch</p>
> > 
> > ### Performance evaluation
> > > memory | numOfThread | STW         | Total      | Compression |
> > > --- | --------- | -------------- | ------------ | -------- |
> > > 8g | 1 thread | 15.612 secs | 15.612 secs | N |
> > > 8g | 32 thread |  2.5617250 secs | 14.498 secs | N |
> > > 8g | 32 thread | 2.3084878 secs | 3.198 secs | Compress1 |
> > > 8g | 32 thread | 10.9355128 secs | 21.882 secs | Compress2 |
> > > 8g | 96 thread | 2.6790452 secs | 14.012 secs | N |
> > > 8g | 96 thread | 2.3044796 secs | 3.589 secs | Compress1 |
> > > 8g | 96 thread | 9.7585151 secs | 20.219 secs| Compress2 |
> > > 16g | 1 thread | 26.278 secs | 26.278 secs | N |
> > > 16g | 32 thread |  5.2313740 secs | 26.417 secs | N |
> > > 16g | 32 thread | 5.6946983 secs | 6.538 secs | Compress1 |
> > > 16g | 32 thread | 21.8211105 secs | 41.133 secs | Compress2 |
> > > 16g | 96 thread | 6.2445556 secs | 27.141 secs | N |
> > > 16g | 96 thread | 4.6007096 secs | 6.259 secs | Compress1 |
> > > 16g | 96 thread | 19.2965783 secs |  39.007 secs | Compress2 |
> > > 32g | 1 thread | 48.149 secs | 48.149 secs | N |
> > > 32g | 32 thread | 10.7734677 secs | 61.643 secs | N |
> > > 32g | 32 thread | 10.1642097 secs | 10.903 secs | Compress1 |
> > > 32g | 32 thread | 43.8407607 secs | 88.152 secs | Compress2 |
> > > 32g | 96 thread | 13.1522042 ...
> 
> Yi Yang has updated the pull request incrementally with one additional commit since \
> the last revision: 
> memory leak

I think commit "use HandshakeClosure instead of VMOperation" is a wrong way to go.
It restricts use of parallel dumping only to attach case.
I have pending changes in heap dumper to support virtual threads and I'm going switch \
heap dumper to always use DumpMerger.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/13667#issuecomment-1615243585


[prev in list] [next in list] [prev in thread] [next in thread]