[prev in list] [next in list] [prev in thread] [next in thread] 

List:       openjdk-hotspot-gc-use
Subject:    Re: Unreachable Memory not freed
From:       Thorsten Goetzke <tg () freigmbh ! de>
Date:       2017-01-30 15:00:51
Message-ID: 0e023b96-b5e6-9642-ee4b-205da144f11a () freigmbh ! de
[Download RAW message or body]

Hello,

I have an update. It seems like the unreported/errorous gc Root is 
somewhere near 
"securiton.uls.tools.configuration.plugins.graphic.zoneeditor.data.svg.dynamics.impl.LiveItemImpl". 

I aggressivly nulled out the fields of instances of this Object and now 
the gc frees sufficient memory. It seems like the LiveItemImpl Objects 
themself are still leaking without any paths from gc root, but they no 
longer keep millions of Nodes alive causing our Application to go 
OutOfmemory within minutes.
The Root Cause seems like something Nashorn is doing is confusing the 
JVM/GC to not free up memory in some specific circumstances.
The LiveItemImpl Objects have incoming references from the internals of 
the nashorn framework.

Best Regards,
Thorsten


Am 28.01.17 um 21:23 schrieb yu.zhang@oracle.com:
> Thorsten,
>
> I have updated the bug with more details. I think there is some
> inaccuracy when reporting the heap dump.
>
> There might be some memory leak in your application as well.
>
> Thanks
>
> Jenny
>
> On 01/27/2017 04:50 PM, Jenny Zhang wrote:
>> Hi, Thorsten,
>>
>> I can reproduce this with a micro. I have filed
>> https://bugs.openjdk.java.net/browse/JDK-8173594
>>
>> The micro is mostly based on
>> http://stackoverflow.com/questions/26027313/how-to-load-and-parse-svg-documents
>> with some modification. I will upload the micro and details later.
>>
>> I think this is a issue how jvmti and gc reporting unreachable
>> objects. Those objects should not be reported as unreachable.
>>
>> Thanks
>> Jenny
>>
>> On 1/25/2017 3:23 PM, yu.zhang@oracle.com wrote:
>>> Thorsten,
>>>
>>> Thanks very much for the heap dump and gc log.
>>>
>>> I assume the gc log and heap dump are from the same run. It is very
>>> interesting, in the gc log:
>>>
>>> 2017-01-24T16:41:17.184-0100: 1301.633: [Full GC (Heap Dump Initiated
>>> GC)  3817M->3816M(4096M), 8.9543444 secs]
>>>    [Eden: 2048.0K(2048.0M)->0.0B(2048.0M) Survivors: 0.0B->0.0B Heap:
>>> 3817.7M(4096.0M)->3816.2M(4096.0M)], [Metaspace:
>>> 142921K->142921K(1175552K)]
>>>  [Times: user=16.35 sys=0.11, real=8.95 secs]
>>>
>>> GC thinks there are 3816.2M of objects alive. But in the heap dump,
>>> there is only 1.8G alive.
>>>
>>> I can see those unreachable objects with yourkit, mostly are
>>>
>>> org.apache.batik.dom.GenerecText
>>>
>>> But apparently gc does not think those are dead. It could be a bug in
>>> gc, or jvmti dumping the heap, or tool(yourkit, apache MAT). I need
>>> to discuss this with dev team.
>>>
>>> Back to the gc log, it seems the application allocates a lot of
>>> humongous objects. I can see some int[] of 1.5m. In general the heap
>>> is very full, with a lot of to-space exhausted, and full gc. And the
>>> heap usage after full gc is almost the whole heap (4096M).
>>>
>>> Thanks
>>>
>>> Jenny
>>>
>>>
>>> On 01/25/2017 01:15 AM, Thorsten Goetzke wrote:
>>>> Hello,
>>>>
>>>> You can find the heap dump and gc logs under
>>>> https://filesender.freigmbh.de/filesender/?vid=10f1aeb6-82b4-3d09-c201-00002042c681
>>>>
>>>>
>>>> Link is valid until 28.01.2017 (1/28/2017). Feel free to message me
>>>> if you need a reupload.
>>>>
>>>> Best Regards,
>>>> Thorsten
>>>>
>>>>
>>>>> Thorsten,
>>>>>
>>>>> Can you please put the heap dump and gc logs somewhere for download?
>>>>>
>>>>> Thanks
>>>>> Jenny
>>>>>
>>>>> On 1/24/2017 8:30 AM, Thorsten Goetzke wrote:
>>>>>> Hello,
>>>>>>
>>>>>> I am working on a closed source software that uses a patched version
>>>>>> of apache-batik to display SVG-Documents one by one. If you keep
>>>>>> loading and discarding some specific SVG-Documents memory consumption
>>>>>> keeps on growing until it reaches more than 95% of the available
>>>>>> memory. The application becomes unusable slow, very little memory
>>>>>> gets
>>>>>> freed, although the gc is constantly running. Sometimes an
>>>>>> OutOfMemoryError gets thrown. I took memory snaphsots and analyzed
>>>>>> them using jvisualvm and yourkit. These tools report the majority of
>>>>>> the memory as unreachable.
>>>>>> I have tried different garbage collectors (G1,ConcurrentMarkSweep),
>>>>>> and different Versions of Java: 1.8_60;1.8_65;1.8_102, the issue is
>>>>>> always reproducible.
>>>>>> Is there something i can do to debug this further? I am currently not
>>>>>> able to reproduce this issue in a somewhat small self contained
>>>>>> example.
>>>>>> I could provide gc-logs and the heap dump (about 1gb zipped) or any
>>>>>> other debuging info if someone is interested.
>>>>>>
>>>>>> Best Regards,
>>>>>> Thorsten Goetzke
>>>>>>
>>>>
>>>> _______________________________________________
>>>> hotspot-gc-use mailing list
>>>> hotspot-gc-use@openjdk.java.net
>>>> http://mail.openjdk.java.net/mailman/listinfo/hotspot-gc-use
>>>
>>
>> _______________________________________________
>> hotspot-gc-use mailing list
>> hotspot-gc-use@openjdk.java.net
>> http://mail.openjdk.java.net/mailman/listinfo/hotspot-gc-use
>
_______________________________________________
hotspot-gc-use mailing list
hotspot-gc-use@openjdk.java.net
http://mail.openjdk.java.net/mailman/listinfo/hotspot-gc-use
[prev in list] [next in list] [prev in thread] [next in thread]