[prev in list] [next in list] [prev in thread] [next in thread]
List: htmlunit-develop
Subject: [HtmlUnit] [htmlunit:bugs] #1679 Memory Leak because of ThreadLocal in HttpWebConnection
From: "RBRi" <rbri () users ! sf ! net>
Date: 2015-04-22 16:37:40
Message-ID: /p/htmlunit/bugs/1679/1d9692b49f509e055aef6256d971ae68633e0ddb.bugs () htmlunit ! p ! sourceforge ! net
[Download RAW message or body]
[Attachment #2 (multipart/related)]
--===============7428511620715425453==
MIME-Version: 1.0
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: 7bit
- **status**: open --> accepted
- **assigned_to**: RBRi
---
** [bugs:#1679] Memory Leak because of ThreadLocal in HttpWebConnection**
**Status:** accepted
**Group:** Latest SVN
**Created:** Wed Apr 22, 2015 02:49 PM UTC by Carsten
**Last Updated:** Wed Apr 22, 2015 02:49 PM UTC
**Owner:** RBRi
If you have long running threads, for example a thread pool, with more than one \
thread using a single WebClient instance it will not be garbage collected because the \
ThreadLocalMap of every thread using it has an indirect reference to it.
For testing purposes I reverted the changes from \
http://sourceforge.net/p/htmlunit/code/10314/ and the memory leak disappeared. A \
better solution would be to use RequestBuilder in makeHttpMethod() instead of \
reconfiguring the same HttpClientBuilder and building a new HttpClient for every \
request.
Below is an export from Eclipse Memory Analyzer of the heap from the attached test.
~~~~~~~~~~~~~~~~
Class Name \
| Shallow Heap | Retained Heap
-------------------------------------------------------------------------------------- \
------------------------------------------------------------------------------------------------------------------
| |
com.gargoylesoftware.htmlunit.WebClient @ 0x64a67de80 \
| 144 | 22.592 '- webClient_ \
com.gargoylesoftware.htmlunit.HttpWebConnection @ 0x64a67de48 \
| 40 | 25.528 |- this$0 \
com.gargoylesoftware.htmlunit.HttpWebConnection$3 @ 0x64a6aa6d8 \
| 24 | 24 | '- redirectStrategy \
org.apache.http.impl.client.HttpClientBuilder @ 0x64a663d48 \
| 200 | 1.048 | '- value \
java.lang.ThreadLocal$ThreadLocalMap$Entry @ 0x64a663d28 \
| 32 | 1.080 | '- [5] \
java.lang.ThreadLocal$ThreadLocalMap$Entry[16] @ 0x64a6623a0 \
| 80 | 3.288 | '- table \
java.lang.ThreadLocal$ThreadLocalMap @ 0x64a662388 \
| 24 | 3.312 | '- threadLocals \
com.gargoylesoftware.htmlunit.HttpWebConnectionMemoryLeakTest$RequestHandler @ \
0x64a610588 Thread-0 Busy Monitor, Native Stack, Thread| 128 | \
3.664 |- this$0 com.gargoylesoftware.htmlunit.HttpWebConnection$3 @ 0x64a6ab9d8 \
| 24 | 24 | '- redirectStrategy \
org.apache.http.impl.client.HttpClientBuilder @ 0x64a6ab208 \
| 200 | 2.328 | '- value \
java.lang.ThreadLocal$ThreadLocalMap$Entry @ 0x64a6ab1e8 \
| 32 | 2.360 | '- [5] \
java.lang.ThreadLocal$ThreadLocalMap$Entry[16] @ 0x64a6aa950 \
| 80 | 4.560 | '- table \
java.lang.ThreadLocal$ThreadLocalMap @ 0x64a6aa938 \
| 24 | 4.584 | '- threadLocals \
com.gargoylesoftware.htmlunit.HttpWebConnectionMemoryLeakTest$RequestHandler @ \
0x64a610408 Thread-1 Busy Monitor, Native Stack, Thread| 128 | \
4.832 |- this$0 com.gargoylesoftware.htmlunit.HttpWebConnection$3 @ 0x64a6c9818 \
| 24 | 24 | '- redirectStrategy \
org.apache.http.impl.client.HttpClientBuilder @ 0x64a6c9048 \
| 200 | 2.328 | '- value \
java.lang.ThreadLocal$ThreadLocalMap$Entry @ 0x64a6c9028 \
| 32 | 2.360 | '- [5] \
java.lang.ThreadLocal$ThreadLocalMap$Entry[16] @ 0x64a6c8e20 \
| 80 | 2.880 | '- table \
java.lang.ThreadLocal$ThreadLocalMap @ 0x64a6c8e08 \
| 24 | 2.904 | '- threadLocals \
com.gargoylesoftware.htmlunit.HttpWebConnectionMemoryLeakTest$RequestHandler @ \
0x64a618358 Thread-3 Busy Monitor, Native Stack, Thread| 128 | \
3.152 |- this$0 com.gargoylesoftware.htmlunit.HttpWebConnection$3 @ 0x64a723148 \
| 24 | 24 | '- redirectStrategy \
org.apache.http.impl.client.HttpClientBuilder @ 0x64a67cd78 \
| 200 | 2.328 | '- value \
java.lang.ThreadLocal$ThreadLocalMap$Entry @ 0x64a67cd58 \
| 32 | 2.360 | '- [5] \
java.lang.ThreadLocal$ThreadLocalMap$Entry[16] @ 0x64a67cb50 \
| 80 | 2.880 | '- table \
java.lang.ThreadLocal$ThreadLocalMap @ 0x64a67cb38 \
| 24 | 2.904 | '- threadLocals \
com.gargoylesoftware.htmlunit.HttpWebConnectionMemoryLeakTest$RequestHandler @ \
0x64a6189e8 Thread-4 Busy Monitor, Native Stack, Thread| 128 | \
3.152 |- this$0 com.gargoylesoftware.htmlunit.HttpWebConnection$3 @ 0x64a7bde40 \
| 24 | 24 | '- redirectStrategy \
org.apache.http.impl.client.HttpClientBuilder @ 0x64a7957b8 \
| 200 | 2.328 | '- value \
java.lang.ThreadLocal$ThreadLocalMap$Entry @ 0x64a795798 \
| 32 | 2.360 | '- [5] \
java.lang.ThreadLocal$ThreadLocalMap$Entry[16] @ 0x64a795590 \
| 80 | 2.880 | '- table \
java.lang.ThreadLocal$ThreadLocalMap @ 0x64a795578 \
| 24 | 2.904 | '- threadLocals \
com.gargoylesoftware.htmlunit.HttpWebConnectionMemoryLeakTest$RequestHandler @ \
0x64a608d80 Thread-2 Busy Monitor, Native Stack, Thread| 128 | \
3.152 '- Total: 5 entries \
| |
-------------------------------------------------------------------------------------- \
------------------------------------------------------------------------------------------------------------------
~~~~~~~~~~~~~~~
---
Sent from sourceforge.net because htmlunit-develop@lists.sourceforge.net is \
subscribed to https://sourceforge.net/p/htmlunit/bugs/
To unsubscribe from further messages, a project admin can change settings at \
https://sourceforge.net/p/htmlunit/admin/bugs/options. Or, if this is a mailing \
list, you can unsubscribe from the mailing list.
--===============7428511620715425453==
MIME-Version: 1.0
Content-Type: text/html; charset="us-ascii"
Content-Transfer-Encoding: 7bit
<div class="markdown_content"><ul>
<li><strong>status</strong>: open --> accepted</li>
<li><strong>assigned_to</strong>: RBRi</li>
</ul>
<hr />
<p><strong> <a class="alink" \
href="http://sourceforge.net/p/htmlunit/bugs/1679">[bugs:#1679]</a> Memory Leak \
because of ThreadLocal in HttpWebConnection</strong></p> <p><strong>Status:</strong> \
accepted<br /> <strong>Group:</strong> Latest SVN<br />
<strong>Created:</strong> Wed Apr 22, 2015 02:49 PM UTC by Carsten<br />
<strong>Last Updated:</strong> Wed Apr 22, 2015 02:49 PM UTC<br />
<strong>Owner:</strong> RBRi</p>
<p>If you have long running threads, for example a thread pool, with more than one \
thread using a single WebClient instance it will not be garbage collected because the \
ThreadLocalMap of every thread using it has an indirect reference to it.</p> <p>For \
testing purposes I reverted the changes from <a \
href="http://sourceforge.net/p/htmlunit/code/10314">http://sourceforge.net/p/htmlunit/code/10314/</a> \
and the memory leak disappeared. A better solution would be to use RequestBuilder in \
makeHttpMethod() instead of reconfiguring the same HttpClientBuilder and building a \
new HttpClient for every request. </p> <p>Below is an export from Eclipse Memory \
Analyzer of the heap from the attached test.</p> <div class="codehilite"><pre><span \
class="x">Class Name \
| Shallow Heap | Retained Heap</span> <span \
class="x">---------------------------------------------------------------------------- \
----------------------------------------------------------------------------------------------------------------------------</span>
<span class="x"> \
| | </span> <span \
class="x">com.gargoylesoftware.htmlunit.WebClient @ 0x64a67de80 \
| 144 | 22.592</span> <span class="x">'- webClient_ \
com.gargoylesoftware.htmlunit.HttpWebConnection @ 0x64a67de48 \
| 40 | 25.528</span> <span class="x"> |- this</span><span \
class="p">$</span><span class="x">0 \
com.gargoylesoftware.htmlunit.HttpWebConnection</span><span class="p">$</span><span \
class="x">3 @ 0x64a6aa6d8 \
| 24 | 24</span> <span class="x"> | '- redirectStrategy \
org.apache.http.impl.client.HttpClientBuilder @ 0x64a663d48 \
| 200 | 1.048</span> <span class="x"> | '- value \
java.lang.ThreadLocal</span><span class="p">$</span><span \
class="nv">ThreadLocalMap</span><span class="p">$</span><span \
class="nv">Entry</span><span class="x"> @ 0x64a663d28 \
| 32 | 1.080</span> <span class="x"> | '- [5] \
java.lang.ThreadLocal</span><span class="p">$</span><span \
class="nv">ThreadLocalMap</span><span class="p">$</span><span \
class="nv">Entry</span><span class="x">[16] @ 0x64a6623a0 \
| 80 | 3.288</span> <span class="x"> | '- table \
java.lang.ThreadLocal</span><span class="p">$</span><span \
class="nv">ThreadLocalMap</span><span class="x"> @ 0x64a662388 \
| 24 | 3.312</span> <span class="x"> | '- \
threadLocals com.gargoylesoftware.htmlunit.HttpWebConnectionMemoryLeakTest</span><span \
class="p">$</span><span class="nv">RequestHandler</span><span class="x"> @ \
0x64a610588 Thread-0 Busy Monitor, Native Stack, Thread| 128 | \
3.664</span> <span class="x"> |- this</span><span class="p">$</span><span \
class="x">0 com.gargoylesoftware.htmlunit.HttpWebConnection</span><span \
class="p">$</span><span class="x">3 @ 0x64a6ab9d8 \
| 24 | 24</span> <span class="x"> | '- redirectStrategy \
org.apache.http.impl.client.HttpClientBuilder @ 0x64a6ab208 \
| 200 | 2.328</span> <span class="x"> | '- value \
java.lang.ThreadLocal</span><span class="p">$</span><span \
class="nv">ThreadLocalMap</span><span class="p">$</span><span \
class="nv">Entry</span><span class="x"> @ 0x64a6ab1e8 \
| 32 | 2.360</span> <span class="x"> | '- [5] \
java.lang.ThreadLocal</span><span class="p">$</span><span \
class="nv">ThreadLocalMap</span><span class="p">$</span><span \
class="nv">Entry</span><span class="x">[16] @ 0x64a6aa950 \
| 80 | 4.560</span> <span class="x"> | '- table \
java.lang.ThreadLocal</span><span class="p">$</span><span \
class="nv">ThreadLocalMap</span><span class="x"> @ 0x64a6aa938 \
| 24 | 4.584</span> <span class="x"> | '- \
threadLocals com.gargoylesoftware.htmlunit.HttpWebConnectionMemoryLeakTest</span><span \
class="p">$</span><span class="nv">RequestHandler</span><span class="x"> @ \
0x64a610408 Thread-1 Busy Monitor, Native Stack, Thread| 128 | \
4.832</span> <span class="x"> |- this</span><span class="p">$</span><span \
class="x">0 com.gargoylesoftware.htmlunit.HttpWebConnection</span><span \
class="p">$</span><span class="x">3 @ 0x64a6c9818 \
| 24 | 24</span> <span class="x"> | '- redirectStrategy \
org.apache.http.impl.client.HttpClientBuilder @ 0x64a6c9048 \
| 200 | 2.328</span> <span class="x"> | '- value \
java.lang.ThreadLocal</span><span class="p">$</span><span \
class="nv">ThreadLocalMap</span><span class="p">$</span><span \
class="nv">Entry</span><span class="x"> @ 0x64a6c9028 \
| 32 | 2.360</span> <span class="x"> | '- [5] \
java.lang.ThreadLocal</span><span class="p">$</span><span \
class="nv">ThreadLocalMap</span><span class="p">$</span><span \
class="nv">Entry</span><span class="x">[16] @ 0x64a6c8e20 \
| 80 | 2.880</span> <span class="x"> | '- table \
java.lang.ThreadLocal</span><span class="p">$</span><span \
class="nv">ThreadLocalMap</span><span class="x"> @ 0x64a6c8e08 \
| 24 | 2.904</span> <span class="x"> | '- \
threadLocals com.gargoylesoftware.htmlunit.HttpWebConnectionMemoryLeakTest</span><span \
class="p">$</span><span class="nv">RequestHandler</span><span class="x"> @ \
0x64a618358 Thread-3 Busy Monitor, Native Stack, Thread| 128 | \
3.152</span> <span class="x"> |- this</span><span class="p">$</span><span \
class="x">0 com.gargoylesoftware.htmlunit.HttpWebConnection</span><span \
class="p">$</span><span class="x">3 @ 0x64a723148 \
| 24 | 24</span> <span class="x"> | '- redirectStrategy \
org.apache.http.impl.client.HttpClientBuilder @ 0x64a67cd78 \
| 200 | 2.328</span> <span class="x"> | '- value \
java.lang.ThreadLocal</span><span class="p">$</span><span \
class="nv">ThreadLocalMap</span><span class="p">$</span><span \
class="nv">Entry</span><span class="x"> @ 0x64a67cd58 \
| 32 | 2.360</span> <span class="x"> | '- [5] \
java.lang.ThreadLocal</span><span class="p">$</span><span \
class="nv">ThreadLocalMap</span><span class="p">$</span><span \
class="nv">Entry</span><span class="x">[16] @ 0x64a67cb50 \
| 80 | 2.880</span> <span class="x"> | '- table \
java.lang.ThreadLocal</span><span class="p">$</span><span \
class="nv">ThreadLocalMap</span><span class="x"> @ 0x64a67cb38 \
| 24 | 2.904</span> <span class="x"> | '- \
threadLocals com.gargoylesoftware.htmlunit.HttpWebConnectionMemoryLeakTest</span><span \
class="p">$</span><span class="nv">RequestHandler</span><span class="x"> @ \
0x64a6189e8 Thread-4 Busy Monitor, Native Stack, Thread| 128 | \
3.152</span> <span class="x"> |- this</span><span class="p">$</span><span \
class="x">0 com.gargoylesoftware.htmlunit.HttpWebConnection</span><span \
class="p">$</span><span class="x">3 @ 0x64a7bde40 \
| 24 | 24</span> <span class="x"> | '- redirectStrategy \
org.apache.http.impl.client.HttpClientBuilder @ 0x64a7957b8 \
| 200 | 2.328</span> <span class="x"> | '- value \
java.lang.ThreadLocal</span><span class="p">$</span><span \
class="nv">ThreadLocalMap</span><span class="p">$</span><span \
class="nv">Entry</span><span class="x"> @ 0x64a795798 \
| 32 | 2.360</span> <span class="x"> | '- [5] \
java.lang.ThreadLocal</span><span class="p">$</span><span \
class="nv">ThreadLocalMap</span><span class="p">$</span><span \
class="nv">Entry</span><span class="x">[16] @ 0x64a795590 \
| 80 | 2.880</span> <span class="x"> | '- table \
java.lang.ThreadLocal</span><span class="p">$</span><span \
class="nv">ThreadLocalMap</span><span class="x"> @ 0x64a795578 \
| 24 | 2.904</span> <span class="x"> | '- \
threadLocals com.gargoylesoftware.htmlunit.HttpWebConnectionMemoryLeakTest</span><span \
class="p">$</span><span class="nv">RequestHandler</span><span class="x"> @ \
0x64a608d80 Thread-2 Busy Monitor, Native Stack, Thread| 128 | \
3.152</span> <span class="x"> '- Total: 5 entries \
| | </span> <span \
class="x">---------------------------------------------------------------------------- \
----------------------------------------------------------------------------------------------------------------------------</span>
</pre></div>
<hr />
<p>Sent from sourceforge.net because htmlunit-develop@lists.sourceforge.net is \
subscribed to <a href="https://sourceforge.net/p/htmlunit/bugs">https://sourceforge.net/p/htmlunit/bugs/</a></p>
<p>To unsubscribe from further messages, a project admin can change settings at <a \
href="https://sourceforge.net/p/htmlunit/admin/bugs/options.">https://sourceforge.net/p/htmlunit/admin/bugs/options.</a> \
Or, if this is a mailing list, you can unsubscribe from the mailing \
list.</p></div>
--===============7428511620715425453==--
------------------------------------------------------------------------------
BPM Camp - Free Virtual Workshop May 6th at 10am PDT/1PM EDT
Develop your own process in accordance with the BPMN 2 standard
Learn Process modeling best practices with Bonita BPM through live exercises
http://www.bonitasoft.com/be-part-of-it/events/bpm-camp-virtual- event?utm_
source=Sourceforge_BPM_Camp_5_6_15&utm_medium=email&utm_campaign=VA_SF
_______________________________________________
HtmlUnit-develop mailing list
HtmlUnit-develop@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/htmlunit-develop
[prev in list] [next in list] [prev in thread] [next in thread]
Configure |
About |
News |
Add a list |
Sponsored by KoreLogic