[prev in list] [next in list] [prev in thread] [next in thread]
List: fop-user
Subject: AW: Forcefully stop org.apache.xalan.transformer.TransformerImpl.transform()
From: "Normen Ruhrus" <n.ruhrus () alphasoft ! biz>
Date: 2017-12-14 6:44:44
Message-ID: 001101d374a7$06d10b40$147321c0$ () alphasoft ! biz
[Download RAW message or body]
Hey Alexios,
thanks a lot for your help. I read some time yesterday and also came to the
conclusion to use the ProcessBuilder; i just wanted to test how much time it
will take to spawn a new process for every transformation and use the Object
In/Out Streams for handling the communication.
But of course you may be right that there is need for a permanent process
(or more than one depending on the hardware) and communicate with it.
I was not aware of the capabilities to create Heap Dumps; i will check it
out, too.
Best Regards and have a nice christmas time
Normen
----------------------------------------------------------------------------
---------------
Alphasoft Software Dienstleistungs- und Datenverarbeitungs GmbH
Prinz-Regent-Str. 68a, 44795 Bochum
fon: +49(0) 234 609 37 118
e-mail: <mailto:info@alphasoft.biz> info@alphasoft.biz
Amtsgericht - Registergericht - Bochum
HRB 1183, Sitz Bochum
USt.ID-Nr.: DE127057789
Geschäftsführer: Jens Plasa, Normen Ruhrus, Axel Freimuth
Von: Alexios Giotis [mailto:alex.giotis@gmail.com]
Gesendet: Donnerstag, 14. Dezember 2017 00:10
An: fop-users@xmlgraphics.apache.org
Betreff: Re: Forcefully stop
org.apache.xalan.transformer.TransformerImpl.transform()
To avoid waiting until the memory usage gets high, start the JVM with the
following options which will write the heap dump when an out of memory error
occurs. I use them in production systems, there is no overhead.
-XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/diskWithFreeSpace/dumps
Note that the heap dump size will be equal to the memory size (12GB in your
case).
On 14 Dec 2017, at 00:53, Alexios Giotis <alex.giotis@gmail.com
<mailto:alex.giotis@gmail.com> > wrote:
Hi Normen,
The Java Thread Model does not support forcefully stopping a thread
executing arbitrary code. Thread.stop() is deprecated as it unlocks all the
monitors it has locked and other threads may view an inconsistent state of
protected objects. Thread.interrupt() does not guarantee when or if the
thread will be interrupted. There are some tricks what work in certain cases
but not in general.
To implement reliably such functionality, you may spawn a new process (e.g.
Process p = processBuilder.start()) and keep a reference to the Process p.
The new process will execute the transformation using Xalan or any other
code. When needed, you may kill it (p.destroy()).
But this is not enough. You will need to choose a way for interprocess
communication (sockets ? files ? or better some library) and implement it.
It is much easier and better to find what is causing the memory leak. There
are plenty of tools in Java to help with this task. Xalan and FOP work quite
well and I am not aware of any memory leak. I suggest to let the application
run for some time and when the memory usage is getting high, connect to it
using Java Mission Control or jVisualVM or even JConsole or jmap to take a
heap dump. Then analyze the heap dump to find out where memory is retained.
If the heap dump is several GBs, I suggest to use Eclipse MAT which works
with huge heap dumps.
HTH,
Alexios Giotis
On 13 Dec 2017, at 14:52, Normen Ruhrus <n.ruhrus@alphasoft.biz
<mailto:n.ruhrus@alphasoft.biz> > wrote:
Hey there,
using FOP for quite some time now, thanks everyone for all the fine
development.
Iam running FOP 2.0 embedded into a Servlet on Tomcat 8.0.x with Java8 on a
Windows Machine (server 2012 R2).
Usually we have no problems but from time to time during the transformation
org.apache.xalan.transformer.TransformerImpl.transform() process an Out of
Memory Error occurs.
When trying to recreate the problem with the same XML and same XSL we have
no problems; the process takes a very short time (<100ms) and uses not much
memory (<50MB).
The application is configured so it can cosume up to 12GB(!) of RAM so it
looks like some indefinite looping is going on, but unfortunately i have no
further logs at this time to help me with the investigation.
This is why i wanted to put the transformation into a seperate Thread and
supervise it and kill it off if a certain timeout is reached, however iam
not able to.
So the question is can you think of any way to stop this processing? I have
implemented the multithreading here with FutureTasks but can switch to just
Threads easily, too. It would be better to kill off the Thread before it
throws an Out of Memory error and with this affecting the whole Servlet so
that subsequent requests cannot be processed anymore.
Naturally it would be best to find the source of the problem, of course, but
that can take me some time and i would like to have a fallback solution
instead with forcefully stopping the Transformation.
Hope someone is able to help me out with this; thanks in advance and best
Greetings
Normen
[Attachment #3 (text/html)]
<html xmlns:v="urn:schemas-microsoft-com:vml" \
xmlns:o="urn:schemas-microsoft-com:office:office" \
xmlns:w="urn:schemas-microsoft-com:office:word" \
xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" \
xmlns="http://www.w3.org/TR/REC-html40"><head><meta http-equiv=Content-Type \
content="text/html; charset=iso-8859-1"><meta name=Generator content="Microsoft Word \
15 (filtered medium)"><style><!-- /* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Consolas;
panose-1:2 11 6 9 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
p.msonormal0, li.msonormal0, div.msonormal0
{mso-style-name:msonormal;
mso-margin-top-alt:auto;
margin-right:0cm;
mso-margin-bottom-alt:auto;
margin-left:0cm;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
span.apple-converted-space
{mso-style-name:apple-converted-space;}
span.E-MailFormatvorlage19
{mso-style-type:personal-reply;
font-family:"Calibri",sans-serif;
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:612.0pt 792.0pt;
margin:70.85pt 70.85pt 2.0cm 70.85pt;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]--></head><body lang=DE link=blue vlink=purple><div \
class=WordSection1><p class=MsoNormal><a name="_MailEndCompose"><span \
style='mso-fareast-language:EN-US'>Hey Alexios,<o:p></o:p></span></a></p><p \
class=MsoNormal><span style='mso-bookmark:_MailEndCompose'><span \
style='mso-fareast-language:EN-US'><o:p> </o:p></span></span></p><p \
class=MsoNormal><span style='mso-bookmark:_MailEndCompose'><span \
style='mso-fareast-language:EN-US'>thanks a lot for your help. I read some time \
yesterday and also came to the conclusion to use the ProcessBuilder; i just wanted to \
test how much time it will take to spawn a new process for every transformation and \
use the Object In/Out Streams for handling the \
communication.<o:p></o:p></span></span></p><p class=MsoNormal><span \
style='mso-bookmark:_MailEndCompose'><span \
style='mso-fareast-language:EN-US'><o:p> </o:p></span></span></p><p \
class=MsoNormal><span style='mso-bookmark:_MailEndCompose'><span \
style='mso-fareast-language:EN-US'>But of course you may be right that there is need \
for a permanent process (or more than one depending on the hardware) and communicate \
with it.<o:p></o:p></span></span></p><p class=MsoNormal><span \
style='mso-bookmark:_MailEndCompose'><span \
style='mso-fareast-language:EN-US'><o:p> </o:p></span></span></p><p \
class=MsoNormal><span style='mso-bookmark:_MailEndCompose'><span \
style='mso-fareast-language:EN-US'>I was not aware of the capabilities to create Heap \
Dumps; i will check it out, too.<o:p></o:p></span></span></p><p class=MsoNormal><span \
style='mso-bookmark:_MailEndCompose'><span \
style='mso-fareast-language:EN-US'><o:p> </o:p></span></span></p><p \
class=MsoNormal><span style='mso-bookmark:_MailEndCompose'><span \
style='mso-fareast-language:EN-US'>Best Regards and have a nice christmas \
time<o:p></o:p></span></span></p><p class=MsoNormal><span \
style='mso-bookmark:_MailEndCompose'><span \
style='mso-fareast-language:EN-US'><o:p> </o:p></span></span></p><p \
class=MsoNormal><span style='mso-bookmark:_MailEndCompose'><span \
style='mso-fareast-language:EN-US'>Normen<o:p></o:p></span></span></p><p \
class=MsoNormal><span style='mso-bookmark:_MailEndCompose'><span \
style='mso-fareast-language:EN-US'><o:p> </o:p></span></span></p><p \
class=MsoNormal><span style='mso-bookmark:_MailEndCompose'><span \
style='mso-fareast-language:EN-US'><o:p> </o:p></span></span></p><div><p \
class=MsoNormal><span \
style='mso-bookmark:_MailEndCompose'>-------------------------------------------------------------------------------------------<o:p></o:p></span></p><p \
class=MsoNormal><span style='mso-bookmark:_MailEndCompose'>Alphasoft Software \
Dienstleistungs- und Datenverarbeitungs GmbH<o:p></o:p></span></p><p \
class=MsoNormal><span \
style='mso-bookmark:_MailEndCompose'><o:p> </o:p></span></p><p \
class=MsoNormal><span style='mso-bookmark:_MailEndCompose'>Prinz-Regent-Str. 68a, \
44795 Bochum<o:p></o:p></span></p><p class=MsoNormal><span \
style='mso-bookmark:_MailEndCompose'>fon: +49(0) 234 609 37 \
118<o:p></o:p></span></p><p class=MsoNormal><span \
style='mso-bookmark:_MailEndCompose'>e-mail: </span><a \
href="mailto:info@alphasoft.biz"><span style='mso-bookmark:_MailEndCompose'><span \
style='color:#0563C1'>info@alphasoft.biz</span></span><span \
style='mso-bookmark:_MailEndCompose'></span></a><span \
style='mso-bookmark:_MailEndCompose'><o:p></o:p></span></p><p class=MsoNormal><span \
style='mso-bookmark:_MailEndCompose'><o:p> </o:p></span></p><p \
class=MsoNormal><span style='mso-bookmark:_MailEndCompose'>Amtsgericht - \
Registergericht - Bochum<o:p></o:p></span></p><p class=MsoNormal><span \
style='mso-bookmark:_MailEndCompose'>HRB 1183, Sitz Bochum<o:p></o:p></span></p><p \
class=MsoNormal><span style='mso-bookmark:_MailEndCompose'>USt.ID-Nr.: \
DE127057789<o:p></o:p></span></p><p class=MsoNormal><span \
style='mso-bookmark:_MailEndCompose'>Geschäftsführer: Jens Plasa, Normen Ruhrus, Axel \
Freimuth<o:p></o:p></span></p></div><p class=MsoNormal><span \
style='mso-bookmark:_MailEndCompose'><span \
style='mso-fareast-language:EN-US'><o:p> </o:p></span></span></p><span \
style='mso-bookmark:_MailEndCompose'></span><div><div \
style='border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0cm 0cm 0cm'><p \
class=MsoNormal><b>Von:</b> Alexios Giotis [mailto:alex.giotis@gmail.com] \
<br><b>Gesendet:</b> Donnerstag, 14. Dezember 2017 00:10<br><b>An:</b> \
fop-users@xmlgraphics.apache.org<br><b>Betreff:</b> Re: Forcefully stop \
org.apache.xalan.transformer.TransformerImpl.transform()<o:p></o:p></p></div></div><p \
class=MsoNormal><o:p> </o:p></p><p class=MsoNormal>To avoid waiting until the \
memory usage gets high, start the JVM with the following options which will write the \
heap dump when an out of memory error occurs. I use them in production systems, there \
is no overhead.<o:p></o:p></p><div><p \
class=MsoNormal><o:p> </o:p></p></div><div><p class=MsoNormal><span \
style='font-size:10.0pt;font-family:Consolas;color:#242729;background:#EFF0F1'>-XX:+HeapDumpOnOutOfMemoryError \
-XX:HeapDumpPath=/diskWithFreeSpace/dumps</span><o:p></o:p></p></div><div><p \
class=MsoNormal><span \
style='font-size:10.0pt;font-family:Consolas;color:#242729;background:#EFF0F1'><br><br></span><o:p></o:p></p></div><div><p \
class=MsoNormal>Note that the heap dump size will be equal to the memory size (12GB \
in your case).<o:p></o:p></p></div><div><p \
class=MsoNormal><o:p> </o:p></p><div><p \
class=MsoNormal><br><br><o:p></o:p></p><blockquote \
style='margin-top:5.0pt;margin-bottom:5.0pt'><div><p class=MsoNormal>On 14 Dec 2017, \
at 00:53, Alexios Giotis <<a \
href="mailto:alex.giotis@gmail.com">alex.giotis@gmail.com</a>> \
wrote:<o:p></o:p></p></div><p class=MsoNormal><o:p> </o:p></p><div><div><p \
class=MsoNormal>Hi Normen,<o:p></o:p></p><div><p \
class=MsoNormal><o:p> </o:p></p></div><div><p class=MsoNormal>The Java Thread \
Model does not support forcefully stopping a thread executing arbitrary code. \
Thread.stop() is deprecated as it unlocks all the monitors it has locked and other \
threads may view an inconsistent state of protected objects. Thread.interrupt() does \
not guarantee when or if the thread will be interrupted. There are some tricks what \
work in certain cases but not in general.<o:p></o:p></p></div><div><p \
class=MsoNormal><o:p> </o:p></p></div><div><p class=MsoNormal>To implement \
reliably such functionality, you may spawn a new process (e.g. Process p = \
processBuilder.start()) and keep a reference to the Process p. The new process will \
execute the transformation using Xalan or any other code. When needed, you may kill \
it (p.destroy()).<o:p></o:p></p></div><div><p \
class=MsoNormal><o:p> </o:p></p></div><div><p class=MsoNormal>But this is not \
enough. You will need to choose a way for interprocess communication (sockets ? files \
? or better some library) and implement it. <o:p></o:p></p></div><div><p \
class=MsoNormal><o:p> </o:p></p></div><div><p class=MsoNormal>It is much easier \
and better to find what is causing the memory leak. There are plenty of tools in Java \
to help with this task. Xalan and FOP work quite well and I am not aware of any \
memory leak. I suggest to let the application run for some time and when the memory \
usage is getting high, connect to it using Java Mission Control or jVisualVM or even \
JConsole or jmap to take a heap dump. Then analyze the heap dump to find out where \
memory is retained. If the heap dump is several GBs, I suggest to use Eclipse MAT \
which works with huge heap dumps. <o:p></o:p></p></div><div><p \
class=MsoNormal><o:p> </o:p></p></div><div><p \
class=MsoNormal><o:p> </o:p></p></div><div><p \
class=MsoNormal>HTH,<o:p></o:p></p></div><div><p class=MsoNormal>Alexios \
Giotis<o:p></o:p></p></div><div><p class=MsoNormal><o:p> </o:p></p></div><div><p \
class=MsoNormal><o:p> </o:p></p></div><div><p \
class=MsoNormal><o:p> </o:p></p></div><div><p \
class=MsoNormal><o:p> </o:p></p></div><div><p \
class=MsoNormal><o:p> </o:p></p></div><div><p \
class=MsoNormal><o:p> </o:p></p><div><p \
class=MsoNormal><br><br><o:p></o:p></p><blockquote \
style='margin-top:5.0pt;margin-bottom:5.0pt'><div><p class=MsoNormal>On 13 Dec 2017, \
at 14:52, Normen Ruhrus <<a \
href="mailto:n.ruhrus@alphasoft.biz">n.ruhrus@alphasoft.biz</a>> \
wrote:<o:p></o:p></p></div><p class=MsoNormal><o:p> </o:p></p><div><div><p \
class=MsoNormal>Hey there,<o:p></o:p></p></div><div><p \
class=MsoNormal> <o:p></o:p></p></div><div><p class=MsoNormal>using FOP for \
quite some time now, thanks everyone for all the fine \
development.<o:p></o:p></p></div><div><p \
class=MsoNormal> <o:p></o:p></p></div><div><p class=MsoNormal>Iam running FOP \
2.0 embedded into a Servlet on Tomcat 8.0.x with Java8 on a Windows Machine (server \
2012 R2).<o:p></o:p></p></div><div><p class=MsoNormal>Usually we have no problems but \
from time to time during the transformation \
org.apache.xalan.transformer.TransformerImpl.transform() process an Out of Memory \
Error occurs.<o:p></o:p></p></div><div><p \
class=MsoNormal> <o:p></o:p></p></div><div><p class=MsoNormal>When trying to \
recreate the problem with the same XML and same XSL we have no problems; the process \
takes a very short time (<100ms) and uses not much memory \
(<50MB).<o:p></o:p></p></div><div><p class=MsoNormal>The application is configured \
so it can cosume up to 12GB(!) of RAM so it looks like some indefinite looping is \
going on, but unfortunately i have no further logs at this time to help me with \
the investigation.<o:p></o:p></p></div><div><p \
class=MsoNormal> <o:p></o:p></p></div><div><p class=MsoNormal>This is why i \
wanted to put the transformation into a seperate Thread and supervise it and kill it \
off if a certain timeout is reached, however iam not able to.<span \
class=apple-converted-space> </span><o:p></o:p></p></div><div><p \
class=MsoNormal>So the question is can you think of any way to stop this processing? \
I have implemented the multithreading here with FutureTasks but can switch to just \
Threads easily, too. It would be better to kill off the Thread before it throws an \
Out of Memory error and with this affecting the whole Servlet so that \
subsequent requests cannot be processed anymore.<o:p></o:p></p></div><div><p \
class=MsoNormal> <o:p></o:p></p></div><div><p class=MsoNormal>Naturally it would \
be best to find the source of the problem, of course, but that can take me some time \
and i would like to have a fallback solution instead with forcefully stopping the \
Transformation.<o:p></o:p></p></div><div><p class=MsoNormal>Hope someone is able to \
help me out with this; thanks in advance and best \
Greetings<o:p></o:p></p></div><div><p \
class=MsoNormal> <o:p></o:p></p></div><div><p \
class=MsoNormal>Normen<o:p></o:p></p></div></div></blockquote></div><p \
class=MsoNormal><o:p> </o:p></p></div></div></div></blockquote></div><p \
class=MsoNormal><o:p> </o:p></p></div></div></body></html>
[prev in list] [next in list] [prev in thread] [next in thread]
Configure |
About |
News |
Add a list |
Sponsored by KoreLogic