[prev in list] [next in list] [prev in thread] [next in thread] 

List:       hadoop-dev
Subject:    [jira] [Resolved] (HADOOP-6858) Enable rotateable JVM garbage collection logs for Hadoop daemons
From:       "Allen Wittenauer (JIRA)" <jira () apache ! org>
Date:       2015-01-27 19:07:37
Message-ID: JIRA.12469196.1279055259000.186876.1422385657663 () Atlassian ! JIRA
[Download RAW message or body]


     [ https://issues.apache.org/jira/browse/HADOOP-6858?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel \
]

Allen Wittenauer resolved HADOOP-6858.
--------------------------------------
    Resolution: Later

Closing this as 'Later'.

patch definitely won't apply in modern versions.  plus there is now enough shell \
infrastructure in place to do this sort of thing without changing the base shell \
code.

> Enable rotateable JVM garbage collection logs for Hadoop daemons
> ----------------------------------------------------------------
> 
> Key: HADOOP-6858
> URL: https://issues.apache.org/jira/browse/HADOOP-6858
> Project: Hadoop Common
> Issue Type: New Feature
> Components: scripts
> Reporter: Andrew Ryan
> Attachments: HADOOP-6858.patch
> 
> 
> The purpose of this enhancement is to make it easier to collect garbage collection \
> logs and insure that they persist across restarts in the same way that the standard \
> output files of Hadoop daemon JVM's currently does. Garbage collection logs are a \
> vital debugging tool for administrators and developers. In our production \
> environments, at some point or another, every single type of Hadoop daemon has \
> OOM'ed or experienced other significant issues related to GC and/or lack of heap \
> memory. For the longest time, we have put in garbage collection logs in our \
> HADOOP_NAMENODE_OPTS, HADOOP_JOBTRACKER_OPTS, etc. by using options like \
> "-XX:+PrintGCDateStamps -XX:+PrintGCDetails \
> -Xloggc:$HADOOP_LOG_DIR/jobtracker.gc.log". Unfortunately, these logs don't survive \
> a restart of the node, so if a node OOM's and then is restarted automatically, or \
> manually by someone who is unaware, we lose the GC logs forever. We also have to \
> manually add GC log options to each daemon. This patch: 1) Creates a single, \
> optional, off by default, parameter for specifying GC logging. 2) If that parameter \
> is set, automatically enables GC logging for all daemons in the cluster. The \
> parameter is flexible enough to allow for the different ways various vendor's JVM's \
> require garbage collection logging to be specified.  3) If GC logging is on, \
> insures that the GC log files for each daemon are rotated with up to 5 copies kept, \
> same as the .out files currently. We are currently running a variation of this \
> patch in our 0.20 install. This patch actually includes changes to common, mapred, \
> and hdfs, so it obviously cannot be applied as-is, but is included here for review \
> and comments.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[prev in list] [next in list] [prev in thread] [next in thread]