[prev in list] [next in list] [prev in thread] [next in thread]
List: activemq-users
Subject: Re: Ativemq instance will not shutdown gracefully.
From: Chirag Pujara <chiragpujara () gmail ! com>
Date: 2013-07-25 21:48:09
Message-ID: CAKineiJ33Py7fZ1dY5p1KAq1zuN+-F70NGrwTYMjW_rigCAKAg () mail ! gmail ! com
[Download RAW message or body]
I think I figured out issue. It was user problem. I started process with
diff. user and was trying to shutdown as root user.
thanks,
chirag
On Thu, Jul 25, 2013 at 2:50 PM, Chirag Pujara <chiragpujara@gmail.com>wrote:
> Hello,
>
> when I try to shutdown activemq instance it throws exception that it
> cannot connect to jmxUrl for process. and then kills the process after 30
> secs.
>
> Here is stack strace:
>
> 14:34:39 # */opt/gwx/apache-activemq-5.8.0/activemq_8300/bin/activemq_8300
> stop "-Dactivemq.jmx.url=service:jmx:rmi:///jndi/rmi://
> 127.0.0.1:8300/jmxrmi"*
> dirname_prg: /opt/gwx/apache-activemq-5.8.0/activemq_8300/bin
> INFO: Loading '/etc/default/activemq'
> INFO: Using java '/usr/bin/java'
> INFO: Waiting at least 30 seconds for regular process termination of pid
> '3234' :
> Java Runtime: Oracle Corporation 1.7.0_25 /usr/java/jdk1.7.0_25/jre
> Heap sizes: current=1004928k free=999683k max=1004928k
> JVM args: -Xms1G -Xmx1G
> -Djava.util.logging.config.file=logging.properties
> -Dactivemq.classpath=/opt/gwx/apache-activemq-5.8.0/activemq_8300/conf;
> -Dactivemq.home=/opt/gwx/apache-activemq-5.8.0
> -Dactivemq.base=/opt/gwx/apache-activemq-5.8.0/activemq_8300
> -Dactivemq.conf=/opt/gwx/apache-activemq-5.8.0/activemq_8300/conf
> -Dactivemq.data=/opt/gwx/apache-activemq-5.8.0/activemq_8300/data
> Extensions classpath:
>
> [/opt/gwx/apache-activemq-5.8.0/activemq_8300/lib,/opt/gwx/apache-activemq-5.8.0/lib \
> ,/opt/gwx/apache-activemq-5.8.0/activemq_8300/lib/camel,/opt/gwx/apache-activemq-5.8 \
> .0/activemq_8300/lib/optional,/opt/gwx/apache-activemq-5.8.0/activemq_8300/lib/web,/ \
> opt/gwx/apache-activemq-5.8.0/activemq_8300/lib/extra,/opt/gwx/apache-activemq-5.8.0 \
> /lib/camel,/opt/gwx/apache-activemq-5.8.0/lib/optional,/opt/gwx/apache-activemq-5.8.0/lib/web,/opt/gwx/apache-activemq-5.8.0/lib/extra]
>
> ACTIVEMQ_HOME: /opt/gwx/apache-activemq-5.8.0
> ACTIVEMQ_BASE: /opt/gwx/apache-activemq-5.8.0/activemq_8300
> ACTIVEMQ_CONF: /opt/gwx/apache-activemq-5.8.0/activemq_8300/conf
> ACTIVEMQ_DATA: /opt/gwx/apache-activemq-5.8.0/activemq_8300/data
> ..Connecting to pid: 3234
> ......INFO: failed to resolve jmxUrl for pid:3234, using default JMX url
> .......Connecting to JMX URL:
> service:jmx:rmi:///jndi/rmi://localhost:1099/jmxrmi
> ..ERROR: java.lang.RuntimeException: Failed to execute stop task. Reason:
> java.io.IOException: Failed to retrieve RMIServer stub:
> javax.naming.ServiceUnavailableException [Root exception is
> java.rmi.ConnectException: Connection refused to host: localhost; nested
> exception is:
> java.net.ConnectException: Connection refused]
> java.lang.RuntimeException: Failed to execute stop task. Reason:
> java.io.IOException: Failed to retrieve RMIServer stub:
> javax.naming.ServiceUnavailableException [Root exception is
> java.rmi.ConnectException: Connection refused to host: localhost; nested
> exception is:
> java.net.ConnectException: Connection refused]
> at
> org.apache.activemq.console.command.ShutdownCommand.runTask(ShutdownCommand.java:116)
> at
> org.apache.activemq.console.command.AbstractCommand.execute(AbstractCommand.java:57)
> at
> org.apache.activemq.console.command.AbstractJmxCommand.execute(AbstractJmxCommand.java:387)
> at
> org.apache.activemq.console.command.ShellCommand.runTask(ShellCommand.java:150)
> at
> org.apache.activemq.console.command.AbstractCommand.execute(AbstractCommand.java:57)
> at
> org.apache.activemq.console.command.ShellCommand.main(ShellCommand.java:104)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at org.apache.activemq.console.Main.runTaskClass(Main.java:262)
> at org.apache.activemq.console.Main.main(Main.java:115)
> ERROR: java.lang.Exception: java.io.IOException: Failed to retrieve
> RMIServer stub: javax.naming.ServiceUnavailableException [Root exception is
> java.rmi.ConnectException: Connection refused to host: localhost; nested
> exception is:
> java.net.ConnectException: Connection refused]
> java.lang.Exception: java.io.IOException: Failed to retrieve RMIServer
> stub: javax.naming.ServiceUnavailableException [Root exception is
> java.rmi.ConnectException: Connection refused to host: localhost; nested
> exception is:
> java.net.ConnectException: Connection refused]
> at
> org.apache.activemq.console.command.ShutdownCommand.runTask(ShutdownCommand.java:117)
> at
> org.apache.activemq.console.command.AbstractCommand.execute(AbstractCommand.java:57)
> at
> org.apache.activemq.console.command.AbstractJmxCommand.execute(AbstractJmxCommand.java:387)
> at
> org.apache.activemq.console.command.ShellCommand.runTask(ShellCommand.java:150)
> at
> org.apache.activemq.console.command.AbstractCommand.execute(AbstractCommand.java:57)
> at
> org.apache.activemq.console.command.ShellCommand.main(ShellCommand.java:104)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at org.apache.activemq.console.Main.runTaskClass(Main.java:262)
> at org.apache.activemq.console.Main.main(Main.java:115)
> Caused by: java.io.IOException: Failed to retrieve RMIServer stub:
> javax.naming.ServiceUnavailableException [Root exception is
> java.rmi.ConnectException: Connection refused to host: localhost; nested
> exception is:
> java.net.ConnectException: Connection refused]
> at
> javax.management.remote.rmi.RMIConnector.connect(RMIConnector.java:358)
> at
> javax.management.remote.JMXConnectorFactory.connect(JMXConnectorFactory.java:268)
> at
> javax.management.remote.JMXConnectorFactory.connect(JMXConnectorFactory.java:227)
> at
> org.apache.activemq.console.command.AbstractJmxCommand.createJmxConnector(AbstractJmxCommand.java:279)
> at
> org.apache.activemq.console.command.AbstractJmxCommand.createJmxConnection(AbstractJmxCommand.java:302)
> at
> org.apache.activemq.console.command.ShutdownCommand.runTask(ShutdownCommand.java:81)
>
> ... 11 more
> Caused by: javax.naming.ServiceUnavailableException [Root exception is
> java.rmi.ConnectException: Connection refused to host: localhost; nested
> exception is:
> java.net.ConnectException: Connection refused]
> at
> com.sun.jndi.rmi.registry.RegistryContext.lookup(RegistryContext.java:118)
> at
> com.sun.jndi.toolkit.url.GenericURLContext.lookup(GenericURLContext.java:203)
> at javax.naming.InitialContext.lookup(InitialContext.java:411)
> at
> javax.management.remote.rmi.RMIConnector.findRMIServerJNDI(RMIConnector.java:1925)
> at
> javax.management.remote.rmi.RMIConnector.findRMIServer(RMIConnector.java:1892)
> at
> javax.management.remote.rmi.RMIConnector.connect(RMIConnector.java:275)
> ... 16 more
> Caused by: java.rmi.ConnectException: Connection refused to host:
> localhost; nested exception is:
> java.net.ConnectException: Connection refused
> at
> sun.rmi.transport.tcp.TCPEndpoint.newSocket(TCPEndpoint.java:619)
> at
> sun.rmi.transport.tcp.TCPChannel.createConnection(TCPChannel.java:216)
> at
> sun.rmi.transport.tcp.TCPChannel.newConnection(TCPChannel.java:202)
> at sun.rmi.server.UnicastRef.newCall(UnicastRef.java:340)
> at sun.rmi.registry.RegistryImpl_Stub.lookup(Unknown Source)
> at
> com.sun.jndi.rmi.registry.RegistryContext.lookup(RegistryContext.java:114)
> ... 21 more
> Caused by: java.net.ConnectException: Connection refused
> at java.net.PlainSocketImpl.socketConnect(Native Method)
> at
> java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
> at
> java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
> at
> java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
> at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
> at java.net.Socket.connect(Socket.java:579)
> at java.net.Socket.connect(Socket.java:528)
> at java.net.Socket.<init>(Socket.java:425)
> at java.net.Socket.<init>(Socket.java:208)
> at
> sun.rmi.transport.proxy.RMIDirectSocketFactory.createSocket(RMIDirectSocketFactory.java:40)
> at
> sun.rmi.transport.proxy.RMIMasterSocketFactory.createSocket(RMIMasterSocketFactory.java:146)
> at
> sun.rmi.transport.tcp.TCPEndpoint.newSocket(TCPEndpoint.java:613)
> ... 26 more
> ............
> INFO: Regular shutdown not successful, sending SIGKILL to process with
> pid '3234'
>
>
> -----------------
>
> Here is what my activemq_8100 script looks like:
>
> ## Figure out the ACTIVEMQ_BASE from the directory this script was run from
> PRG="$0"
> progname=`basename "$0"`
> saveddir=`pwd`
> # need this for relative symlinks
> dirname_prg=`dirname "$PRG"`
> cd "$dirname_prg"
> echo "dirname_prg: $dirname_prg"
> while [ -h "$PRG" ] ; do
> ls=`ls -ld "$PRG"`
> link=`expr "$ls" : '.*-> \(.*\)$'`
> if expr "$link" : '.*/.*' > /dev/null; then
> PRG="$link"
> else
> PRG=`dirname "$PRG"`"/$link"
> fi
> done
> ACTIVEMQ_BASE=`dirname "$PRG"`/..
> cd "$saveddir"
>
> ACTIVEMQ_BASE=`cd "$ACTIVEMQ_BASE" && pwd`
>
> ## Add system properties for this instance here (if needed), e.g
> #export ACTIVEMQ_OPTS_MEMORY="-Xms256M -Xmx1G"
> #export ACTIVEMQ_OPTS="$ACTIVEMQ_OPTS_MEMORY
> -Dorg.apache.activemq.UseDedicatedTaskRunner=true
> -Djava.util.logging.config.file=logging.properties"
>
> export
> ACTIVEMQ_SUNJMX_CONTROL="-Dactivemq.jmx.url=service:jmx:rmi:///jndi/rmi://
> 127.0.0.1:8100/jmxrmi"
> #
> ACTIVEMQ_SUNJMX_START="-Dcom.sun.management.jmxremote.port=8100 "
> ACTIVEMQ_SUNJMX_START="$ACTIVEMQ_SUNJMX_START
> -Dcom.sun.management.jmxremote.password.file=${ACTIVEMQ_BASE}/conf/jmx.password"
> ACTIVEMQ_SUNJMX_START="$ACTIVEMQ_SUNJMX_START
> -Dcom.sun.management.jmxremote.access.file=${ACTIVEMQ_BASE}/conf/jmx.access"
> ACTIVEMQ_SUNJMX_START="$ACTIVEMQ_SUNJMX_START
> -Dcom.sun.management.jmxremote.ssl=false"
> #ACTIVEMQ_SUNJMX_START="$ACTIVEMQ_SUNJMX_START
> -Dcom.sun.management.jmxremote"
>
> export ACTIVEMQ_SUNJMX_START=$ACTIVEMQ_SUNJMX_START
> export ACTIVEMQ_HOME=/opt/gwx/apache-activemq-5.8.0
> export ACTIVEMQ_BASE=$ACTIVEMQ_BASE
>
> ${ACTIVEMQ_HOME}/bin/activemq "$@"
>
> -----------------------------
>
> here is my activemq_8100/conf/activemq.xml file:
>
> <?xml version="1.0" encoding="UTF-8"?><beans xmlns="
> http://www.springframework.org/schema/beans" xmlns:amq="
> http://activemq.apache.org/schema/core" xmlns:xsi="
> http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="
> http://www.springframework.org/schema/beans
> http://www.springframework.org/schema/beans/spring-beans.xsd
> http://activemq.apache.org/schema/core
> http://activemq.apache.org/schema/core/activemq-core.xsd">
>
> <!-- Allows us to use system properties as variables in this
> configuration file -->
> <bean
> class="org.springframework.beans.factory.config.PropertyPlaceholderConfigurer">
> <property name="locations">
> <value>file:${activemq.conf}/credentials.properties</value>
> </property>
> </bean>
>
> <!--
> The <broker> element is used to configure the ActiveMQ broker.
> -->
> <broker xmlns="http://activemq.apache.org/schema/core"
> brokerName="activemq_8100" dataDirectory="${activemq.data}">
>
> <!--
> For better performances use VM cursor and small memory limit.
> For more information, see:
>
> http://activemq.apache.org/message-cursors.html
>
> Also, if your producer is "hanging", it's probably due to
> producer flow control.
> For more information, see:
> http://activemq.apache.org/producer-flow-control.html
> -->
>
> <destinationPolicy>
> <policyMap>
> <policyEntries>
> <policyEntry producerFlowControl="true" topic=">">
> <!-- The constantPendingMessageLimitStrategy is used
> to prevent
> slow topic consumers to block producers and
> affect other consumers
> by limiting the number of messages that are
> retained
> For more information, see:
>
>
> http://activemq.apache.org/slow-consumer-handling.html
>
> -->
> <pendingMessageLimitStrategy>
> <constantPendingMessageLimitStrategy limit="1000"/>
> </pendingMessageLimitStrategy>
> </policyEntry>
> <policyEntry memoryLimit="1mb" producerFlowControl="true"
> queue=">">
> <!-- Use VM cursor for better latency
> For more information, see:
>
> http://activemq.apache.org/message-cursors.html
>
> <pendingQueuePolicy>
> <vmQueueCursor/>
> </pendingQueuePolicy>
> -->
> </policyEntry>
> </policyEntries>
> </policyMap>
> </destinationPolicy>
>
> <managementContext>
> <managementContext createConnector="false"/>
> </managementContext>
>
> <persistenceAdapter>
> <kahaDB directory="${activemq.data}/kahadb"/>
> </persistenceAdapter>
>
> <systemUsage>
> <systemUsage>
> <memoryUsage>
> <memoryUsage limit="64 mb"/>
> </memoryUsage>
> <storeUsage>
> <storeUsage limit="100 gb"/>
> </storeUsage>
> <tempUsage>
> <tempUsage limit="50 gb"/>
> </tempUsage>
> </systemUsage>
> </systemUsage>
>
> <!--
> The transport connectors expose ActiveMQ over a given protocol
> to
> clients and other brokers. For more information, see:
>
> http://activemq.apache.org/configuring-transports.html
> -->
> <transportConnectors>
> <!-- DOS protection, limit concurrent connections to 1000 and
> frame size to 100MB -->
> <transportConnector name="openwire" uri="tcp://0.0.0.0:
> ${openwirePort}?maximumConnections=1000&wireformat.maxFrameSize=104857600"/>
> <transportConnector name="amqp" uri="amqp://0.0.0.0:
> ${amqpPort}?maximumConnections=1000&wireformat.maxFrameSize=104857600"/>
> </transportConnectors>
>
> <!-- destroy the spring context on shutdown to stop jetty -->
> <shutdownHooks>
> <bean xmlns="http://www.springframework.org/schema/beans"
> class="org.apache.activemq.hooks.SpringContextHook"/>
> </shutdownHooks>
>
> </broker>
>
> <!--
> Enable web consoles, REST and Ajax APIs and demos
>
> Take a look at ${ACTIVEMQ_HOME}/conf/jetty.xml for more details
> -->
> <!-- <import resource="jetty.xml"/> -->
>
> </beans>
>
>
> I have tried to stop without jmxUrl in in stop command and it had same
> result.
>
>
> thanks,
> Chirag
>
>
[prev in list] [next in list] [prev in thread] [next in thread]
Configure |
About |
News |
Add a list |
Sponsored by KoreLogic