[prev in list] [next in list] [prev in thread] [next in thread] 

List:       hadoop-dev
Subject:    Hadoop's slow and returns a OutOfMemoryError in pseudo-distributed
From:       relpak <kapler () irit ! fr>
Date:       2011-05-26 7:16:06
Message-ID: 31705364.post () talk ! nabble ! com
[Download RAW message or body]


Hi everybody,
 

I developped an application using Hadoop. It runs perfectly with a
stand-alone mode but when I try to run it with the pseudo-distributed mode I
get an error "java.lang.OutOfMemoryError: GC overhead limit exceeded" (but
with the stand-alone mode I don't get this error. The application runs until
its end.). But the JVM is initialized with 2048MB (-Xmx2048m -Xms2048m).
Moveover I find that's slower than the stand-alone mode, is normal (1h20 to
process just 30% of the file whereas the file is 1MB size big and it must be
processed in 1 minute!!!)?

For information, the application is used to index a documents pool and it
uses Hadoop 0.20.2.

My config files are the above :

<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<configuration>
    <property>
        <name>mapred.job.tracker</name>
        <value>localhost:8021</value>
    </property>
    <property>
        <name>mapred.child.java.opts</name>
            <value>-Xmx2048m -Xms2048m</value>
    </property>
</configuration>

 

<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<configuration>
    <property>
        <name>fs.default.name</name>
        <value>hdfs://localhost/</value>
    </property>
</configuration>

 

<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<configuration>
    <property>
        <name>dfs.replication</name>
        <value>1</value>
    </property>
</configuration>

 

I start Hadoop as this :

./hadoop-0.20.2/bin/start-all.sh

And after I run my application as :

hadoop jar myApp.jar [my parameters...]
 

Could anybody help me, please?
 

Thanks.
-- 
View this message in context: \
http://old.nabble.com/Hadoop%27s-slow-and-returns-a-OutOfMemoryError-in-pseudo-distributed-mode-tp31705364p31705364.html
 Sent from the Hadoop core-dev mailing list archive at Nabble.com.


[prev in list] [next in list] [prev in thread] [next in thread]