[prev in list] [next in list] [prev in thread] [next in thread] 

List:       hadoop-user
Subject:    Error using DistributedCache
From:       "Tanton Gibbs" <tanton.gibbs () gmail ! com>
Date:       2008-05-29 21:47:25
Message-ID: 871c84e60805291447u633f6e2dpc0029d1b144c08cd () mail ! gmail ! com
[Download RAW message or body]

I'm trying to use a DistributedCache in Hadoop 0.17, but am having problems.

I add a file to the distributed cache with the following code:

DistributedCache.addCacheFile(new
URI("hdfs://ngcwydev102:50023/user/tgibbs/myDir/myOtherDir/part-00000"),
conf);

where conf is the JobConf I've created.

Now, in configure(), I do the following to read  the file:

localFiles = DistributedCache.getLocalCacheFiles(conf);
Path path = localFiles[0];
FileSystem fs = path.getFileSystem(conf);
BufferedReader bdr = new BufferedReader(new InputStreamReader(new
FSDataInputStream(fs.open(path))));

This fails and tells me that it can't open:
hdfs://ngcwydev102:50023/user/tgibbs/myDir/myOtherDir/part-00000/part-00000

I've tried leaving off the part-00000 but that didn't work, it always
seems to replace the last part of my file.

Any ideas?

Thanks,
Tanton
[prev in list] [next in list] [prev in thread] [next in thread]