[prev in list] [next in list] [prev in thread] [next in thread] 

List:       hadoop-user
Subject:    RE: Sqoop Issue
From:       Akash Sharma <sharma101 () hotmail ! com>
Date:       2012-06-27 3:03:40
Message-ID: SNT120-W62174673801A0254762A86FBE70 () phx ! gbl
[Download RAW message or body]


Thanks John and Minh for sharing your thoughts and hepling provide the direction to \
resolve the issue.

I will add the connector as part of Hadoop Jar command. However there is one thought \
that crossed my mind as I was reading these emails.

My initial problem was related to connecting with Oracle DB using the Oracle RAC \
environment. I am still able to connect directly to the physical database however due \
to network issues between Access Node and DB, a routing deamon is unable to route the \
requests appropriately. 

Are we sure we are really addressing the issue at hand? I mean I could definately \
discover if I can really use Oracle RAC connection URL as opposed to direct physical \
connection by writing the Java program. How will this new implementation be different \
from say sqoop tool which does not work when I source Oracle RAC url (proxy) instead \
of direct url. 

Any help is appreciated. 

-Akash

> From: mdnguyen@gmail.com
> Date: Tue, 26 Jun 2012 22:41:24 -0400
> Subject: Re: Sqoop Issue
> To: common-user@hadoop.apache.org
> 
> Akash,
> 
> Instead of adding the connector jar to $HADOOP_HOME/lib, when
> running your map-reduce job using "hadoop jar" you can pass your
> connector jar using the "-libjars" flag.
> 
> For example: hadoop jar hadoop-examples.jar wordcount -files
> cachefile.txt -libjars mylib.jar input output
> 
> ~ Minh
> 
> On Tue, Jun 26, 2012 at 9:52 PM, John Hancock <jhancock1975@gmail.com> wrote:
> > Akash,
> > 
> > I forgot to mention you will need to make sure the connector for your
> > oracle db is available to Hadoop.  There are many ways to do this, but what
> > works for me is I copy the library to <hadoop-install>/lib in all nodes in
> > my cluster.
> > 
> > I've written a similar program to do what you are asking about but using a
> > mysql db, so I have mysql-connector-java-5.0.8-bin.jar in the lib directory
> > on all my hadoop nodes.
> > 
> > -John
> > 
> > On Tue, Jun 26, 2012 at 8:38 AM, Akash Sharma <sharma101@hotmail.com> wrote:
> > 
> > > 
> > > Thanks for the response John,
> > > 
> > > This is exactly what I was looking for. I will pursue this direction and
> > > create a Java program to connect with Oracle DB, extract relevant data and
> > > push it into HDFS.
> > > 
> > > I am guessing the Hadoop/HDFS API's will contain all the relevant calls
> > > required to execute these tasks.
> > > 
> > > Thanks,
> > > 
> > > -Akash
> > > 
> > > > Date: Tue, 26 Jun 2012 05:36:27 -0400
> > > > Subject: Re: Sqoop Issue
> > > > From: jhancock1975@gmail.com
> > > > To: common-user@hadoop.apache.org
> > > > 
> > > > Akash,
> > > > 
> > > > You can write a simple Java program that queries your Oracle DB and uses
> > > > whatever kind of file output object from java.io that you like to write
> > > the
> > > > data to a file.
> > > > 
> > > > Compile the program and package it into a jar file.
> > > > 
> > > > Then run the program using <path-to-hadoop>/'hadoop jar <jar-file-name>
> > > > <output-file-name> in your Hadoop cluster.
> > > > 
> > > > The result will be in HDFS.
> > > > 
> > > > -John
> > > > 
> > > > On Mon, Jun 25, 2012 at 8:02 AM, Akash Sharma <sharma101@hotmail.com>
> > > wrote:
> > > > 
> > > > > 
> > > > > 
> > > > > Hi,
> > > > > 
> > > > > Seeking some advise/options on sqoop of data from Oracle DB to HDFS. We
> > > > > are on Cloudera 3.
> > > > > 
> > > > > Sqoop/JDBC connection to Oracle RAC fails within the Integration
> > > cluster.
> > > > > In the Oracle RAC there  is an additional layer between the Edge
> > > > > Node/Server and Database. Routing the connection occurs to  appropriate
> > > > > database listener. The specific issue is ORA-12516, TNS:listener could
> > > not
> > > > > find available handler with matching protocol stack. This issue is
> > > > > primarily a network configuration issue which is stopping the team to
> > > > > progress further.
> > > > > 
> > > > > I would like to know other than sqoop, what are the other  options to
> > > > > bring the data into HDFS. Our primary goal is to land data from Oracle
> > > DB
> > > > > into HDFS. Since sqoop does not work, we are thinking of using Oracle
> > > > > unload, ftp, and then put command to import the data into HDFS. Please
> > > > > guide if there  is any other option other than this. Will it help in
> > > anyway
> > > > > to code these options in java using HDFS/FTP Api within Java itself
> > > rather
> > > > > than using tools?
> > > > > 
> > > > > Any help is appreciative.
> > > > > 
> > > > > -Akash
> > > > > 
> > > > > 
> > > > > 
> > > 
> > > 
 		 	   		  



[prev in list] [next in list] [prev in thread] [next in thread]