[prev in list] [next in list] [prev in thread] [next in thread]
List: flume-user
Subject: Re: Twitter Source problem
From: Ronald Van de Kuil <ronald.van.de.kuil () nl ! ibm ! com>
Date: 2016-05-12 14:05:19
Message-ID: 201605121405.u4CE5ZGr007417 () d06av06 ! portsmouth ! uk ! ibm ! com
[Download RAW message or body]
This is a multipart message in MIME format.
--=_alternative 004D4751C1257FB1_=
Content-Type: text/plain; charset="US-ASCII"
Hello,
I used the following class:
a1.sources.r1.type = com.cloudera.flume.source.TwitterSource
It is running stable since Februari 2016.
Hope that helps.
Met Vriendelijke Groet,
Ronald van de Kuil
From: chandra sekar <thittaisekar@hotmail.com>
To: "user@flume.apache.org" <user@flume.apache.org>
Date: 12-05-16 03:38
Subject: Re: Twitter Source problem
Dear Ronald,
I have a twitter apps account and created all the keys. here I did removed
the keys due to the security purpose.
Regards
Chandrasekar.
Sent from Outlook
From: Ronald Van de Kuil <ronald.van.de.kuil@nl.ibm.com>
Sent: Tuesday, May 10, 2016 2:10 PM
To: user@flume.apache.org
Subject: Re: Twitter Source problem
You would have to create a twitter app (account) to get your key, token
and so on.
Op 10 mei 2016 om 05:04 heeft chandra sekar <thittaisekar@hotmail.com> het
volgende geschreven:
Hi I am using Twitter source to collect the data from twitter. Suddenly
the process throw an error and there is no data from twitter feed.
Error:
404:The URI requested is invalid or the resource requested, such as a
user, does not exist.
Unknown URL. See Twitter Streaming API documentation at
http://dev.twitter.com/pages/streaming_api
Configuration:
TwitterAgent.sources = Twitter
TwitterAgent.channels = MemChannel
TwitterAgent.sinks = HDFS
TwitterAgent.sources.Twitter.type =
com.cloudera.flume.source.TwitterSource
# TwitterAgent.sources.Twitter.type =
org.apache.flume.source.twitter.TwitterSource
TwitterAgent.sources.Twitter.channels = MemChannel
TwitterAgent.sources.Twitter.consumerKey =
TwitterAgent.sources.Twitter.consumerSecret =
TwitterAgent.sources.Twitter.accessToken =
TwitterAgent.sources.Twitter.accessTokenSecret =
TwitterAgent.sources.Twitter.keywords = data analytics, data science ,
hadoop , big data
TwitterAgent.sinks.HDFS.channel = MemChannel
TwitterAgent.sinks.HDFS.type = hdfs
TwitterAgent.sinks.HDFS.hdfs.path = hdfs://localhost:8020/data_analytics
TwitterAgent.sinks.HDFS.hdfs.fileType = DataStream
TwitterAgent.sinks.HDFS.hdfs.writeFormat = Text
TwitterAgent.sinks.HDFS.hdfs.batchSize = 10000
TwitterAgent.sinks.HDFS.hdfs.rollSize = 0
TwitterAgent.sinks.HDFS.hdfs.rollCount = 100000
TwitterAgent.channels.MemChannel.type = memory
TwitterAgent.channels.MemChannel.capacity = 500000
TwitterAgent.channels.MemChannel.transactionCapacity = 3000
Thanks & Regards
Chandrasekar
Tenzij hierboven anders aangegeven: / Unless stated otherwise above:
IBM Nederland B.V.
Gevestigd te Amsterdam
Inschrijving Handelsregister Amsterdam Nr. 33054214
Tenzij hierboven anders aangegeven: / Unless stated otherwise above:
IBM Nederland B.V.
Gevestigd te Amsterdam
Inschrijving Handelsregister Amsterdam Nr. 33054214
--=_alternative 004D4751C1257FB1_=
Content-Type: text/html; charset="US-ASCII"
<font size=2 face="sans-serif">Hello,</font>
<br>
<br><font size=2 face="sans-serif">I used the following class:</font>
<br>
<br><font size=2 face="sans-serif">a1.sources.r1.type = \
com.cloudera.flume.source.TwitterSource<br> </font>
<br><font size=2 face="sans-serif">It is running stable since Februari
2016.</font>
<br>
<br><font size=2 face="sans-serif">Hope that helps.</font>
<br>
<br>
<br>
<br><font size=3>Met Vriendelijke Groet, <br>
Ronald van de Kuil <br>
<br>
</font>
<br>
<br>
<br><font size=1 color=#5f5f5f face="sans-serif">From:
</font><font size=1 face="sans-serif">chandra sekar \
<thittaisekar@hotmail.com></font> <br><font size=1 color=#5f5f5f \
face="sans-serif">To: </font><font size=1 \
face="sans-serif">"user@flume.apache.org" \
<user@flume.apache.org></font> <br><font size=1 color=#5f5f5f \
face="sans-serif">Date: </font><font size=1 \
face="sans-serif">12-05-16 03:38</font> <br><font size=1 color=#5f5f5f \
face="sans-serif">Subject: </font><font size=1 \
face="sans-serif">Re: Twitter Source problem</font>
<br>
<hr noshade>
<br>
<br>
<br><font size=3 face="Calibri">Dear Ronald, </font>
<br>
<br><font size=3 face="Calibri">I have a twitter apps account and created
all the keys. here I did removed the keys due to the security purpose.
</font>
<br>
<br><font size=3 face="Calibri">Regards </font>
<br><font size=3 face="Calibri">Chandrasekar.</font>
<br>
<br><font size=3 face="Calibri">Sent from </font><a \
href=http://aka.ms/weboutlook><font size=3 color=blue \
face="Calibri"><u>Outlook</u></font></a> <p><font size=3 face="Calibri"><br>
</font>
<br>
<hr>
<br><font size=2 face="Calibri"><b>From:</b> Ronald Van de Kuil \
<ronald.van.de.kuil@nl.ibm.com><b><br> Sent:</b> Tuesday, May 10, 2016 2:10 \
PM<b><br> To:</b> user@flume.apache.org<b><br>
Subject:</b> Re: Twitter Source problem</font><font size=3 face="Calibri">
</font>
<br><font size=3 face="Calibri"> </font>
<br><font size=3 face="Calibri">You would have to create a twitter app
(account) to get your key, token and so on.</font>
<br><font size=3 face="Calibri"><br>
Op 10 mei 2016 om 05:04 heeft chandra sekar <</font><a \
href=mailto:thittaisekar@hotmail.com><font size=3 color=blue \
face="Calibri"><u>thittaisekar@hotmail.com</u></font></a><font size=3 \
face="Calibri">> het volgende geschreven:<br>
</font>
<br><font size=3 face="Calibri">Hi I am using Twitter source to collect
the data from twitter. Suddenly the process throw an error and there
is no data from twitter feed. </font>
<br>
<br><font size=3 face="Calibri"><b>Error: </b></font>
<br>
<br><font size=3 face="Calibri"><b>404:The URI requested is invalid or
the resource requested, such as a user, does not exist.</b></font>
<br><font size=3 face="Calibri"><b>Unknown URL. See Twitter Streaming API
documentation at </b></font><a href=http://dev.twitter.com/pages/streaming_api><font \
size=3 color=blue face="Calibri"><b><u>http://dev.twitter.com/pages/streaming_api</u></b></font></a>
<br>
<br><font size=3 face="Calibri"><b>Configuration</b>:</font>
<br>
<br><font size=3 face="Calibri">TwitterAgent.sources = Twitter</font>
<br><font size=3 face="Calibri">TwitterAgent.channels = MemChannel</font>
<br><font size=3 face="Calibri">TwitterAgent.sinks = HDFS</font>
<br>
<br><font size=3 face="Calibri">TwitterAgent.sources.Twitter.type = \
com.cloudera.flume.source.TwitterSource</font> <br><font size=3 face="Calibri"># \
TwitterAgent.sources.Twitter.type = \
org.apache.flume.source.twitter.TwitterSource</font> <br><font size=3 \
face="Calibri">TwitterAgent.sources.Twitter.channels = MemChannel</font>
<br><font size=3 face="Calibri">TwitterAgent.sources.Twitter.consumerKey
= </font>
<br><font size=3 face="Calibri">TwitterAgent.sources.Twitter.consumerSecret
= </font>
<br><font size=3 face="Calibri">TwitterAgent.sources.Twitter.accessToken
= </font>
<br><font size=3 face="Calibri">TwitterAgent.sources.Twitter.accessTokenSecret
= </font>
<br>
<br><font size=3 face="Calibri">TwitterAgent.sources.Twitter.keywords =
data analytics, data science , hadoop , big data </font>
<br>
<br><font size=3 face="Calibri">TwitterAgent.sinks.HDFS.channel = MemChannel</font>
<br><font size=3 face="Calibri">TwitterAgent.sinks.HDFS.type = hdfs</font>
<br><font size=3 face="Calibri">TwitterAgent.sinks.HDFS.hdfs.path = \
hdfs://localhost:8020/data_analytics</font> <br><font size=3 \
face="Calibri">TwitterAgent.sinks.HDFS.hdfs.fileType = DataStream</font>
<br><font size=3 face="Calibri">TwitterAgent.sinks.HDFS.hdfs.writeFormat
= Text</font>
<br><font size=3 face="Calibri">TwitterAgent.sinks.HDFS.hdfs.batchSize
= 10000</font>
<br><font size=3 face="Calibri">TwitterAgent.sinks.HDFS.hdfs.rollSize =
0</font>
<br><font size=3 face="Calibri">TwitterAgent.sinks.HDFS.hdfs.rollCount
= 100000</font>
<br>
<br><font size=3 face="Calibri">TwitterAgent.channels.MemChannel.type =
memory</font>
<br><font size=3 face="Calibri">TwitterAgent.channels.MemChannel.capacity
= 500000</font>
<br><font size=3 face="Calibri">TwitterAgent.channels.MemChannel.transactionCapacity
= 3000</font>
<br>
<br>
<br><font size=3 face="Calibri">Thanks & Regards </font>
<br><font size=3 face="Calibri">Chandrasekar</font>
<br><font size=3 face="Calibri">Tenzij hierboven anders aangegeven: / Unless
stated otherwise above:<br>
IBM Nederland B.V.<br>
Gevestigd te Amsterdam<br>
Inschrijving Handelsregister Amsterdam Nr. 33054214</font>
<br><font size=2 face="sans-serif"><br>
Tenzij hierboven anders aangegeven: / Unless stated otherwise above:<br>
IBM Nederland B.V.<br>
Gevestigd te Amsterdam<br>
Inschrijving Handelsregister Amsterdam Nr. 33054214</font>
--=_alternative 004D4751C1257FB1_=--
[prev in list] [next in list] [prev in thread] [next in thread]
Configure |
About |
News |
Add a list |
Sponsored by KoreLogic