[prev in list] [next in list] [prev in thread] [next in thread] 

List:       avro-user
Subject:    Avro RPC in Apache Flume fails with IBM JDK
From:       You Hoken <houkeny () gmail ! com>
Date:       2013-11-17 16:19:08
Message-ID: CAJ0E+JknMs8Hfmz6xbiHt+1VFgWg0WAEkcDup3uGg747Ogksqw () mail ! gmail ! com
[Download RAW message or body]

Hi,

I'm using Avro (1.7.2) RPC in Apache Flume 1.3.1 with the following environment.

 -OS
   SUSE Linux Enterprise Server 11 (x86_64)
   VERSION = 11
   PATCHLEVEL = 2

 -IBM JDK
  java version "1.6.0"
  Java(TM) SE Runtime Environment (build pxi3260sr10fp1-20120321_01(SR10
FP1))
  IBM J9 VM (build 2.4, JRE 1.6.0 IBM J9 2.4 Linux x86-32
jvmxi3260sr10fp1-20120202_101568 (JIT enabled, AOT enabled)
  J9VM - 20120202_101568
  JIT  - r9_20111107_21307ifx1
  GC   - 20120202_AA)
  JCL  - 20120320_01

Avro RPC (AvroSource / AvroSink) is used to collect logs.

My problem is that Avro RPC fails sometimes...

in the case of success in Avro RPC, I could find the following messages in
"flume.log".

==============================
[TIME] INFO  [lifecycleSupervisor-1-1]
(org.apache.flume.source.AvroSource.start:143)  - Starting Avro source
avroSrc: { bindAddress: xxxxx, port: xxxx }...
[TIME] INFO  [lifecycleSupervisor-1-3]
(org.apache.flume.source.AvroSource.start:185)  - Avro source avroSrc
started.
==============================

but in the case of failure, I could not find " Avro source avroSrc started.
" message.
# though I found "Starting Avro source avroSrc".

so, I think that the biggest problem is failure of starting Avro source.

and furthermore, implemented debug log in the source code to trace in
detail, and start and stop repeatedly,
then I found that MessageDigest#getInstance("MD5") in
org.apache.avro.Protocol#getMD5() hang up.

does anyone know the cause of this problem and how to solve ?

I guess this problem might be caused when using IBM JDK. how do you think ?
is this problem related to [AVRO-1221]?

and one more question,
when Avro source started successfully, I suppose that MD5 digest is same
value each time.
so, what does MD5 digest need for ?

regards,

You

[Attachment #3 (text/html)]

<div dir="ltr"><pre style="white-space:pre-wrap;color:rgb(0,0,0)">Hi,

I&#39;m using Avro (1.7.2) RPC in Apache Flume 1.3.1 with the following environment.

 -OS
   SUSE Linux Enterprise Server 11 (x86_64)
   VERSION = 11
   PATCHLEVEL = 2

 -IBM JDK
  java version &quot;1.6.0&quot;
  Java(TM) SE Runtime Environment (build pxi3260sr10fp1-20120321_01(SR10
FP1))
  IBM J9 VM (build 2.4, JRE 1.6.0 IBM J9 2.4 Linux x86-32
jvmxi3260sr10fp1-20120202_101568 (JIT enabled, AOT enabled)
  J9VM - 20120202_101568
  JIT  - r9_20111107_21307ifx1
  GC   - 20120202_AA)
  JCL  - 20120320_01

Avro RPC (AvroSource / AvroSink) is used to collect logs.</pre><pre \
style="white-space:pre-wrap;color:rgb(0,0,0)">My problem is that Avro RPC fails \
sometimes...

in the case of success in Avro RPC, I could find the following messages in
&quot;flume.log&quot;.

==============================
[TIME] INFO  [lifecycleSupervisor-1-1]
(org.apache.flume.source.AvroSource.start:143)  - Starting Avro source
avroSrc: { bindAddress: xxxxx, port: xxxx }...
[TIME] INFO  [lifecycleSupervisor-1-3]
(org.apache.flume.source.AvroSource.start:185)  - Avro source avroSrc
started.
==============================

but in the case of failure, I could not find &quot; Avro source avroSrc started.
&quot; message.
# though I found &quot;Starting Avro source avroSrc&quot;.

so, I think that the biggest problem is failure of starting Avro source.

and furthermore, implemented debug log in the source code to trace in
detail, and start and stop repeatedly,
then I found that MessageDigest#getInstance(&quot;MD5&quot;) in
org.apache.avro.Protocol#getMD5() hang up.

does anyone know the cause of this problem and how to solve ?

I guess this problem might be caused when using IBM JDK. how do you think ?
is this problem related to [AVRO-1221]?</pre><pre \
style="white-space:pre-wrap;color:rgb(0,0,0)">and one more question, when Avro source \
started successfully, I suppose that MD5 digest is same value each time.
so, what does MD5 digest need for ?

regards,

You</pre></div>



[prev in list] [next in list] [prev in thread] [next in thread]