[prev in list] [next in list] [prev in thread] [next in thread]
List: wsf-javascript-dev
Subject: Re: [Dev] Upgrading to Cassandra 1.1.0
From: Kasun Weranga <kasunw () wso2 ! com>
Date: 2012-04-30 12:29:32
Message-ID: CAMjEDYNNpm3myVXxskyLo42GKjoszOs+oLP=kudjkgAR6M4EEQ () mail ! gmail ! com
[Download RAW message or body]
[Attachment #2 (multipart/alternative)]
Hi all,
Above issue comes due to snappy-java loads the native library to java
extension class loader but in our osgi environment parent classloader of
the org.eclipse.osgi.baseadaptor.BaseAdaptor classloader is null (which is
the parent classloader of the snappy-java bundle). Is that the
default behavior in OSGI ? or is it a bug ?
because of that bundle class loader can't find the native library loaded by
the java extension classloader.
I was able to solve this issue by patching the above snappy-java bundle to
load the native library to bundle classloader it self.
If it is not a bug in osgi environment, shall I commit this patched
snappy-java source to our dependencies?
Thanks,
KasunW
On Fri, Apr 27, 2012 at 8:16 PM, Kasun Weranga <kasunw@wso2.com> wrote:
> Hi,
>
> After upgrading to Cassandra 1.1.0, I am facing some issue when starting
> the Cassandra server.
>
> This is the exception
>
> [2012-04-27 19:31:38,740] INFO {org.apache.cassandra.db.Memtable} -
> Writing Memtable-go@1431496612(656/820 serialized/live bytes, 12 ops)
> [2012-04-27 19:31:38,745] ERROR
> {org.apache.cassandra.service.AbstractCassandraDaemon} - Exception in
> thread Thread[FlushWriter:1,5,main]
> java.lang.UnsatisfiedLinkError:
> org.xerial.snappy.SnappyNative.maxCompressedLength(I)I
> at org.xerial.snappy.SnappyNative.maxCompressedLength(Native Method)
> at org.xerial.snappy.Snappy.maxCompressedLength(Snappy.java:316)
> at
> org.apache.cassandra.io.compress.SnappyCompressor.initialCompressedBufferLength(SnappyCompressor.java:42)
> at
> org.apache.cassandra.io.compress.CompressedSequentialWriter.<init>(CompressedSequentialWriter.java:63)
> at
> org.apache.cassandra.io.compress.CompressedSequentialWriter.open(CompressedSequentialWriter.java:34)
> at
> org.apache.cassandra.io.sstable.SSTableWriter.<init>(SSTableWriter.java:91)
> at
> org.apache.cassandra.db.ColumnFamilyStore.createFlushWriter(ColumnFamilyStore.java:1848)
> at
> org.apache.cassandra.db.Memtable.writeSortedContents(Memtable.java:281)
> at org.apache.cassandra.db.Memtable.access$600(Memtable.java:48)
> at org.apache.cassandra.db.Memtable$5.runMayThrow(Memtable.java:318)
> at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:30)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:619)
>
>
> This issue comes from snappy-java-1.0.4.1 library and it might be because
> it uses a native library (libsnappyjava.so) and there are some issues
> when it try to load that native library in osgi environment.
>
> According to this [1] previously it didn't work with tomcat too. but they
> have done some hack to get it work.
>
> Does anybody know how to overcome this issue?
>
> [1] http://code.google.com/p/snappy-java/issues/detail?id=21
>
>
> Thanks,
> KasunW
>
>
> On Wed, Apr 25, 2012 at 8:31 PM, Shammi Jayasinghe <shammi@wso2.com>wrote:
>
> >
> >
> > On Wed, Apr 25, 2012 at 5:55 PM, Deependra Ariyadewa <deep@wso2.com>wrote:
> >
> > >
> > >
> > > On Wed, Apr 25, 2012 at 2:23 PM, Kasun Weranga <kasunw@wso2.com> wrote:
> > >
> > > > Hi all,
> > > >
> > > > Shall we do the $subject. That is because currently we are facing some
> > > > issues when trying to integrate Cassandra storage handler (available
> > > > externally) into Hive. Since hive uses libthrift 0.7 and Cassandra 1.0.8
> > > > (current carbon version) uses libthrift 0.6, but the problem is
> > > > cassandra-handler needs to communicate with both hive and cassandra so it
> > > > needs to use only one thrift version.
> > > > We need to fix this issue before our first milestone coming in few
> > > > days. We can overcome this issue easily by upgrading to new cassandra
> > > > 1.1.0 version because it uses libthrift 0.7.
> > > > Also we can take the advantage of the new improvements coming with the
> > > > new Cassandra release.
> > > > Shall we do it ?
> > > >
> > >
> > > It is good have the latest Cassandra version with 4.0.0 release.
> > >
> > > There are changes done in Cassandra dependency to enable Carbon user
> > > permission scheme which We have to merge with Cassandra 1.1.x. Also we have
> > > to verify dependency library list for possible version conflicts.
> > >
> > > It is good to get feed back from MB developers before do this change.
> > >
> > > +1
> >
> > Thanks
> > Shammi
> >
> > > Thanks,
> > >
> > > Deependra.
> > >
> > >
> > >
> > > > Thanks,
> > > > KasunW
> > > >
> > > >
> > >
> > >
> > > --
> > > Deependra Ariyadewa
> > > WSO2, Inc. http://wso2.com/ http://wso2.org
> > >
> > > email deep@wso2.com; cell +94 71 403 5996 ;
> > > Blog http://risenfall.wordpress.com/
> > > PGP info: KeyID: 'DC627E6F'
> > >
> > >
> >
> >
> > --
> > Best Regards,*
> >
> > Shammi Jayasinghe*
> > Senior Software Engineer; WSO2, Inc.; http://wso2.com,
> > mobile: +94 71 4493085
> >
> >
> >
>
>
> --
> *Kasun Weranga*
> Software Engineer
> **
> *WSO2, Inc.
> *lean.enterprise.middleware.
> mobile : +94 772314602
> <http://sanjeewamalalgoda.blogspot.com/>blog \
> :<http://sanjeewamalalgoda.blogspot.com/> http://kasunweranga.blogspot.com/
>
--
*Kasun Weranga*
Software Engineer
**
*WSO2, Inc.
*lean.enterprise.middleware.
mobile : +94 772314602
<http://sanjeewamalalgoda.blogspot.com/>blog
> <http://sanjeewamalalgoda.blogspot.com/>
http://kasunweranga.blogspot.com/
[Attachment #5 (text/html)]
Hi all,<br><br>Above issue comes due to snappy-java loads the native library to java \
extension class loader but in our osgi environment parent classloader of the \
org.eclipse.osgi.baseadaptor.BaseAdaptor classloader is null (which is the parent \
classloader of the snappy-java bundle). Is that the default behavior in OSGI ? or is \
it a bug ?<div>
because of that bundle class loader can't find the native library loaded by the \
java extension classloader. <div><div><br></div><div>I was able to solve this issue \
by patching the above snappy-java bundle to load the native library to bundle \
classloader it self. </div>
<div><br></div><div>If it is not a bug in osgi environment, shall I commit this \
patched snappy-java source to our \
dependencies?</div><div><br></div><div>Thanks,</div><div>KasunW<br><div><br></div><div><br><div \
class="gmail_quote">
On Fri, Apr 27, 2012 at 8:16 PM, Kasun Weranga <span dir="ltr"><<a \
href="mailto:kasunw@wso2.com" target="_blank">kasunw@wso2.com</a>></span> \
wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px \
#ccc solid;padding-left:1ex">
<div class="gmail_extra">Hi,</div><div class="gmail_extra"><br></div><div \
class="gmail_extra">After upgrading to Cassandra 1.1.0, I am facing some issue when \
starting the Cassandra server. </div><div class="gmail_extra">
<br>
</div><div class="gmail_extra">This is the exception </div><div \
class="gmail_extra"><br></div><div class="gmail_extra"><div \
class="gmail_extra">[2012-04-27 19:31:38,740] INFO \
{org.apache.cassandra.db.Memtable} - Writing Memtable-go@1431496612(656/820 \
serialized/live bytes, 12 ops)</div>
<div class="gmail_extra">[2012-04-27 19:31:38,745] ERROR \
{org.apache.cassandra.service.AbstractCassandraDaemon} - Exception in thread \
Thread[FlushWriter:1,5,main]</div><div \
class="gmail_extra">java.lang.UnsatisfiedLinkError: \
org.xerial.snappy.SnappyNative.maxCompressedLength(I)I</div>
<div class="gmail_extra"><span style="white-space:pre-wrap"> </span>at \
org.xerial.snappy.SnappyNative.maxCompressedLength(Native Method)</div><div \
class="gmail_extra"><span style="white-space:pre-wrap"> </span>at \
org.xerial.snappy.Snappy.maxCompressedLength(Snappy.java:316)</div>
<div class="gmail_extra"><span style="white-space:pre-wrap"> </span>at \
org.apache.cassandra.io.compress.SnappyCompressor.initialCompressedBufferLength(SnappyCompressor.java:42)</div><div \
class="gmail_extra"> <span style="white-space:pre-wrap"> </span>at \
org.apache.cassandra.io.compress.CompressedSequentialWriter.<init>(CompressedSequentialWriter.java:63)</div><div \
class="gmail_extra"><span style="white-space:pre-wrap"> </span>at \
org.apache.cassandra.io.compress.CompressedSequentialWriter.open(CompressedSequentialWriter.java:34)</div>
<div class="gmail_extra"><span style="white-space:pre-wrap"> </span>at \
org.apache.cassandra.io.sstable.SSTableWriter.<init>(SSTableWriter.java:91)</div><div \
class="gmail_extra"><span style="white-space:pre-wrap"> </span>at \
org.apache.cassandra.db.ColumnFamilyStore.createFlushWriter(ColumnFamilyStore.java:1848)</div>
<div class="gmail_extra"><span style="white-space:pre-wrap"> </span>at \
org.apache.cassandra.db.Memtable.writeSortedContents(Memtable.java:281)</div><div \
class="gmail_extra"><span style="white-space:pre-wrap"> </span>at \
org.apache.cassandra.db.Memtable.access$600(Memtable.java:48)</div>
<div class="gmail_extra"><span style="white-space:pre-wrap"> </span>at \
org.apache.cassandra.db.Memtable$5.runMayThrow(Memtable.java:318)</div><div \
class="gmail_extra"><span style="white-space:pre-wrap"> </span>at \
org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:30)</div>
<div class="gmail_extra"><span style="white-space:pre-wrap"> </span>at \
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)</div><div \
class="gmail_extra"><span style="white-space:pre-wrap"> </span>at \
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)</div>
<div class="gmail_extra"><span style="white-space:pre-wrap"> </span>at \
java.lang.Thread.run(Thread.java:619)</div><div><br></div><div><br></div><div>This \
issue comes from snappy-java-1.0.4.1 library and it might be because it uses a <span \
style><span style="font-size:12px;white-space:pre-wrap">native library \
(libsnappyjava.so) and there are some issues when it try to load that native library \
in osgi environment. </span></span></div>
<div><span style><span \
style="font-size:12px;white-space:pre-wrap"><br></span></span></div><div><span \
style="font-size:12px;white-space:pre-wrap">According</span><span style><span \
style="font-size:12px;white-space:pre-wrap"> to this [1] previously it didn't \
work with tomcat too. but they have done some hack to get it \
work.</span></span></div>
<div><span style="font-size:12px;white-space:pre-wrap"><br></span></div><div><span \
style><span style="font-size:12px;white-space:pre-wrap">Does anybody know how to \
overcome this issue? </span></span></div> <div><span style><span \
style="font-size:12px;white-space:pre-wrap"><br></span></span></div><div><span \
style><span style="font-size:12px;white-space:pre-wrap">[1] </span></span><a \
href="http://code.google.com/p/snappy-java/issues/detail?id=21" \
target="_blank">http://code.google.com/p/snappy-java/issues/detail?id=21</a></div>
<div class="gmail_extra"><br></div><div \
class="gmail_extra"><br></div>Thanks,</div><div class="gmail_extra">KasunW</div><div \
class="gmail_extra"><br></div><div class="gmail_extra"><div><div class="h5"><br><div \
class="gmail_quote">
On Wed, Apr 25, 2012 at 8:31 PM, Shammi Jayasinghe <span dir="ltr"><<a \
href="mailto:shammi@wso2.com" target="_blank">shammi@wso2.com</a>></span> \
wrote:<br> <blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px \
#ccc solid;padding-left:1ex"><div class="gmail_extra"><br><br><div \
class="gmail_quote"><div>On Wed, Apr 25, 2012 at 5:55 PM, Deependra Ariyadewa <span \
dir="ltr"><<a href="mailto:deep@wso2.com" \
target="_blank">deep@wso2.com</a>></span> wrote:<br>
</div><div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px \
#ccc solid;padding-left:1ex"> <div class="gmail_extra"><br><br><div \
class="gmail_quote"><div><div>On Wed, Apr 25, 2012 at 2:23 PM, Kasun Weranga <span \
dir="ltr"><<a href="mailto:kasunw@wso2.com" \
target="_blank">kasunw@wso2.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc \
solid;padding-left:1ex">
Hi all,<div><br></div><div>Shall we do the $subject. That is because currently we are \
facing some issues when trying to integrate Cassandra storage handler (available \
externally) into Hive. Since hive uses libthrift 0.7 and Cassandra 1.0.8 (current \
carbon version) uses libthrift 0.6, but the problem is cassandra-handler needs to \
communicate with both hive and cassandra so it needs to use only one thrift version. \
</div>
<div>We need to fix this issue before our first milestone coming in few days. We can \
overcome this issue easily by upgrading to new cassandra 1.1.0 version because it \
uses libthrift 0.7. </div><div>Also we can take the advantage of the new improvements \
coming with the new Cassandra release. </div>
<div>Shall we do it ?</div></blockquote><div><br></div></div></div><div>It is good \
have the latest Cassandra version with 4.0.0 release. </div><div><br></div><div>There \
are changes done in Cassandra dependency to enable Carbon user permission scheme \
which We have to merge with Cassandra 1.1.x. Also we have to verify dependency \
library list for possible version conflicts.</div>
<div><br></div><div>It is good to get feed back from MB developers before do this \
change.</div><div><br></div></div></div></blockquote></div><div>+1</div><div><br></div><div>Thanks</div><div>Shammi \
</div><div> <blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px \
#ccc solid;padding-left:1ex"> <div class="gmail_extra"><div \
class="gmail_quote"><div></div><div>Thanks,</div><div><br></div><div>Deependra.</div><div><br></div><div><br></div><blockquote \
class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc \
solid;padding-left:1ex">
<div><br></div><div><div>Thanks,</div><div>KasunW</div><br><span><font \
color="#888888"> </font></span></div><span><font color="#888888">
</font></span></blockquote></div><span><font color="#888888"><br><br \
clear="all"><div><br></div>-- <br><div>Deependra Ariyadewa</div><div>WSO2, Inc. <a \
href="http://wso2.com/" target="_blank">http://wso2.com/</a> <a \
href="http://wso2.org" target="_blank">http://wso2.org</a></div>
<div><br></div><div>email <a href="mailto:deep@wso2.com" \
target="_blank">deep@wso2.com</a>; cell <a href="tel:%2B94%2071%20403%205996" \
value="+94714035996" target="_blank">+94 71 403 5996</a> ;</div><div>Blog <a \
href="http://risenfall.wordpress.com/" \
target="_blank">http://risenfall.wordpress.com/</a></div>
<div><span style="font-family:monospace;white-space:pre-wrap;font-size:medium">PGP \
info: KeyID</span>: 'DC627E6F'</div><br> </font></span></div>
</blockquote></div></div><span><font color="#888888"><br><br \
clear="all"><div><br></div>-- <br><span \
style="font-family:arial,sans-serif;font-size:13px;border-collapse:collapse">Best \
Regards,<b> <br> <br>Shammi Jayasinghe</b><br>
<font face="verdana, sans-serif"><span style="font-size:x-small">Senior Software \
Engineer</span></font>; <span style="font-size:x-small">WSO2, Inc.; </span><a \
href="http://wso2.com/" style="color:rgb(42,93,176)" target="_blank"><span \
style="font-size:x-small">http://wso2.com</span></a><span \
style="font-size:x-small">,</span></span><div>
<span style="font-family:arial,sans-serif;font-size:13px;border-collapse:collapse"><span \
style="font-size:x-small">mobile: <a href="tel:%2B94%2071%204493085" \
value="+94714493085" target="_blank">+94 71 4493085</a> </span></span><br>
<div><br></div></div><br>
</font></span></div>
</blockquote></div><br><br clear="all"><div><br></div></div></div><span \
class="HOEnZb"><font color="#888888">-- <br><span style="border-collapse:collapse"><b \
style="font-size:13px"><font face="georgia, serif">Kasun Weranga</font></b><font \
face="arial, sans-serif" style="font-size:13px"><span style></span></font><br>
<div style="color:rgb(136,136,136)"><font color="#888888"><font color="#000000"><font \
face="georgia, serif">Software Engineer</font></font></font><font \
color="#000000"><font face="georgia, serif"><br></font></font><font><font \
face="georgia, serif"><b></b></font></font></div>
<div style="color:rgb(136,136,136)"><font color="#000000"><b><span \
style="font-weight:normal"><font face="georgia, serif">WSO2, Inc.</font></span><font \
face="georgia, serif"><br></font></b><font face="georgia, \
serif">lean.enterprise.middleware.<br>
</font></font></div><div><font face="georgia, serif">mobile : <a \
href="tel:%2B94%20772314602" value="+94772314602" target="_blank">+94 \
772314602</a><br></font></div><font face="georgia, serif"><a \
href="http://sanjeewamalalgoda.blogspot.com/" style="color:rgb(0,0,153)" \
target="_blank"></a><span style>blog</span><a \
href="http://sanjeewamalalgoda.blogspot.com/" style="color:rgb(0,0,153)" \
target="_blank"><span style="color:rgb(0,0,0)"> </span>:</a></font></span><a \
href="http://kasunweranga.blogspot.com/" target="_blank"><font face="georgia, \
serif">http://kasunweranga.blogspot.com/</font></a><br>
</font></span></div>
</blockquote></div><br><br clear="all"><div><br></div>-- <br><span \
style="border-collapse:collapse"><b style="color:rgb(0,0,0);font-size:13px"><font \
face="georgia, serif">Kasun Weranga</font></b><font face="arial, sans-serif" \
style="font-size:13px"><span style="color:rgb(0,0,0)"></span></font><br>
<div style="color:rgb(136,136,136)"><font color="#888888"><font color="#000000"><font \
face="georgia, serif">Software Engineer</font></font></font><font \
color="#000000"><font face="georgia, serif"><br></font></font><font><font \
face="georgia, serif"><b></b></font></font></div>
<div style="color:rgb(136,136,136)"><font color="#000000"><b><span \
style="font-weight:normal"><font face="georgia, serif">WSO2, Inc.</font></span><font \
face="georgia, serif"><br></font></b><font face="georgia, \
serif">lean.enterprise.middleware.<br>
</font></font></div><div><font face="georgia, serif">mobile : +94 \
772314602<br></font></div><font face="georgia, serif"><a \
href="http://sanjeewamalalgoda.blogspot.com/" style="color:rgb(0,0,153)" \
target="_blank"></a><span \
style="background-color:rgb(255,255,255);color:rgb(0,0,0)">blog</span><a \
href="http://sanjeewamalalgoda.blogspot.com/" style="color:rgb(0,0,153)" \
target="_blank"><span style="background-color:rgb(255,255,255);color:rgb(0,0,0)"> \
</span>:</a></font></span><a href="http://kasunweranga.blogspot.com/" \
target="_blank"><font face="georgia, \
serif">http://kasunweranga.blogspot.com/</font></a><br>
</div></div></div></div>
_______________________________________________
Dev mailing list
Dev@wso2.org
http://wso2.org/cgi-bin/mailman/listinfo/dev
[prev in list] [next in list] [prev in thread] [next in thread]
Configure |
About |
News |
Add a list |
Sponsored by KoreLogic