[prev in list] [next in list] [prev in thread] [next in thread] 

List:       cassandra-dev
Subject:    Re: streaming_connections_per_host - speeding up CPU bound bootstrap
From:       Corentin Chary <corentin.chary () gmail ! com>
Date:       2016-12-12 6:50:32
Message-ID: CAHR064gMuJUD95JtRN758pHKe6=swztr1mT09+AbTidnQp7RmQ () mail ! gmail ! com
[Download RAW message or body]

Hard to say because this comment doesn't show the code that was tried.

My proposed change
(https://issues.apache.org/jira/secure/attachment/12842717/0001-streaming-add-a-way-to-configure-the-number-of-conne.patch)
 should open multiple connections per hosts: this fixes blocking writes
on a single connection or CPU-bound (de)-serialization.


On Mon, Dec 12, 2016 at 2:04 AM, Nate McCall <zznate.m@gmail.com> wrote:
> I have not dug too deeply yet, but how would you compare/reconcile
> your proposed changes with this comment:
> https://issues.apache.org/jira/browse/CASSANDRA-4663?focusedCommentId=15342248&page= \
> com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15342248 
> On Thu, Dec 8, 2016 at 4:10 AM, Corentin Chary <corentin.chary@gmail.com> wrote:
> > Currently the StreamPlan created for bootstrap (and rebuild) will only
> > create one connection per host. If you have less nodes than cores,
> > this is likely to be CPU bound (a CPU seems to be able to process
> > ~5MB/s).
> > 
> > Is there any reason why something naive like
> > https://github.com/iksaif/cassandra/commit/8352c21284811ca15d63183ceae0b11586623f31
> >  would not work ?
> > 
> > I believe this is what is about
> > https://issues.apache.org/jira/browse/CASSANDRA-4663
> > See also: https://issues.apache.org/jira/browse/CASSANDRA-12229, but I
> > don't believe non-blocking I/O would change anything here.
> > 
> > --
> > Corentin Chary
> > http://xf.iksaif.net



-- 
Corentin Chary
http://xf.iksaif.net


[prev in list] [next in list] [prev in thread] [next in thread]