[prev in list] [next in list] [prev in thread] [next in thread] 

List:       cassandra-dev
Subject:    Re: UML sequence diagrams on Wiki for explaining read/write path
From:       Michael Edge <edge.michael () gmail ! com>
Date:       2015-12-04 4:06:22
Message-ID: CABQvE8_KkPs+--u6zh=0vctVYFY+biWPrpYbBcgovU3bmgk=nw () mail ! gmail ! com
[Download RAW message or body]


Thanks for reviewing and providing comments. I've created links on the Wiki
front page for the Read/Write path pages.

Details on MV population are probably better left to another page, so I'll
leave those for now.

One thing I'm unclear on in the Read Path is the relationship of Table to
MemTable:

CollationController code shows that multiple MemTables are merged. My
understanding is that only 1 MemTable is active – once it is flushed the
MemTable is no longer used. I'm not sure why we are scanning multiple
MemTables. Is it due to the handover between MemTables at flush time? When
an existing MemTable is about to become non-active and a new one taking
it's place?

On 3 December 2015 at 00:05, Jack Krupansky <jack.krupansky@gmail.com>
wrote:

> Thanks. I took a quick look and it seems fine, but... I don't have the
> depth of expertise in that code to be 100% sure of each detail. Hopefully
> Carl or Jake, et al can review.
>
> One additional point on MV: When a new MV is created for a base table that
> is already populated, that kicks off a backfilling process that will read
> each existing row from the base table and apply the MV update process. I'm
> not sure of the precise details there either (like, which consistency those
> backfill writes use.) And that backfilling process has to occur on each
> node of the cluster (but not each replica.) Again, Carl, Jake, et al need
> to review precise details.
>
> -- Jack Krupansky
>
> On Wed, Dec 2, 2015 at 1:02 AM, Michael Edge <edge.michael@gmail.com>
> wrote:
>
> > Yes - good point.
> >
> > I've updated the text for the write path page below - could you please
> > review? Once my understanding is correct I'll update the sequence diagram
> > to match.
> >
> > https://wiki.apache.org/cassandra/WritePathForUsers
> >
> > On 1 December 2015 at 22:51, Jack Krupansky <jack.krupansky@gmail.com>
> > wrote:
> >
> > > I just remembered... the new Materialized View support in 3.0 - writes
> to
> > > the materialized views get triggered when a write occurs to the base
> > table.
> > > That needs to be in the write path flow/description as well.
> > >
> > > -- Jack Krupansky
> > >
> > > On Mon, Nov 30, 2015 at 9:30 PM, Michael Edge <edge.michael@gmail.com>
> > > wrote:
> > >
> > > > Thanks for the feedback guys. I've made the updates.
> > > >
> > > > On 1 December 2015 at 00:56, Jack Krupansky <
> jack.krupansky@gmail.com>
> > > > wrote:
> > > >
> > > > > Great stuff!
> > > > >
> > > > > You wrote "When the replica node comes back online the coordinator
> > node
> > > > > will send the data to the replica node", which is partially true -
> if
> > > the
> > > > > replica comes back online within the timeout window of three hours.
> > So,
> > > > you
> > > > > probably want to say something like:
> > > > >
> > > > > "If the replica node comes back online within the hinted handoff
> > window
> > > > the
> > > > > coordinator node will send the data to the replica node, otherwise
> > the
> > > > > replica node will need to be repaired."
> > > > >
> > > > > And maybe mention the configuration of the window. Change "The data
> > is
> > > > > stored for a default period of 3 hours" to "The data is stored for
> a
> > > > > default period of 3 hours, configurable using the
> > > > > max_hint_window_in_ms property
> > > > > in cassandra.yaml."
> > > > >
> > > > > -- Jack Krupansky
> > > > >
> > > > > On Mon, Nov 30, 2015 at 2:24 AM, Michael Edge <
> > edge.michael@gmail.com>
> > > > > wrote:
> > > > >
> > > > > > Write path docs updated on Wiki - please review diagram/text and
> > let
> > > me
> > > > > > have your comments (or update text in place).
> > > > > >
> > > > > > https://wiki.apache.org/cassandra/WritePathForUsers
> > > > > >
> > > > > > Cheers,
> > > > > >
> > > > > > Michael
> > > > > >
> > > > > > On 26 November 2015 at 11:25, Michael Shuler <
> > michael@pbandjelly.org
> > > >
> > > > > > wrote:
> > > > > >
> > > > > > > On 11/25/2015 07:36 PM, Michael Edge wrote:
> > > > > > > > I'd like to update the read/write path description on the
> wiki
> > > (see
> > > > > > link
> > > > > > > > below) by adding a couple of UML sequence diagrams I drew a
> > while
> > > > > ago.
> > > > > > I
> > > > > > > > think they are much better than long textual descriptions for
> > > > > > describing
> > > > > > > > the order of operations on components. Before publishing them
> > I'd
> > > > > > prefer
> > > > > > > a
> > > > > > > > knowledgeable volunteer to review them to ensure they are
> > > accurate.
> > > > > > > >
> > > > > > > > Let me know if you're interested and I'll send you a copy.
> > > > > > > >
> > > > > > > > https://wiki.apache.org/cassandra/ArchitectureOverview
> > > > > > >
> > > > > > > I don't think the mailing list likes attachments, so throw them
> > up
> > > > on a
> > > > > > > web server somewhere and send the list link(s) to your
> diagrams.
> > > > > > > Alternatively, go ahead and post them on the wiki and ask for
> > > > feedback.
> > > > > > >
> > > > > > > --
> > > > > > > Kind regards,
> > > > > > > Michael
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>


[prev in list] [next in list] [prev in thread] [next in thread]