[prev in list] [next in list] [prev in thread] [next in thread] 

List:       cassandra-commits
Subject:    [jira] [Commented] (CASSANDRA-8409) Node generating a huge number of tiny sstable_activity flushes
From:       "Benedict (JIRA)" <jira () apache ! org>
Date:       2014-12-31 13:44:13
Message-ID: JIRA.12758994.1417557078000.121449.1420033453393 () Atlassian ! JIRA
[Download RAW message or body]


    [ https://issues.apache.org/jira/browse/CASSANDRA-8409?page=com.atlassian.jira.plu \
gin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14262173#comment-14262173 \
] 

Benedict commented on CASSANDRA-8409:
-------------------------------------

That sounds like a bug, unless you have configured your various limits to high. If \
you file a bug report and include the system log, config files and a heapdump from \
the affected node, we can take a look.

> Node generating a huge number of tiny sstable_activity flushes
> --------------------------------------------------------------
> 
> Key: CASSANDRA-8409
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8409
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Environment: Cassandra 2.1.0, Oracle JDK 1.8.0_25, Ubuntu 12.04
> Reporter: Fred Wulff
> Fix For: 2.1.3
> 
> Attachments: system-sstable_activity-ka-67802-Data.db
> 
> 
> On one of my nodes, I'm seeing hundreds per second of "INFO  21:28:05 Enqueuing \
> flush of sstable_activity: 0 (0%) on-heap, 33 (0%) off-heap". tpstats shows a \
> steadily climbing # of pending MemtableFlushWriter/MemtablePostFlush until the node \
> OOMs. When the flushes actually happen the sstable written is invariably 121 bytes. \
> I'm writing pretty aggressively to one of my user tables (sev.mdb_group_pit), but \
> that table's flushing behavior seems reasonable. tpstats:
> {quote}
> frew@hostname:~/s_dist/apache-cassandra-2.1.0$ bin/nodetool -h hostname tpstats
> Pool Name                    Active   Pending      Completed   Blocked  All time \
> blocked MutationStage                   128      4429          36810         0      \
> 0 ReadStage                         0         0           1205         0            \
> 0 RequestResponseStage              0         0          24910         0            \
> 0 ReadRepairStage                   0         0             26         0            \
> 0 CounterMutationStage              0         0              0         0            \
> 0 MiscStage                         0         0              0         0            \
> 0 HintedHandoff                     2         2              9         0            \
> 0 GossipStage                       0         0           5157         0            \
> 0 CacheCleanupExecutor              0         0              0         0            \
> 0 InternalResponseStage             0         0              0         0            \
> 0 CommitLogArchiver                 0         0              0         0            \
> 0 CompactionExecutor                4        28            429         0            \
> 0 ValidationExecutor                0         0              0         0            \
> 0 MigrationStage                    0         0              0         0            \
> 0 AntiEntropyStage                  0         0              0         0            \
> 0 PendingRangeCalculator            0         0             11         0            \
> 0 MemtableFlushWriter               8     38644           8987         0            \
> 0 MemtablePostFlush                 1     38940           8735         0            \
> 0 MemtableReclaimMemory             0         0           8987         0            \
> 0 Message type           Dropped
> READ                         0
> RANGE_SLICE                  0
> _TRACE                       0
> MUTATION                 10457
> COUNTER_MUTATION             0
> BINARY                       0
> REQUEST_RESPONSE             0
> PAGED_RANGE                  0
> READ_REPAIR                208
> {quote}
> I've attached one of the produced sstables.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[prev in list] [next in list] [prev in thread] [next in thread]