[prev in list] [next in list] [prev in thread] [next in thread] 

List:       cassandra-user
Subject:    Re: commitlog content
From:       Vitaliy Semochkin <vitaliy.se () gmail ! com>
Date:       2018-08-30 16:57:10
Message-ID: CAHyKpfOSh63vxzG5xB3YFR2Qk=WA7+R794XyR3tSDy2gKJxhTw () mail ! gmail ! com
[Download RAW message or body]

Thank you for the  excellent response Alain!
On Thu, Aug 30, 2018 at 5:25 PM Alain RODRIGUEZ <arodrime@gmail.com> wrote:
>
> Hello Vitaly.
>
> This sounds weird to me (unless we are speaking about a small size MB, a =
few GB maybe). Then the commit log size is limited, by default (see below) =
and the data should grow bigger in most cases.
>
> According to the documentation (http://cassandra.apache.org/doc/latest/ar=
chitecture/storage_engine.html#commitlog):
>
>> commitlog_total_space_in_mb: Total space to use for commit logs on disk.
>> If space gets above this value, Cassandra will flush every dirty CF in t=
he oldest segment and remove it. So a small total commitlog space will tend=
 to cause more flush activity on less-active columnfamilies.
>> The default value is the smaller of 8192, and 1/4 of the total space of =
the commitlog volume.
>> Default Value: 8192
>
>
> The commit log is supposed to be cleaned on flush, thus the solution to r=
educe the disk space used by commit logs are multiple:
> - Decrease the value for 'commitlog_total_space_in_mb' (probably the best=
 option, you say what you want, and you get it)
> - Use the table option 'memtable_flush_period_in_ms' (default is 0, pick =
what you would like here - has to be done on all the table you want it to a=
pply)
> - Manually run: 'nodetool flush' should also clean the commit logs
> - Reduce the size of the memtables
> - Limit the maximum size per table before a flush is triggered with 'memt=
able_cleanup_threshold'. According to the doc it's not a good idea though (=
http://cassandra.apache.org/doc/latest/configuration/cassandra_config_file.=
html#memtable-cleanup-threshold).
>
> Also, the data in Cassandra is compacted and compressed. Over a short tim=
e period of test or if the data is small compared to the memory available a=
nd fits mostly in memory, I can imagine that what you describe can happen.
>
> C*heers,
> -----------------------
> Alain Rodriguez - @arodream - alain@thelastpickle.com
> France / Spain
>
> The Last Pickle - Apache Cassandra Consulting
> http://www.thelastpickle.com
>
> Le mar. 28 ao=C3=BBt 2018 =C3=A0 18:24, Vitaliy Semochkin <vitaliy.se@gma=
il.com> a =C3=A9crit :
>>
>> Hello,
>>
>> I've noticed that after a stress test that does only inserts a
>> commitlog content exceeds data dir 20 times.
>> What can be cause of such behavior?
>>
>> Running nodetool compact did not change anything.
>>
>> Regards,
>> Vitaliy
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: user-unsubscribe@cassandra.apache.org
>> For additional commands, e-mail: user-help@cassandra.apache.org
>>

---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@cassandra.apache.org
For additional commands, e-mail: user-help@cassandra.apache.org

[prev in list] [next in list] [prev in thread] [next in thread]