[prev in list] [next in list] [prev in thread] [next in thread]
List: cassandra-user
Subject: Re: Practical limit on number of column families
From: Robert Wille <rwille () fold3 ! com>
Date: 2016-02-29 19:35:46
Message-ID: F2E96A17-EF2C-435D-9C75-42AB3D3FD485 () fold3 ! com
[Download RAW message or body]
Yes, there is memory overhead for each column family, effectively limiting =
the number of column families. The general wisdom is that you should limit =
yourself to a few hundred.
Robert
On Feb 29, 2016, at 10:30 AM, Fernando Jimenez <fernando.jimenez@wealth-por=
t.com<mailto:fernando.jimenez@wealth-port.com>> wrote:
Hi all
I have a use case for Cassandra that would require creating a large number =
of column families. I have found references to early versions of Cassandra =
where each column family would require a fixed amount of memory on all node=
s, effectively imposing an upper limit on the total number of CFs. I have a=
lso seen rumblings that this may have been fixed in later versions.
To put the question to rest, I have setup a DSE sandbox and created some co=
de to generate column families populated with 3,000 entries each.
Unfortunately I have now hit this issue: https://issues.apache.org/jira/bro=
wse/CASSANDRA-9291
So I will have to retest against Cassandra 3.0 instead
However, I would like to understand the limitations regarding creation of c=
olumn families.
* Is there a practical upper limit?
* is this a fixed limit, or does it scale as more nodes are added into the =
cluster?
* Is there a difference between one keyspace with thousands of column famil=
ies, vs thousands of keyspaces with only a few column families each?
I haven=92t found any hard evidence/documentation to help me here, but if y=
ou can point me in the right direction, I will oblige and RTFM away.
Many thanks for your help!
Cheers
FJ
[Attachment #3 (text/html)]
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=Windows-1252">
</head>
<body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: \
after-white-space;"> Yes, there is memory overhead for each column family, \
effectively limiting the number of column families. The general wisdom is that you \
should limit yourself to a few hundred. <div><br>
</div>
<div>Robert<br>
<div><br>
<div>
<div>On Feb 29, 2016, at 10:30 AM, Fernando Jimenez <<a \
href="mailto:fernando.jimenez@wealth-port.com">fernando.jimenez@wealth-port.com</a>> \
wrote:</div> <br class="Apple-interchange-newline">
<blockquote type="cite">
<div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: \
after-white-space;" class=""> <div class="">
<div style="font-family: Helvetica; font-size: 12px; font-style: normal; \
font-variant: normal; font-weight: normal; letter-spacing: normal; orphans: auto; \
text-align: start; text-indent: 0px; text-transform: none; white-space: normal; \
widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; word-wrap: \
break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" \
class=""> <div class="">Hi all</div>
<div class=""><br class="">
</div>
<div class="">I have a use case for Cassandra that would require creating a large \
number of column families. I have found references to early versions of Cassandra \
where each column family would require a fixed amount of memory on all nodes, \
effectively imposing an upper limit on the total number of CFs. I have also seen \
rumblings that this may have been fixed in later versions.</div> <div class=""><br \
class=""> </div>
<div class="">To put the question to rest, I have setup a DSE sandbox and created \
some code to generate column families populated with 3,000 entries each.</div> <div \
class=""><br class=""> </div>
<div class="">Unfortunately I have now hit this issue: <a \
href="https://issues.apache.org/jira/browse/CASSANDRA-9291" \
class="">https://issues.apache.org/jira/browse/CASSANDRA-9291</a></div> </div>
<div class=""><br class="">
</div>
So I will have to retest against Cassandra 3.0 instead</div>
<div class=""><br class="">
</div>
<div class="">However, I would like to understand the limitations regarding creation \
of column families. </div> <div class=""><br class="">
</div>
<div class=""><span class="Apple-tab-span" style="white-space:pre"></span>* Is there \
a practical upper limit? </div> <div class=""><span class="Apple-tab-span" \
style="white-space:pre"></span>* is this a fixed limit, or does it scale as more \
nodes are added into the cluster? </div> <div class=""><span \
class="Apple-tab-span" style="white-space:pre"></span>* Is there a difference \
between one keyspace with thousands of column families, vs thousands of keyspaces \
with only a few column families each?</div> <div class=""><br class="">
</div>
<div class="">I haven’t found any hard evidence/documentation to help me here, but if \
you can point me in the right direction, I will oblige and RTFM away.</div> <div \
class=""><br class=""> </div>
<div class="">Many thanks for your help!</div>
<div class=""><br class="">
</div>
<div class="">Cheers</div>
<div class="">FJ<br class="Apple-interchange-newline" style="font-family: Helvetica; \
font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; \
letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; \
text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; \
-webkit-text-stroke-width: 0px;"> <br class="Apple-interchange-newline">
</div>
<br class="">
</div>
</blockquote>
</div>
<br>
</div>
</div>
</body>
</html>
[prev in list] [next in list] [prev in thread] [next in thread]
Configure |
About |
News |
Add a list |
Sponsored by KoreLogic