[prev in list] [next in list] [prev in thread] [next in thread] 

List:       solr-user
Subject:    Re: SolrCloud indexing -- 2 collections, 2 indexes, sharing the same nodes possible?
From:       Susheel Kumar <susheel2777 () gmail ! com>
Date:       2017-08-30 17:40:58
Message-ID: CAJuqOKKKv4Rc+7BELDyL-57DFqKLjeBn53vfd7yDSEd-H5iqeg () mail ! gmail ! com
[Download RAW message or body]


1) As regards naming of the shards: Is using the same naming for the shards
o.k. in this constellation? I.e. does it create trouble to have e.g.
"Shard001", "Shard002", etc. in collection1 and "Shard001", "Shard002",
etc. as well in collection2?
>> The default naming convention for shards would be
"<collectionname>_shard#_replica#".  So complete name will be different
like coll1_shard1_replica1 and coll2_shard1_replica1

2) Performance: In my current single collection setup, I have 2 shards per
node. After creating the second collection, there will be 4 shards per
node. Do I have to edit the RAM per node value (raise the -m parameter when
starting the node)? In my case, I am quite sure that the collections will
never be queried simultaneously. So will the "running but idle" collection
slow me down?
>> Its up to you how you setup JVM.  You can have one JVM instance running
on port assume 8080 and have multiple shards/collections or you can setup
two JVM/solr instances on a node running on different ports like 8080 and
8081 etc. I would suggest to start and test with one JVM and setup multiple
collections until run into performance bottleneck and then split into JVM
with different heaps etc.



On Wed, Aug 30, 2017 at 12:42 PM, Johannes Knaus <Knaus@mpdl.mpg.de> wrote:

> Thank you, Susheel, for the quick response.
>
> So, that means that when I create a new collection, it shards will be
> newly created at each node, right?
> Thus, if I have two collections with
> numShards=38,
> maxShardsPerNode=2 and
> replicationFactor=2
> on my 38 nodes, then this would result in each node "hosting" 4 shards
> (two from each collection).
>
> If this is correct, I have two follow up questions:
>
> 1) As regards naming of the shards: Is using the same naming for the
> shards o.k. in this constellation? I.e. does it create trouble to have e.g.
> "Shard001", "Shard002", etc. in collection1 and "Shard001", "Shard002",
> etc. as well in collection2?
>
> 2) Performance: In my current single collection setup, I have 2 shards per
> node. After creating the second collection, there will be 4 shards per
> node. Do I have to edit the RAM per node value (raise the -m parameter when
> starting the node)? In my case, I am quite sure that the collections will
> never be queried simultaneously. So will the "running but idle" collection
> slow me down?
>
> Johannes
>
> -----Ursprüngliche Nachricht-----
> Von: Susheel Kumar [mailto:susheel2777@gmail.com]
> Gesendet: Mittwoch, 30. August 2017 17:36
> An: solr-user@lucene.apache.org
> Betreff: Re: SolrCloud indexing -- 2 collections, 2 indexes, sharing the
> same nodes possible?
>
> Yes, absolutely.  You can create as many as collections you need (like you
> would create table in relational world).
>
> On Wed, Aug 30, 2017 at 10:13 AM, Johannes Knaus <Knaus@mpdl.mpg.de>
> wrote:
>
> > I have a working SolrCloud-Setup with 38 nodes with a collection
> > spanning over these nodes with 2 shards per node and replication
> > factor 2 and a router field.
> >
> > Now I got some new data for indexing which has the same structure and
> > size as my existing index in the described collection.
> > However, although it has the same structure the new data to be indexed
> > should not be mixed with the old data.
> >
> > Do I have create another 38 new nodes and a new collection and index
> > the new data or is there a better / more efficient way I could use the
> > existing nodes?
> > Is it possible that the 2 collections could share the 38 nodes without
> > the indexes being mixed?
> >
> > Thanks for your help.
> >
> > Johannes
> >
>


[prev in list] [next in list] [prev in thread] [next in thread]