[prev in list] [next in list] [prev in thread] [next in thread]
List: lucene-dev
Subject: [jira] [Updated] (SOLR-12294) System collection - Lazy loading mechanism not working for custom Upda
From: "Johannes Brucher (JIRA)" <jira () apache ! org>
Date: 2018-04-30 14:48:00
Message-ID: JIRA.13156119.1525099186000.15307.1525099680248 () Atlassian ! JIRA
[Download RAW message or body]
[ https://issues.apache.org/jira/browse/SOLR-12294?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel \
]
Johannes Brucher updated SOLR-12294:
------------------------------------
Description:
Hi all,
I'm facing an issue regarding custom code inside a .system-collection and starting up \
a Solr Cloud cluster.
I thought, like its stated in the documentation, that in case using the .system \
collection custom code is lazy loaded, because it can happen that a collection that \
uses custom code is initialized before the .system collection is up and running.
I did all the necessary configuration and while debugging, I can see that the custom \
code is wrapped via a PluginBag$LazyPluginHolder. So far its seems good, but I still \
get Exceptions when starting the Solr Cloud with the following errors:
SolrException: Blob loading failed: .no active replica available for .system \
collection...
In my case I'm using custom code for a couple of UpdateProcessors. So it seems, that \
this lazy mechanism is not working well for UpdateProcessors.
Inside the calzz LazyPluginHolder the comment says:
"A class that loads plugins Lazily. When the get() method is invoked the Plugin is \
initialized and returned."
When a core is initialized and you have a custom UpdateProcessor, the get-method is \
invoked directly and the lazy loading mechanism tries to get the custom class from \
the MemClassLoader, but in most scenarios the system collection is not up and the \
above Exception is thrown...
So maybe it's the case that for UpdateProcessors while initializing a core, the \
routine is not implemented optimal for the lazy loading mechanism?
Here are the steps to reproduce the issue:
# Unpack solr 7.3.0
1.1 Add SOLR_OPTS="$SOLR_OPTS -Denable.runtime.lib=true" to solr.in.sh
1.2 Start Solr with -c option
# Setup .system collection:
2.1 Upload custom test jar --> curl -X POST -H 'Content-Type: \
application/octet-stream' --data-binary @<path to custom \
jar>/update-processor-0.0.1-SNAPSHOT.jar http://<your \
solr>/solr/.system/blob/test_blob 2.2 Alter maxShardsPerNode --> \
.../admin/collections?action=MODIFYCOLLECTION&collection=.system&maxShardsPerNode=2 \
2.2 Add Replica to .system collection --> \
.../admin/collections?action=ADDREPLICA&collection=.system&shard=shard1 # Setup test \
collection: 3.1 Upload test conf to ZK --> ./zkcli.sh -zkhost <your ZK host> -cmd \
upconfig -confdir <your config dir> -confname test_conf 3.2 Create a test1 \
collection with commented UP-chain inside solrconfig.xml via Admin UI 3.3 Add blob \
to test collection --> curl http://<your Solr>/solr/test1/config -H \
'Content-type:application/json' -d '\{"add-runtimelib": { "name":"test_blob", \
"version":1 }}' 3.4 Uncomment the UP-chain and upload test-conf again --> ./zkcli.sh \
-zkhost <your ZK host> -cmd upconfig -confdir <your config dir> -confname test_conf \
3.5 Reload test1 collection 3.6 Everything should work as expected now (no erros are \
shown) # Restart SOLR
4.1 Now you can see: SolrException: Blob loading failed: No active replica available \
for .system collection
Expected: The lazy loading mechanism should work for UpdateProcessors inside core \
init routine, but it isn't due to above Exception.
Sometimes you are lucky and the test1 collection will be initialize after the .system \
collection. But in ~90% of the time this isn't the case...
Let me know if you need further details here,
Johannes
was:
Hi all,
I'm facing an issue regarding custom code inside a .system-collection and starting up \
a Solr Cloud cluster.
I thought, like its stated in the documentation, that in case using the .system \
collection custom code is lazy loaded, because it can happen that a collection that \
uses custom code is initialized before the .system collection is up and running.
I did all the necessary configuration and while debugging, I can see that the custom \
code is wrapped via a PluginBag$LazyPluginHolder. So far its seems good, but I still \
get Exceptions when starting the Solr Cloud with the following errors:
SolrException: Blob loading failed: .no active replica available for .system \
collection...
In my case I'm using custom code for a couple of UpdateProcessors. So it seems, that \
this lazy mechanism is not working well for UpdateProcessors.
Inside the calzz LazyPluginHolder the comment says:
"A class that loads plugins Lazily. When the get() method is invoked the Plugin is \
initialized and returned."
When a core is initialized and you have a custom UpdateProcessor, the get-method is \
invoked directly and the lazy loading mechanism tries to get the custom class from \
the MemClassLoader, but in most scenarios the system collection is not up and the \
above Exception is thrown...
So maybe it's the case that for UpdateProcessors while initializing a core, the \
routine is not implemented optimal for the lazy loading mechanism?
Here are the steps to reproduce the issue:
# Unpack solr 7.3.0
1.1 Add SOLR_OPTS="$SOLR_OPTS -Denable.runtime.lib=true" to solr.in.sh
1.2 Start Solr with -c option
# Setup .system collection:
2.1 Upload custom test jar --> curl -X POST -H 'Content-Type: \
application/octet-stream' --data-binary @<path to custom \
jar>/update-processor-0.0.1-SNAPSHOT.jar http://<your \
solr>/solr/.system/blob/test_blob 2.2 Alter maxShardsPerNode --> \
.../admin/collections?action=MODIFYCOLLECTION&collection=.system&maxShardsPerNode=2 \
2.2 Add Replica to .system collection --> \
.../admin/collections?action=ADDREPLICA&collection=.system&shard=shard1 # Setup test \
collection: 3.1 Upload test conf to ZK --> ./zkcli.sh -zkhost <your ZK host> -cmd \
upconfig -confdir <your config dir> -confname test_conf 3.2 Create a test1 \
collection with commented UP-chain inside solrconfig.xml via Admin UI 3.3 Add blob \
to test collection --> curl http://<your Solr>/solr/test1/config -H \
'Content-type:application/json' -d '\{"add-runtimelib": { "name":"test_blob", \
"version":1 }}' 3.4 Uncomment the UP-chain and upload test-conf again --> ./zkcli.sh \
-zkhost <your ZK host> -cmd upconfig -confdir <your config dir> -confname test_conf \
3.5 Reload test1 collection 3.6 Everything should work as expected now (no erros are \
shown) # Restart SOLR
4.1 Now you can see: SolrException: Blob loading failed: No active replica available \
for .system collection
Expected: The lazy loading mechanism should work for UpdateProcessors inside core \
init routine, but it isn't due to above Exception.
Sometimes you are lucky and the test1 collection will be initialize after the .system \
collection. But in ~90% of the time this isn't the case...
Let me know if you need further details here,
Johannes
> System collection - Lazy loading mechanism not working for custom UpdateProcessors
> ----------------------------------------------------------------------------------
>
> Key: SOLR-12294
> URL: https://issues.apache.org/jira/browse/SOLR-12294
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
> Components: UpdateRequestProcessors
> Affects Versions: 7.3
> Reporter: Johannes Brucher
> Priority: Critical
> Attachments: no_active_replica_available.png, schema.xml, solrconfig.xml, \
> update-processor-0.0.1-SNAPSHOT.jar
>
> Hi all,
> I'm facing an issue regarding custom code inside a .system-collection and starting \
> up a Solr Cloud cluster. I thought, like its stated in the documentation, that in \
> case using the .system collection custom code is lazy loaded, because it can happen \
> that a collection that uses custom code is initialized before the .system \
> collection is up and running. I did all the necessary configuration and while \
> debugging, I can see that the custom code is wrapped via a \
> PluginBag$LazyPluginHolder. So far its seems good, but I still get Exceptions when \
> starting the Solr Cloud with the following errors:
> SolrException: Blob loading failed: .no active replica available for .system \
> collection... In my case I'm using custom code for a couple of UpdateProcessors. So \
> it seems, that this lazy mechanism is not working well for UpdateProcessors. Inside \
> the calzz LazyPluginHolder the comment says: "A class that loads plugins Lazily. \
> When the get() method is invoked the Plugin is initialized and returned." When a \
> core is initialized and you have a custom UpdateProcessor, the get-method is \
> invoked directly and the lazy loading mechanism tries to get the custom class from \
> the MemClassLoader, but in most scenarios the system collection is not up and the \
> above Exception is thrown... So maybe it's the case that for UpdateProcessors while \
> initializing a core, the routine is not implemented optimal for the lazy loading \
> mechanism?
> Here are the steps to reproduce the issue:
> # Unpack solr 7.3.0
> 1.1 Add SOLR_OPTS="$SOLR_OPTS -Denable.runtime.lib=true" to solr.in.sh
> 1.2 Start Solr with -c option
> # Setup .system collection:
> 2.1 Upload custom test jar --> curl -X POST -H 'Content-Type: \
> application/octet-stream' --data-binary @<path to custom \
> jar>/update-processor-0.0.1-SNAPSHOT.jar http://<your \
> solr>/solr/.system/blob/test_blob 2.2 Alter maxShardsPerNode --> \
> .../admin/collections?action=MODIFYCOLLECTION&collection=.system&maxShardsPerNode=2 \
> 2.2 Add Replica to .system collection --> \
> .../admin/collections?action=ADDREPLICA&collection=.system&shard=shard1 # Setup \
> test collection: 3.1 Upload test conf to ZK --> ./zkcli.sh -zkhost <your ZK host> \
> -cmd upconfig -confdir <your config dir> -confname test_conf 3.2 Create a test1 \
> collection with commented UP-chain inside solrconfig.xml via Admin UI 3.3 Add blob \
> to test collection --> curl http://<your Solr>/solr/test1/config -H \
> 'Content-type:application/json' -d '\{"add-runtimelib": { "name":"test_blob", \
> "version":1 }}' 3.4 Uncomment the UP-chain and upload test-conf again --> \
> ./zkcli.sh -zkhost <your ZK host> -cmd upconfig -confdir <your config dir> \
> -confname test_conf 3.5 Reload test1 collection
> 3.6 Everything should work as expected now (no erros are shown)
> # Restart SOLR
> 4.1 Now you can see: SolrException: Blob loading failed: No active replica \
> available for .system collection
> Expected: The lazy loading mechanism should work for UpdateProcessors inside core \
> init routine, but it isn't due to above Exception. Sometimes you are lucky and the \
> test1 collection will be initialize after the .system collection. But in ~90% of \
> the time this isn't the case... Let me know if you need further details here,
>
> Johannes
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[prev in list] [next in list] [prev in thread] [next in thread]
Configure |
About |
News |
Add a list |
Sponsored by KoreLogic