'Re: How to sync lib directory in SolrCloud?'

[prev in list] [next in list] [prev in thread] [next in thread] 

List:       solr-user
Subject:    Re: How to sync lib directory in SolrCloud?
From:       Timothy Potter <thelabdude () gmail ! com>
Date:       2014-07-31 17:44:23
Message-ID: CAJt9WnhR0uqu3MUVRiP7t1xWha4aQ-+7cFnzAhQN4MAJu0K-iw () mail ! gmail ! com
[Download RAW message or body]

You'll need to scp the JAR files to all nodes in the cluster. ZK is
not a great distribution mechanism for large binary files since it has
a 1MB znode size limit (by default)

On Thu, Jul 31, 2014 at 10:26 AM, P Williams
<williams.tricia.list@gmail.com> wrote:
> Hi,
> 
> I have an existing collection that I'm trying to add to a new SolrCloud.
> This collection has all the normal files in conf but also has a lib
> directory to support the filters schema.xml uses.
> 
> wget
> https://github.com/projectblacklight/blacklight-jetty/archive/v4.9.0.zip
> unzip v4.9.0.zip
> 
> I add the configuration to Zookeeper
> 
> cd /solr-4.9.0/example/scripts
> cloud-scripts/zkcli.sh -cmd upconfig -confname blacklight -zkhost
> zk1:2181,zk2:2181,zk3:2181 -confdir
> ~/blacklight-jetty-4.9.0/solr/blacklight-core/conf/
> 
> I try to create the collection
> curl "
> http://solr1:8080/solr/admin/collections?action=CREATE&name=blacklight&numShards=3&collection.configName=blacklight&replicationFactor=2&maxShardsPerNode=2
>  "
> 
> but it looks like the jars in the lib directory aren't available and this
> is what is causing my collection creation to fail.  I guess this makes
> sense because it's not one of the files that I added to Zookeeper to share.
> How do I share the lib directory via Zookeeper?
> 
> Thanks,
> Tricia
> 
> [pjenkins@solr1 scripts]$ cloud-scripts/zkcli.sh -cmd upconfig -zkhost
> zk1:2181,zk2:2181,zk3:2181 -confdir
> ~/blacklight-jetty-4.9.0/solr/blacklight-core/conf/ -confname blacklight
> INFO  - 2014-07-31 09:28:06.289; org.apache.zookeeper.Environment; Client
> environment:zookeeper.version=3.4.6-1569965, built on 02/20/2014 09:09 GMT
> INFO  - 2014-07-31 09:28:06.292; org.apache.zookeeper.Environment; Client
> environment:host.name=solr1.library.ualberta.ca
> INFO  - 2014-07-31 09:28:06.295; org.apache.zookeeper.Environment; Client
> environment:java.version=1.7.0_65
> INFO  - 2014-07-31 09:28:06.295; org.apache.zookeeper.Environment; Client
> environment:java.vendor=Oracle Corporation
> INFO  - 2014-07-31 09:28:06.295; org.apache.zookeeper.Environment; Client
> environment:java.home=/usr/lib/jvm/java-1.7.0-openjdk-1.7.0.65.x86_64/jre
> INFO  - 2014-07-31 09:28:06.295; org.apache.zookeeper.Environment; Client
> environment:java.class.path=cloud-scripts/../../solr-webapp/webapp/WEB-INF/lib/hppc- \
> 0.5.2.jar:cloud-scripts/../../solr-webapp/webapp/WEB-INF/lib/hadoop-auth-2.2.0.jar:c \
> loud-scripts/../../solr-webapp/webapp/WEB-INF/lib/asm-commons-4.1.jar:cloud-scripts/ \
> ../../solr-webapp/webapp/WEB-INF/lib/lucene-queries-4.9.0.jar:cloud-scripts/../../so \
> lr-webapp/webapp/WEB-INF/lib/lucene-memory-4.9.0.jar:cloud-scripts/../../solr-webapp \
> /webapp/WEB-INF/lib/commons-codec-1.9.jar:cloud-scripts/../../solr-webapp/webapp/WEB \
> -INF/lib/lucene-join-4.9.0.jar:cloud-scripts/../../solr-webapp/webapp/WEB-INF/lib/jo \
> da-time-2.2.jar:cloud-scripts/../../solr-webapp/webapp/WEB-INF/lib/lucene-codecs-4.9 \
> .0.jar:cloud-scripts/../../solr-webapp/webapp/WEB-INF/lib/lucene-analyzers-common-4. \
> 9.0.jar:cloud-scripts/../../solr-webapp/webapp/WEB-INF/lib/hadoop-common-2.2.0.jar:c \
> loud-scripts/../../solr-webapp/webapp/WEB-INF/lib/httpmime-4.3.1.jar:cloud-scripts/. \
> ./../solr-webapp/webapp/WEB-INF/lib/hadoop-hdfs-2.2.0.jar:cloud-scripts/../../solr-w \
> ebapp/webapp/WEB-INF/lib/noggit-0.5.jar:cloud-scripts/../../solr-webapp/webapp/WEB-I \
> NF/lib/lucene-analyzers-kuromoji-4.9.0.jar:cloud-scripts/../../solr-webapp/webapp/WE \
> B-INF/lib/guava-14.0.1.jar:cloud-scripts/../../solr-webapp/webapp/WEB-INF/lib/common \
> s-configuration-1.6.jar:cloud-scripts/../../solr-webapp/webapp/WEB-INF/lib/lucene-ex \
> pressions-4.9.0.jar:cloud-scripts/../../solr-webapp/webapp/WEB-INF/lib/lucene-highli \
> ghter-4.9.0.jar:cloud-scripts/../../solr-webapp/webapp/WEB-INF/lib/hadoop-annotation \
> s-2.2.0.jar:cloud-scripts/../../solr-webapp/webapp/WEB-INF/lib/asm-4.1.jar:cloud-scr \
> ipts/../../solr-webapp/webapp/WEB-INF/lib/dom4j-1.6.1.jar:cloud-scripts/../../solr-w \
> ebapp/webapp/WEB-INF/lib/commons-io-2.3.jar:cloud-scripts/../../solr-webapp/webapp/W \
> EB-INF/lib/zookeeper-3.4.6.jar:cloud-scripts/../../solr-webapp/webapp/WEB-INF/lib/sp \
> atial4j-0.4.1.jar:cloud-scripts/../../solr-webapp/webapp/WEB-INF/lib/httpcore-4.3.ja \
> r:cloud-scripts/../../solr-webapp/webapp/WEB-INF/lib/protobuf-java-2.5.0.jar:cloud-s \
> cripts/../../solr-webapp/webapp/WEB-INF/lib/lucene-spatial-4.9.0.jar:cloud-scripts/. \
> ./../solr-webapp/webapp/WEB-INF/lib/lucene-grouping-4.9.0.jar:cloud-scripts/../../so \
> lr-webapp/webapp/WEB-INF/lib/lucene-misc-4.9.0.jar:cloud-scripts/../../solr-webapp/w \
> ebapp/WEB-INF/lib/lucene-suggest-4.9.0.jar:cloud-scripts/../../solr-webapp/webapp/WE \
> B-INF/lib/lucene-analyzers-phonetic-4.9.0.jar:cloud-scripts/../../solr-webapp/webapp \
> /WEB-INF/lib/lucene-core-4.9.0.jar:cloud-scripts/../../solr-webapp/webapp/WEB-INF/li \
> b/commons-cli-1.2.jar:cloud-scripts/../../solr-webapp/webapp/WEB-INF/lib/solr-core-4 \
> .9.0.jar:cloud-scripts/../../solr-webapp/webapp/WEB-INF/lib/solr-solrj-4.9.0.jar:clo \
> ud-scripts/../../solr-webapp/webapp/WEB-INF/lib/antlr-runtime-3.5.jar:cloud-scripts/ \
> ../../solr-webapp/webapp/WEB-INF/lib/concurrentlinkedhashmap-lru-1.2.jar:cloud-scrip \
> ts/../../solr-webapp/webapp/WEB-INF/lib/lucene-queryparser-4.9.0.jar:cloud-scripts/. \
> ./../solr-webapp/webapp/WEB-INF/lib/org.restlet.ext.servlet-2.1.1.jar:cloud-scripts/ \
> ../../solr-webapp/webapp/WEB-INF/lib/commons-fileupload-1.2.1.jar:cloud-scripts/../. \
> ./solr-webapp/webapp/WEB-INF/lib/wstx-asl-3.2.7.jar:cloud-scripts/../../solr-webapp/ \
> webapp/WEB-INF/lib/commons-lang-2.6.jar:cloud-scripts/../../solr-webapp/webapp/WEB-I \
> NF/lib/httpclient-4.3.1.jar:cloud-scripts/../../solr-webapp/webapp/WEB-INF/lib/org.r \
> estlet-2.1.1.jar:cloud-scripts/../../lib/ext/jcl-over-slf4j-1.7.6.jar:cloud-scripts/ \
> ../../lib/ext/jul-to-slf4j-1.7.6.jar:cloud-scripts/../../lib/ext/slf4j-log4j12-1.7.6 \
> .jar:cloud-scripts/../../lib/ext/log4j-1.2.17.jar:cloud-scripts/../../lib/ext/slf4j-api-1.7.6.jar
>  INFO  - 2014-07-31 09:28:06.296; org.apache.zookeeper.Environment; Client
> environment:java.library.path=/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib
>  INFO  - 2014-07-31 09:28:06.296; org.apache.zookeeper.Environment; Client
> environment:java.io.tmpdir=/tmp
> INFO  - 2014-07-31 09:28:06.296; org.apache.zookeeper.Environment; Client
> environment:java.compiler=<NA>
> INFO  - 2014-07-31 09:28:06.296; org.apache.zookeeper.Environment; Client
> environment:os.name=Linux
> INFO  - 2014-07-31 09:28:06.297; org.apache.zookeeper.Environment; Client
> environment:os.arch=amd64
> INFO  - 2014-07-31 09:28:06.297; org.apache.zookeeper.Environment; Client
> environment:os.version=2.6.32-431.20.5.el6.x86_64
> INFO  - 2014-07-31 09:28:06.297; org.apache.zookeeper.Environment; Client
> environment:user.name=pjenkins
> INFO  - 2014-07-31 09:28:06.297; org.apache.zookeeper.Environment; Client
> environment:user.home=/home/pjenkins
> INFO  - 2014-07-31 09:28:06.297; org.apache.zookeeper.Environment; Client
> environment:user.dir=/home/pjenkins/solr-4.9.0/example/scripts
> INFO  - 2014-07-31 09:28:06.299; org.apache.zookeeper.ZooKeeper; Initiating
> client connection, connectString=zk1:2181,zk2:2181,zk3:2181
> sessionTimeout=30000
> watcher=org.apache.solr.common.cloud.ConnectionManager@214fe17
> INFO  - 2014-07-31 09:28:06.328;
> org.apache.solr.common.cloud.ConnectionManager; Waiting for client to
> connect to ZooKeeper
> INFO  - 2014-07-31 09:28:06.335;
> org.apache.zookeeper.ClientCnxn$SendThread; Opening socket connection to
> server zk1.library.ualberta.ca/129.128.217.69:2181. Will not attempt to
> authenticate using SASL (unknown error)
> INFO  - 2014-07-31 09:28:06.341;
> org.apache.zookeeper.ClientCnxn$SendThread; Socket connection established
> to zk1.library.ualberta.ca/129.128.217.69:2181, initiating session
> INFO  - 2014-07-31 09:28:06.501;
> org.apache.zookeeper.ClientCnxn$SendThread; Session establishment complete
> on server zk1.library.ualberta.ca/129.128.217.69:2181, sessionid =
> 0x14782b81486000b, negotiated timeout = 30000
> INFO  - 2014-07-31 09:28:06.503;
> org.apache.solr.common.cloud.ConnectionManager; Watcher
> org.apache.solr.common.cloud.ConnectionManager@214fe17
> name:ZooKeeperConnection Watcher:zk1:2181,zk2:2181,zk3:2181 got event
> WatchedEvent state:SyncConnected type:None path:null path:null type:None
> INFO  - 2014-07-31 09:28:06.504;
> org.apache.solr.common.cloud.ConnectionManager; Client is connected to
> ZooKeeper
> INFO  - 2014-07-31 09:28:06.533; org.apache.solr.common.cloud.SolrZkClient;
> makePath: /configs/blacklight/solrconfig.xml
> INFO  - 2014-07-31 09:28:06.761; org.apache.solr.common.cloud.SolrZkClient;
> makePath: /configs/blacklight/spellings.txt
> INFO  - 2014-07-31 09:28:06.815; org.apache.solr.common.cloud.SolrZkClient;
> makePath: /configs/blacklight/mapping-ISOLatin1Accent.txt
> INFO  - 2014-07-31 09:28:06.894; org.apache.solr.common.cloud.SolrZkClient;
> makePath: /configs/blacklight/xslt/example_rss.xsl
> INFO  - 2014-07-31 09:28:06.966; org.apache.solr.common.cloud.SolrZkClient;
> makePath: /configs/blacklight/xslt/example_atom.xsl
> INFO  - 2014-07-31 09:28:07.019; org.apache.solr.common.cloud.SolrZkClient;
> makePath: /configs/blacklight/xslt/example.xsl
> INFO  - 2014-07-31 09:28:07.098; org.apache.solr.common.cloud.SolrZkClient;
> makePath: /configs/blacklight/xslt/luke.xsl
> INFO  - 2014-07-31 09:28:07.172; org.apache.solr.common.cloud.SolrZkClient;
> makePath: /configs/blacklight/scripts.conf
> INFO  - 2014-07-31 09:28:07.200; org.apache.solr.common.cloud.SolrZkClient;
> makePath: /configs/blacklight/elevate.xml
> INFO  - 2014-07-31 09:28:07.248; org.apache.solr.common.cloud.SolrZkClient;
> makePath: /configs/blacklight/stopwords.txt
> INFO  - 2014-07-31 09:28:07.331; org.apache.solr.common.cloud.SolrZkClient;
> makePath: /configs/blacklight/admin-extra.html
> INFO  - 2014-07-31 09:28:07.378; org.apache.solr.common.cloud.SolrZkClient;
> makePath: /configs/blacklight/synonyms.txt
> INFO  - 2014-07-31 09:28:07.425; org.apache.solr.common.cloud.SolrZkClient;
> makePath: /configs/blacklight/stopwords_en.txt
> INFO  - 2014-07-31 09:28:07.502; org.apache.solr.common.cloud.SolrZkClient;
> makePath: /configs/blacklight/protwords.txt
> INFO  - 2014-07-31 09:28:07.586; org.apache.solr.common.cloud.SolrZkClient;
> makePath: /configs/blacklight/schema.xml
> INFO  - 2014-07-31 09:28:07.732; org.apache.zookeeper.ZooKeeper; Session:
> 0x14782b81486000b closed
> INFO  - 2014-07-31 09:28:07.732;
> org.apache.zookeeper.ClientCnxn$EventThread; EventThread shut down
> [pjenkins@solr1 scripts]$ curl "
> http://solr1:8080/solr/admin/collections?action=CREATE&name=blacklight&numShards=3&collection.configName=blacklight&replicationFactor=2&maxShardsPerNode=2
>  "
> <?xml version="1.0" encoding="UTF-8"?>
> <response>
> <lst name="responseHeader"><int name="status">0</int><int
> name="QTime">6146</int></lst><lst
> name="failure"><str>org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException:Error
>  CREATEing SolrCore 'blacklight_shard3_replica2': Unable to create core:
> blacklight_shard3_replica2 Caused by:
> solr.ICUFoldingFilterFactory</str><str>org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException:Error
>  CREATEing SolrCore 'blacklight_shard2_replica2': Unable to create core:
> blacklight_shard2_replica2 Caused by:
> solr.ICUFoldingFilterFactory</str><str>org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException:Error
>  CREATEing SolrCore 'blacklight_shard1_replica2': Unable to create core:
> blacklight_shard1_replica2 Caused by:
> solr.ICUFoldingFilterFactory</str><str>org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException:Error
>  CREATEing SolrCore 'blacklight_shard2_replica1': Unable to create core:
> blacklight_shard2_replica1 Caused by:
> solr.ICUFoldingFilterFactory</str><str>org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException:Error
>  CREATEing SolrCore 'blacklight_shard1_replica1': Unable to create core:
> blacklight_shard1_replica1 Caused by:
> solr.ICUFoldingFilterFactory</str><str>org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException:Error
>  CREATEing SolrCore 'blacklight_shard3_replica1': Unable to create core:
> blacklight_shard3_replica1 Caused by:
> solr.ICUFoldingFilterFactory</str></lst>
> </response>


[prev in list] [next in list] [prev in thread] [next in thread]
Configure | About | News | Add a list | Sponsored by KoreLogic