[prev in list] [next in list] [prev in thread] [next in thread] 

List:       lucene-dev
Subject:    [jira] (SOLR-9861) Support Cross Data Center Replication under HDFS
From:       "Kevin Risden (JIRA)" <jira () apache ! org>
Date:       2017-01-31 21:40:52
Message-ID: JIRA.13027742.1481623340000.2833.1485898852544 () Atlassian ! JIRA
[Download RAW message or body]


    [ https://issues.apache.org/jira/browse/SOLR-9861?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15847554#comment-15847554 \
] 

Kevin Risden commented on SOLR-9861:
------------------------------------

I looked into this some and just sharing what I found in case it helps:
* CDCR has CdcrUpdateLog and CdcrTransactionLog which do a bulk of the CDCR work
* HDFS has HdfsUpdateLog and HdfsTransactionLog which do a bulk of the HDFS work
* CdcrUpdateLog and HdfsUpdateLog both extend the UpdateLog class
* CdcrTransactionLog and CdcrUpdateLog both extend the TransactionLog class
* CdcrUpdateLog and CdcrTransactionLog both deal with Linux files

The items above make it hard for the existing CDCR implementation classes to reuse \
the HDFS implementation classes. It would probably be useful to refactor some first \
to allow for more reuse between classes. The UpdateLog class is pretty big and its \
not clear what all the pieces are for. There are inner classes and enums within the \
UpdateLog class used elsewhere.

> Support Cross Data Center Replication under HDFS
> ------------------------------------------------
> 
> Key: SOLR-9861
> URL: https://issues.apache.org/jira/browse/SOLR-9861
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public) 
> Components: CDCR
> Affects Versions: 6.3
> Reporter: Gero Willmes
> 
> Currently Solr Cloud does not support CDCR under HDFS (according to \
> http://search-lucene.com/m/Solr/eHNlLSriaVCpuU1?subj=Re+Using+Solr+CDCR+with+HdfsDirectoryFactory). \
>  If the CDCR configuration is performed according to
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=62687462
> and at the same time the HdfsDirectoryFactory is configured on both Source cluster \
> and Target cluster according to \
> https://cwiki.apache.org/confluence/display/solr/Running+Solr+on+HDFS it leads to \
> the following error message during reload of the collection "collection_source" of \
> the source cluster: {code}
> 2016-11-23 12:05:35.604 ERROR (qtp1134712904-8045) [c:collection_source s:shard1 \
> r:core_node1 x:collection_source_shard1_replica1] o.a.s.s.HttpSolrCall \
> null:org.apache.solr.common.SolrException: Error handling 'reload' action at \
> org.apache.solr.handler.admin.CoreAdminOperation$3.call(CoreAdminOperation.java:150)
>  at org.apache.solr.handler.admin.CoreAdminHandler$CallInfo.call(CoreAdminHandler.java:367)
>  at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:158)
>  at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:156)
>  at org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:663)
> at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:445)
> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:257)
> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:208)
> at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1668)
>  at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:581)
> at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
> at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)
> at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:226)
>  at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1160)
>  at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:511)
> at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
> at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1092)
>  at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
> at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:213)
>  at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:119)
>  at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
> at org.eclipse.jetty.server.Server.handle(Server.java:518)
> at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:308)
> at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:244)
> at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273)
>  at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)
> at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
> at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceAndRun(ExecuteProduceConsume.java:246)
>  at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:156)
>  at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:654)
>  at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: org.apache.solr.common.SolrException: Unable to reload core \
> [collection_source_shard1_replica1] at \
> org.apache.solr.core.CoreContainer.reload(CoreContainer.java:920) at \
> org.apache.solr.handler.admin.CoreAdminOperation$3.call(CoreAdminOperation.java:148)
>                 
> ... 31 more
> Caused by: org.apache.solr.common.SolrException: Solr instance is not configured \
> with the cdcr update log. {code}
> However, the solr instance was configured correctly. CDCR should work in the same \
> way with HdfsDirectoryFactory as it works with StandardDirectoryFactory or \
> NRTCachingDirectoryFactory.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


[prev in list] [next in list] [prev in thread] [next in thread]