[prev in list] [next in list] [prev in thread] [next in thread] 

List:       solr-user
Subject:    Re: Slow response times when using query paramters
From:       Jan_Høydahl <jan.asf () cominvent ! com>
Date:       2021-09-21 20:58:31
Message-ID: DBFEB796-379F-4F80-9F9F-837D67207CC9 () cominvent ! com
[Download RAW message or body]


I'm sure you have a usecase for the huge "fl", but here you force Solr to evaluate \
tons of semi-complex queries on each returned document, and I'm sure it is the result \
enrichment that adds time to your query. The fact that you ask for 100 results only \
makes it worse since all these needs to be evaluated for 100 docs.

I recently had a similar challence with a customer, where we used "fl" to calculate a \
normalized score to be returned on every match. It was super slow, so we instead \
moved that processing to the client side.

So perhaps if you instead return the 20 raw fields needed for the permission \
calculations, you can move that part to the client side?

Another option is of course to dive into the "slow" code used by "fl" and find ways \
to optimize it.

Could also be that a custom Document Transformer \
(https://solr.apache.org/guide/8_9/transforming-result-documents.html) would perform \
better at this task than the built-in generic functions?

Jan

> 21. sep. 2021 kl. 14:37 skrev Andreas Gusenbauer [CELUM] \
> <Andreas.Gusenbauer@celum.com>: 
> Hi all,
> 
> We encountered slow response times when querying solr with additional raw queries.
> The response times decrease by a factor of 10 after deleting and rebuilding the \
> index from scratch. After some minor modifications , the queries response times \
> increase by a factor of 10 again. 
> Some information: 
> solr index size is about 7GB
> 3,1 Mio documents
> Document/filter cache seem to be correctly used (hit count increases when we \
> execute the same query twice) We can't use the result query cache for our query \
>                 (see below)
> q: containers:80
> fq: (createdBy:106 assetStatus:0) ((allowPermissions:80 allowPermissions:22 \
>                 allowPermissions:106) -assetStatus:0)
> rows: 100
> 20 Raw query parameters ( e.g \
> qq_viewFieldPermission_customFields_202={!v='(((viewFieldPermission_customFields_202:80 \
> viewFieldPermission_customFields_202:22 \
> viewFieldPermission_customFields_202:106)))'}) 20 additional fl values for raw \
> query parameters ( e.g \
> permission_customFields_date_202:exists(query($qq_viewFieldPermission_customFields_202)) \
>                 )
> wt: json ( but xml does not make a difference)
> 
> 
> Request:
> <solr>/solr/assets/select?fl=id%2Cname%2Cpermission_customFields_date_200%3Aexists(q \
> uery(%24qq_viewFieldPermission_customFields_200))%2Cpermission_customFields_date_201 \
> %3Aexists(query(%24qq_viewFieldPermission_customFields_201))%2Cpermission_customFiel \
> ds_date_202%3Aexists(query(%24qq_viewFieldPermission_customFields_202))%2Cpermission \
> _customFields_date_207%3Aexists(query(%24qq_viewFieldPermission_customFields_207))%2 \
> Cpermission_customFields_date_208%3Aexists(query(%24qq_viewFieldPermission_customFie \
> lds_208))%2Cpermission_customFields_date_209%3Aexists(query(%24qq_viewFieldPermissio \
> n_customFields_209))%2Cpermission_customFields_date_216%3Aexists(query(%24qq_viewFie \
> ldPermission_customFields_216))%2Cpermission_customFields_date_217%3Aexists(query(%2 \
> 4qq_viewFieldPermission_customFields_217))%2Cpermission_customFields_date_218%3Aexis \
> ts(query(%24qq_viewFieldPermission_customFields_218))%2Cpermission_customFields_date \
> _219%3Aexists(query(%24qq_viewFieldPermission_customFields_219))%2Cpermission_custom \
> Fields_date_222%3Aexists(query(%24qq_viewFieldPermission_customFields_222))%2Cpermis \
> sion_customFields_date_223%3Aexists(query(%24qq_viewFieldPermission_customFields_223 \
> ))%2Cpermission_customFields_date_160%3Aexists(query(%24qq_viewFieldPermission_custo \
> mFields_160))%2Cpermission_customFields_date_224%3Aexists(query(%24qq_viewFieldPermi \
> ssion_customFields_224))%2Cpermission_customFields_date_225%3Aexists(query(%24qq_vie \
> wFieldPermission_customFields_225))%2Cpermission_customFields_date_226%3Aexists(quer \
> y(%24qq_viewFieldPermission_customFields_226))%2Cpermission_customFields_date_100%3A \
> exists(query(%24qq_viewFieldPermission_customFields_100))%2Cpermission_customFields_ \
> date_231%3Aexists(query(%24qq_viewFieldPermission_customFields_231))%2Cpermission_cu \
> stomFields_date_170%3Aexists(query(%24qq_viewFieldPermission_customFields_170))%2Cpe \
> rmission_customFields_date_235%3Aexists(query(%24qq_viewFieldPermission_customFields \
> _235))&fq=(createdBy%3A106%20assetStatus%3A0)%20((allowPermissions%3A80%20allowPermi \
> ssions%3A22%20allowPermissions%3A106)%20-assetStatus%3A0)&q=containers%3A80&qq_viewF \
> ieldPermission_customFields_100=%7B!v%3D%27(((viewFieldPermission_customFields_100%3 \
> A80%20viewFieldPermission_customFields_100%3A22%20viewFieldPermission_customFields_1 \
> 00%3A106)))%27%7D&qq_viewFieldPermission_customFields_110=%7B!v%3D%27(((viewFieldPer \
> mission_customFields_110%3A80%20viewFieldPermission_customFields_110%3A22%20viewFiel \
> dPermission_customFields_110%3A106)))%27%7D&qq_viewFieldPermission_customFields_120= \
> %7B!v%3D%27(((viewFieldPermission_customFields_120%3A80%20viewFieldPermission_custom \
> Fields_120%3A22%20viewFieldPermission_customFields_120%3A106)))%27%7D&qq_viewFieldPe \
> rmission_customFields_130=%7B!v%3D%27(((viewFieldPermission_customFields_130%3A80%20 \
> viewFieldPermission_customFields_130%3A22%20viewFieldPermission_customFields_130%3A1 \
> 06)))%27%7D&qq_viewFieldPermission_customFields_160=%7B!v%3D%27(((viewFieldPermissio \
> n_customFields_160%3A80%20viewFieldPermission_customFields_160%3A22%20viewFieldPermi \
> ssion_customFields_160%3A106)))%27%7D&qq_viewFieldPermission_customFields_170=%7B!v% \
> 3D%27(((viewFieldPermission_customFields_170%3A80%20viewFieldPermission_customFields \
> _170%3A22%20viewFieldPermission_customFields_170%3A106)))%27%7D&qq_viewFieldPermissi \
> on_customFields_180=%7B!v%3D%27(((viewFieldPermission_customFields_180%3A80%20viewFi \
> eldPermission_customFields_180%3A22%20viewFieldPermission_customFields_180%3A106)))% \
> 27%7D&qq_viewFieldPermission_customFields_190=%7B!v%3D%27(((viewFieldPermission_cust \
> omFields_190%3A80%20viewFieldPermission_customFields_190%3A22%20viewFieldPermission_ \
> customFields_190%3A106)))%27%7D&qq_viewFieldPermission_customFields_200=%7B!v%3D%27( \
> ((viewFieldPermission_customFields_200%3A80%20viewFieldPermission_customFields_200%3 \
> A22%20viewFieldPermission_customFields_200%3A106)))%27%7D&qq_viewFieldPermission_cus \
> tomFields_201=%7B!v%3D%27(((viewFieldPermission_customFields_201%3A80%20viewFieldPer \
> mission_customFields_201%3A22%20viewFieldPermission_customFields_201%3A106)))%27%7D& \
> qq_viewFieldPermission_customFields_202=%7B!v%3D%27(((viewFieldPermission_customFiel \
> ds_202%3A80%20viewFieldPermission_customFields_202%3A22%20viewFieldPermission_custom \
> Fields_202%3A106)))%27%7D&qq_viewFieldPermission_customFields_207=%7B!v%3D%27(((view \
> FieldPermission_customFields_207%3A80%20viewFieldPermission_customFields_207%3A22%20 \
> viewFieldPermission_customFields_207%3A106)))%27%7D&qq_viewFieldPermission_customFie \
> lds_208=%7B!v%3D%27(((viewFieldPermission_customFields_208%3A80%20viewFieldPermissio \
> n_customFields_208%3A22%20viewFieldPermission_customFields_208%3A106)))%27%7D&qq_vie \
> wFieldPermission_customFields_209=%7B!v%3D%27(((viewFieldPermission_customFields_209 \
> %3A80%20viewFieldPermission_customFields_209%3A22%20viewFieldPermission_customFields \
> _209%3A106)))%27%7D&qq_viewFieldPermission_customFields_216=%7B!v%3D%27(((viewFieldP \
> ermission_customFields_216%3A80%20viewFieldPermission_customFields_216%3A22%20viewFi \
> eldPermission_customFields_216%3A106)))%27%7D&qq_viewFieldPermission_customFields_21 \
> 7=%7B!v%3D%27(((viewFieldPermission_customFields_217%3A80%20viewFieldPermission_cust \
> omFields_217%3A22%20viewFieldPermission_customFields_217%3A106)))%27%7D&qq_viewField \
> Permission_customFields_218=%7B!v%3D%27(((viewFieldPermission_customFields_218%3A80% \
> 20viewFieldPermission_customFields_218%3A22%20viewFieldPermission_customFields_218%3 \
> A106)))%27%7D&qq_viewFieldPermission_customFields_219=%7B!v%3D%27(((viewFieldPermiss \
> ion_customFields_219%3A80%20viewFieldPermission_customFields_219%3A22%20viewFieldPer \
> mission_customFields_219%3A106)))%27%7D&qq_viewFieldPermission_customFields_222=%7B! \
> v%3D%27(((viewFieldPermission_customFields_222%3A80%20viewFieldPermission_customFiel \
> ds_222%3A22%20viewFieldPermission_customFields_222%3A106)))%27%7D&qq_viewFieldPermis \
> sion_customFields_223=%7B!v%3D%27(((viewFieldPermission_customFields_223%3A80%20view \
> FieldPermission_customFields_223%3A22%20viewFieldPermission_customFields_223%3A106)) \
> )%27%7D&qq_viewFieldPermission_customFields_224=%7B!v%3D%27(((viewFieldPermission_cu \
> stomFields_224%3A80%20viewFieldPermission_customFields_224%3A22%20viewFieldPermissio \
> n_customFields_224%3A106)))%27%7D&qq_viewFieldPermission_customFields_225=%7B!v%3D%2 \
> 7(((viewFieldPermission_customFields_225%3A80%20viewFieldPermission_customFields_225 \
> %3A22%20viewFieldPermission_customFields_225%3A106)))%27%7D&qq_viewFieldPermission_c \
> ustomFields_226=%7B!v%3D%27(((viewFieldPermission_customFields_226%3A80%20viewFieldP \
> ermission_customFields_226%3A22%20viewFieldPermission_customFields_226%3A106)))%27%7 \
> D&qq_viewFieldPermission_customFields_231=%7B!v%3D%27(((viewFieldPermission_customFi \
> elds_231%3A80%20viewFieldPermission_customFields_231%3A22%20viewFieldPermission_cust \
> omFields_231%3A106)))%27%7D&qq_viewFieldPermission_customFields_235=%7B!v%3D%27(((vi \
> ewFieldPermission_customFields_235%3A80%20viewFieldPermission_customFields_235%3A22% \
> 20viewFieldPermission_customFields_235%3A106)))%27%7D&qq_viewFieldPermission_customF \
> ields_236=%7B!v%3D%27(((viewFieldPermission_customFields_236%3A80%20viewFieldPermiss \
> ion_customFields_236%3A22%20viewFieldPermission_customFields_236%3A106)))%27%7D&qq_v \
> iewFieldPermission_customFields_237=%7B!v%3D%27(((viewFieldPermission_customFields_2 \
> 37%3A80%20viewFieldPermission_customFields_237%3A22%20viewFieldPermission_customFiel \
> ds_237%3A106)))%27%7D&qq_viewFieldPermission_customFields_238=%7B!v%3D%27(((viewFiel \
> dPermission_customFields_238%3A80%20viewFieldPermission_customFields_238%3A22%20view \
> FieldPermission_customFields_238%3A106)))%27%7D&qq_viewFieldPermission_customFields_ \
> 241=%7B!v%3D%27(((viewFieldPermission_customFields_241%3A80%20viewFieldPermission_cu \
> stomFields_241%3A22%20viewFieldPermission_customFields_241%3A106)))%27%7D&qq_viewFie \
> ldPermission_customFields_242=%7B!v%3D%27(((viewFieldPermission_customFields_242%3A8 \
> 0%20viewFieldPermission_customFields_242%3A22%20viewFieldPermission_customFields_242%3A106)))%27%7D&rows=100&sort=id%20asc&start=300
>  
> When it is executed a second time the QTime is about 4 – 11 but the total \
> response time is about 15 seconds !! 
> 
> When it does NOT come from cache the QTime is about 1 second which seems realistic \
> (see below): But the total response time is 15 seconds and approx. the same as \
> above ( ~ 140 KB)! {
> "responseHeader":{
> "status":0,
> "QTime":914,
> "params":{
> "qq_viewFieldPermission_customFields_209":"{!v='(((viewFieldPermission_customFields_209:80 \
> viewFieldPermission_customFields_209:22 \
> viewFieldPermission_customFields_209:106)))'}", \
> "fl":"id,name,permission_customFields_date_200:exists(query($qq_viewFieldPermission_ \
> customFields_200)),permission_customFields_date_201:exists(query($qq_viewFieldPermis \
> sion_customFields_201)),permission_customFields_date_202:exists(query($qq_viewFieldP \
> ermission_customFields_202)),permission_customFields_date_207:exists(query($qq_viewF \
> ieldPermission_customFields_207)),permission_customFields_date_208:exists(query($qq_ \
> viewFieldPermission_customFields_208)),permission_customFields_date_209:exists(query \
> ($qq_viewFieldPermission_customFields_209)),permission_customFields_date_216:exists( \
> query($qq_viewFieldPermission_customFields_216)),permission_customFields_date_217:ex \
> ists(query($qq_viewFieldPermission_customFields_217)),permission_customFields_date_2 \
> 18:exists(query($qq_viewFieldPermission_customFields_218)),permission_customFields_d \
> ate_219:exists(query($qq_viewFieldPermission_customFields_219)),permission_customFie \
> lds_date_222:exists(query($qq_viewFieldPermission_customFields_222)),permission_cust \
> omFields_date_223:exists(query($qq_viewFieldPermission_customFields_223)),permission \
> _customFields_date_160:exists(query($qq_viewFieldPermission_customFields_160)),permi \
> ssion_customFields_date_224:exists(query($qq_viewFieldPermission_customFields_224)), \
> permission_customFields_date_225:exists(query($qq_viewFieldPermission_customFields_2 \
> 25)),permission_customFields_date_226:exists(query($qq_viewFieldPermission_customFie \
> lds_226)),permission_customFields_date_100:exists(query($qq_viewFieldPermission_cust \
> omFields_100)),permission_customFields_date_231:exists(query($qq_viewFieldPermission \
> _customFields_231)),permission_customFields_date_170:exists(query($qq_viewFieldPermi \
> ssion_customFields_170)),permission_customFields_date_235:exists(query($qq_viewFieldPermission_customFields_235))",
>  "fq":"(createdBy:106 assetStatus:0) ((allowPermissions:80 allowPermissions:22 \
> allowPermissions:106) -assetStatus:0)", \
> "qq_viewFieldPermission_customFields_170":"{!v='(((viewFieldPermission_customFields_170:80 \
> viewFieldPermission_customFields_170:22 \
> viewFieldPermission_customFields_170:106)))'}", \
> "qq_viewFieldPermission_customFields_190":"{!v='(((viewFieldPermission_customFields_190:80 \
> viewFieldPermission_customFields_190:22 \
> viewFieldPermission_customFields_190:106)))'}", \
> "qq_viewFieldPermission_customFields_130":"{!v='(((viewFieldPermission_customFields_130:80 \
> viewFieldPermission_customFields_130:22 \
> viewFieldPermission_customFields_130:106)))'}", \
> "qq_viewFieldPermission_customFields_110":"{!v='(((viewFieldPermission_customFields_110:80 \
> viewFieldPermission_customFields_110:22 \
> viewFieldPermission_customFields_110:106)))'}", \
> "qq_viewFieldPermission_customFields_231":"{!v='(((viewFieldPermission_customFields_231:80 \
> viewFieldPermission_customFields_231:22 \
> viewFieldPermission_customFields_231:106)))'}", \
> "qq_viewFieldPermission_customFields_236":"{!v='(((viewFieldPermission_customFields_236:80 \
> viewFieldPermission_customFields_236:22 \
> viewFieldPermission_customFields_236:106)))'}", \
> "qq_viewFieldPermission_customFields_237":"{!v='(((viewFieldPermission_customFields_237:80 \
> viewFieldPermission_customFields_237:22 \
> viewFieldPermission_customFields_237:106)))'}", \
> "qq_viewFieldPermission_customFields_235":"{!v='(((viewFieldPermission_customFields_235:80 \
> viewFieldPermission_customFields_235:22 \
> viewFieldPermission_customFields_235:106)))'}", \
> "qq_viewFieldPermission_customFields_218":"{!v='(((viewFieldPermission_customFields_218:80 \
> viewFieldPermission_customFields_218:22 \
> viewFieldPermission_customFields_218:106)))'}", \
> "qq_viewFieldPermission_customFields_219":"{!v='(((viewFieldPermission_customFields_219:80 \
> viewFieldPermission_customFields_219:22 \
> viewFieldPermission_customFields_219:106)))'}", \
> "qq_viewFieldPermission_customFields_216":"{!v='(((viewFieldPermission_customFields_216:80 \
> viewFieldPermission_customFields_216:22 \
> viewFieldPermission_customFields_216:106)))'}", \
> "qq_viewFieldPermission_customFields_238":"{!v='(((viewFieldPermission_customFields_238:80 \
> viewFieldPermission_customFields_238:22 \
> viewFieldPermission_customFields_238:106)))'}", \
> "qq_viewFieldPermission_customFields_217":"{!v='(((viewFieldPermission_customFields_217:80 \
> viewFieldPermission_customFields_217:22 \
> viewFieldPermission_customFields_217:106)))'}", "debug":"true",
> "start":"0",
> "sort":"id asc",
> "qq_viewFieldPermission_customFields_180":"{!v='(((viewFieldPermission_customFields_180:80 \
> viewFieldPermission_customFields_180:22 \
> viewFieldPermission_customFields_180:106)))'}", "rows":"100",
> "q":"containers:80",
> "qq_viewFieldPermission_customFields_160":"{!v='(((viewFieldPermission_customFields_160:80 \
> viewFieldPermission_customFields_160:22 \
> viewFieldPermission_customFields_160:106)))'}", \
> "qq_viewFieldPermission_customFields_100":"{!v='(((viewFieldPermission_customFields_100:80 \
> viewFieldPermission_customFields_100:22 \
> viewFieldPermission_customFields_100:106)))'}", \
> "qq_viewFieldPermission_customFields_200":"{!v='(((viewFieldPermission_customFields_200:80 \
> viewFieldPermission_customFields_200:22 \
> viewFieldPermission_customFields_200:106)))'}", \
> "qq_viewFieldPermission_customFields_222":"{!v='(((viewFieldPermission_customFields_222:80 \
> viewFieldPermission_customFields_222:22 \
> viewFieldPermission_customFields_222:106)))'}", \
> "qq_viewFieldPermission_customFields_120":"{!v='(((viewFieldPermission_customFields_120:80 \
> viewFieldPermission_customFields_120:22 \
> viewFieldPermission_customFields_120:106)))'}", \
> "qq_viewFieldPermission_customFields_241":"{!v='(((viewFieldPermission_customFields_241:80 \
> viewFieldPermission_customFields_241:22 \
> viewFieldPermission_customFields_241:106)))'}", \
> "qq_viewFieldPermission_customFields_242":"{!v='(((viewFieldPermission_customFields_242:80 \
> viewFieldPermission_customFields_242:22 \
> viewFieldPermission_customFields_242:106)))'}", \
> "qq_viewFieldPermission_customFields_225":"{!v='(((viewFieldPermission_customFields_225:80 \
> viewFieldPermission_customFields_225:22 \
> viewFieldPermission_customFields_225:106)))'}", \
> "qq_viewFieldPermission_customFields_226":"{!v='(((viewFieldPermission_customFields_226:80 \
> viewFieldPermission_customFields_226:22 \
> viewFieldPermission_customFields_226:106)))'}", \
> "qq_viewFieldPermission_customFields_201":"{!v='(((viewFieldPermission_customFields_201:80 \
> viewFieldPermission_customFields_201:22 \
> viewFieldPermission_customFields_201:106)))'}", \
> "qq_viewFieldPermission_customFields_223":"{!v='(((viewFieldPermission_customFields_223:80 \
> viewFieldPermission_customFields_223:22 \
> viewFieldPermission_customFields_223:106)))'}", \
> "qq_viewFieldPermission_customFields_202":"{!v='(((viewFieldPermission_customFields_202:80 \
> viewFieldPermission_customFields_202:22 \
> viewFieldPermission_customFields_202:106)))'}", \
> "qq_viewFieldPermission_customFields_224":"{!v='(((viewFieldPermission_customFields_224:80 \
> viewFieldPermission_customFields_224:22 \
> viewFieldPermission_customFields_224:106)))'}", \
> "qq_viewFieldPermission_customFields_207":"{!v='(((viewFieldPermission_customFields_207:80 \
> viewFieldPermission_customFields_207:22 \
> viewFieldPermission_customFields_207:106)))'}", \
> "qq_viewFieldPermission_customFields_208":"{!v='(((viewFieldPermission_customFields_208:80 \
> viewFieldPermission_customFields_208:22 \
> viewFieldPermission_customFields_208:106)))'}"}}, \
> "response":{"numFound":927,"start":0,"numFoundExact":true,"docs":[ {
> "id":20,
> "name":"b301112_27_u_32_u_40.tif",
> "permission_customFields_date_200":false,
> "permission_customFields_date_201":true,
> "permission_customFields_date_202":true,
> "permission_customFields_date_207":true,
> "permission_customFields_date_208":true,
> "permission_customFields_date_209":true,
> "permission_customFields_date_216":false,
> "permission_customFields_date_217":false,
> "permission_customFields_date_218":false,
> "permission_customFields_date_219":false,
> "permission_customFields_date_222":false,
> "permission_customFields_date_223":false,
> "permission_customFields_date_160":false,
> "permission_customFields_date_224":false,
> "permission_customFields_date_225":false,
> "permission_customFields_date_226":false,
> "permission_customFields_date_100":false,
> "permission_customFields_date_231":true,
> "permission_customFields_date_170":false,
> "permission_customFields_date_235":false},
> [2 … 99 other documents in response]
> {
> "id":4930,
> "name":"b307682_03-10.tif",
> "permission_customFields_date_200":false,
> "permission_customFields_date_201":true,
> "permission_customFields_date_202":true,
> "permission_customFields_date_207":true,
> "permission_customFields_date_208":true,
> "permission_customFields_date_209":true,
> "permission_customFields_date_216":false,
> "permission_customFields_date_217":false,
> "permission_customFields_date_218":false,
> "permission_customFields_date_219":false,
> "permission_customFields_date_222":false,
> "permission_customFields_date_223":false,
> "permission_customFields_date_160":false,
> "permission_customFields_date_224":false,
> "permission_customFields_date_225":false,
> "permission_customFields_date_226":false,
> "permission_customFields_date_100":false,
> "permission_customFields_date_231":true,
> "permission_customFields_date_170":false,
> "permission_customFields_date_235":false}]
> },
> "debug":{
> "rawquerystring":"containers:80",
> "querystring":"containers:80",
> "parsedquery":"IndexOrDocValuesQuery(containers:[80 TO 80])",
> "parsedquery_toString":"containers:[80 TO 80]",
> "explain":{
> "20":"\n1.0 = containers:[80 TO 80]\n",
> "76":"\n1.0 = containers:[80 TO 80]\n",
> "1514":"\n1.0 = containers:[80 TO 80]\n",
> "1565":"\n1.0 = containers:[80 TO 80]\n",
> "1566":"\n1.0 = containers:[80 TO 80]\n",
> "1607":"\n1.0 = containers:[80 TO 80]\n",
> "1675":"\n1.0 = containers:[80 TO 80]\n",
> "2489":"\n1.0 = containers:[80 TO 80]\n",
> "2642":"\n1.0 = containers:[80 TO 80]\n",
> "2643":"\n1.0 = containers:[80 TO 80]\n",
> "2680":"\n1.0 = containers:[80 TO 80]\n",
> "2681":"\n1.0 = containers:[80 TO 80]\n",
> "2682":"\n1.0 = containers:[80 TO 80]\n",
> "2683":"\n1.0 = containers:[80 TO 80]\n",
> "2684":"\n1.0 = containers:[80 TO 80]\n",
> "2685":"\n1.0 = containers:[80 TO 80]\n",
> "2686":"\n1.0 = containers:[80 TO 80]\n",
> "2696":"\n1.0 = containers:[80 TO 80]\n",
> "2697":"\n1.0 = containers:[80 TO 80]\n",
> "2698":"\n1.0 = containers:[80 TO 80]\n",
> "2699":"\n1.0 = containers:[80 TO 80]\n",
> "2700":"\n1.0 = containers:[80 TO 80]\n",
> "2701":"\n1.0 = containers:[80 TO 80]\n",
> "2702":"\n1.0 = containers:[80 TO 80]\n",
> "2719":"\n1.0 = containers:[80 TO 80]\n",
> "2720":"\n1.0 = containers:[80 TO 80]\n",
> "2727":"\n1.0 = containers:[80 TO 80]\n",
> "2739":"\n1.0 = containers:[80 TO 80]\n",
> "2740":"\n1.0 = containers:[80 TO 80]\n",
> "2741":"\n1.0 = containers:[80 TO 80]\n",
> "2742":"\n1.0 = containers:[80 TO 80]\n",
> "2743":"\n1.0 = containers:[80 TO 80]\n",
> "2744":"\n1.0 = containers:[80 TO 80]\n",
> "2745":"\n1.0 = containers:[80 TO 80]\n",
> "2746":"\n1.0 = containers:[80 TO 80]\n",
> "2747":"\n1.0 = containers:[80 TO 80]\n",
> "2748":"\n1.0 = containers:[80 TO 80]\n",
> "2749":"\n1.0 = containers:[80 TO 80]\n",
> "2750":"\n1.0 = containers:[80 TO 80]\n",
> "2755":"\n1.0 = containers:[80 TO 80]\n",
> "2756":"\n1.0 = containers:[80 TO 80]\n",
> "2757":"\n1.0 = containers:[80 TO 80]\n",
> "2758":"\n1.0 = containers:[80 TO 80]\n",
> "2759":"\n1.0 = containers:[80 TO 80]\n",
> "2760":"\n1.0 = containers:[80 TO 80]\n",
> "2761":"\n1.0 = containers:[80 TO 80]\n",
> "2762":"\n1.0 = containers:[80 TO 80]\n",
> "2763":"\n1.0 = containers:[80 TO 80]\n",
> "2799":"\n1.0 = containers:[80 TO 80]\n",
> "2800":"\n1.0 = containers:[80 TO 80]\n",
> "2801":"\n1.0 = containers:[80 TO 80]\n",
> "2802":"\n1.0 = containers:[80 TO 80]\n",
> "2803":"\n1.0 = containers:[80 TO 80]\n",
> "2804":"\n1.0 = containers:[80 TO 80]\n",
> "2805":"\n1.0 = containers:[80 TO 80]\n",
> "2829":"\n1.0 = containers:[80 TO 80]\n",
> "2838":"\n1.0 = containers:[80 TO 80]\n",
> "2839":"\n1.0 = containers:[80 TO 80]\n",
> "2840":"\n1.0 = containers:[80 TO 80]\n",
> "2841":"\n1.0 = containers:[80 TO 80]\n",
> "2842":"\n1.0 = containers:[80 TO 80]\n",
> "2843":"\n1.0 = containers:[80 TO 80]\n",
> "2878":"\n1.0 = containers:[80 TO 80]\n",
> "2889":"\n1.0 = containers:[80 TO 80]\n",
> "2911":"\n1.0 = containers:[80 TO 80]\n",
> "2945":"\n1.0 = containers:[80 TO 80]\n",
> "2946":"\n1.0 = containers:[80 TO 80]\n",
> "2947":"\n1.0 = containers:[80 TO 80]\n",
> "2972":"\n1.0 = containers:[80 TO 80]\n",
> "2973":"\n1.0 = containers:[80 TO 80]\n",
> "2974":"\n1.0 = containers:[80 TO 80]\n",
> "2975":"\n1.0 = containers:[80 TO 80]\n",
> "2976":"\n1.0 = containers:[80 TO 80]\n",
> "2999":"\n1.0 = containers:[80 TO 80]\n",
> "3001":"\n1.0 = containers:[80 TO 80]\n",
> "3062":"\n1.0 = containers:[80 TO 80]\n",
> "3063":"\n1.0 = containers:[80 TO 80]\n",
> "4335":"\n1.0 = containers:[80 TO 80]\n",
> "4604":"\n1.0 = containers:[80 TO 80]\n",
> "4606":"\n1.0 = containers:[80 TO 80]\n",
> "4608":"\n1.0 = containers:[80 TO 80]\n",
> "4705":"\n1.0 = containers:[80 TO 80]\n",
> "4706":"\n1.0 = containers:[80 TO 80]\n",
> "4707":"\n1.0 = containers:[80 TO 80]\n",
> "4723":"\n1.0 = containers:[80 TO 80]\n",
> "4724":"\n1.0 = containers:[80 TO 80]\n",
> "4732":"\n1.0 = containers:[80 TO 80]\n",
> "4736":"\n1.0 = containers:[80 TO 80]\n",
> "4748":"\n1.0 = containers:[80 TO 80]\n",
> "4749":"\n1.0 = containers:[80 TO 80]\n",
> "4764":"\n1.0 = containers:[80 TO 80]\n",
> "4766":"\n1.0 = containers:[80 TO 80]\n",
> "4768":"\n1.0 = containers:[80 TO 80]\n",
> "4769":"\n1.0 = containers:[80 TO 80]\n",
> "4792":"\n1.0 = containers:[80 TO 80]\n",
> "4793":"\n1.0 = containers:[80 TO 80]\n",
> "4794":"\n1.0 = containers:[80 TO 80]\n",
> "4815":"\n1.0 = containers:[80 TO 80]\n",
> "4883":"\n1.0 = containers:[80 TO 80]\n",
> "4930":"\n1.0 = containers:[80 TO 80]\n"},
> "QParser":"LuceneQParser",
> "filter_queries":["(createdBy:106 assetStatus:0) ((allowPermissions:80 \
> allowPermissions:22 allowPermissions:106) -assetStatus:0)"], \
> "parsed_filter_queries":["(IndexOrDocValuesQuery(createdBy:[106 TO 106]) \
> IndexOrDocValuesQuery(assetStatus:[0 TO 0])) \
> ((IndexOrDocValuesQuery(allowPermissions:[80 TO 80]) \
> IndexOrDocValuesQuery(allowPermissions:[22 TO 22]) \
> IndexOrDocValuesQuery(allowPermissions:[106 TO 106])) \
> -IndexOrDocValuesQuery(assetStatus:[0 TO 0]))"], "timing":{
> "time":912.0,
> "prepare":{
> "time":36.0,
> "query":{
> "time":35.0},
> "facet":{
> "time":0.0},
> "facet_module":{
> "time":0.0},
> "mlt":{
> "time":0.0},
> "highlight":{
> "time":0.0},
> "stats":{
> "time":0.0},
> "expand":{
> "time":0.0},
> "terms":{
> "time":0.0},
> "debug":{
> "time":0.0}},
> "process":{
> "time":870.0,
> "query":{
> "time":845.0},
> "facet":{
> "time":0.0},
> "facet_module":{
> "time":0.0},
> "mlt":{
> "time":0.0},
> "highlight":{
> "time":0.0},
> "stats":{
> "time":0.0},
> "expand":{
> "time":0.0},
> "terms":{
> "time":0.0},
> "debug":{
> "time":23.0}}}}}
> 
> I cannot explain the very long waiting times to download or deliver the http chunks \
> on localhost. Even the TTFB is much higher than QTime. Moreover I have no further \
> idea what I can do to improve the response times. 
> Additionally I have measured solr with yourkit. it takes obviously about 11 seconds \
> to process (BKDReader.intersect): 
> 
> When we remove more and more of the raw query evaluations \
> (permission_customFields_date_202:exists(query($qq_viewFieldPermission_customFields_202)) \
> etc.) from fl the following linear response times could be observed: 
> keep only 10 instead of 15 -> total time of 7 seconds to response
> keep only 5 -> total time of 2,7 seconds to Response
> remove completely, then only a few milliseconds are measured (20 – 30 ms)
> 
> It is comprehensible that the query evaluation and fl parameters influence the \
> processing, but it is not clear why the performance degrades that dramatic after \
> some modifications in index. As mentioned if I drop the whole index and rebuild \
> everything again, the response times are about 1,5 seconds instead of 15 seconds. \
> "Optimize index" didn't help. 
> 
> Please let me know if someone has some ideas what causes the described problems or \
> what we can change to improve. Please also let me know if any additional details \
> are required from my end. 
> Thanks in advance,
> ako



[prev in list] [next in list] [prev in thread] [next in thread]