[prev in list] [next in list] [prev in thread] [next in thread]
List: solr-user
Subject: Re: Query on changing FieldType
From: Shubham Goswami <shubham.goswami () hotwax ! co>
Date: 2019-10-25 11:56:33
Message-ID: CAC4RquBm_hbXwZexno++PW5VYPZED2wt=Z-aeMK7ibD4y-=6Dw () mail ! gmail ! com
[Download RAW message or body]
Hello Erick/Emir
Thanks for your valuable suggestions. I will it keep in mind while doing
such operations.
Best,
Shubham
On Wed, Oct 23, 2019 at 5:56 PM Erick Erickson <erickerickson@gmail.com>
wrote:
> Really, just don't do this. Please. As others have pointed out, it may
> look like it works, but it won't. I've spent many hours tracking down why
> clients got weird errors after making changes like this, sometimes weeks
> later. Or more accurately, if you choose to change field types without
> reindexing, please don't ask for others to troubleshoot it when something
> blows up.
>
> As far as creating a new core, if that takes a significant amount of time
> relative to re-indexing, then you must be working with a very small index.
> Those operations should take a couple of minutes, tops.
>
> Best,
> Erick
>
> > On Oct 23, 2019, at 5:13 AM, Emir Arnautović <
> emir.arnautovic@sematext.com> wrote:
> >
> > Hi Shubham,
> > My guess that it might be working for text because it uses o.toString()
> so there are no runtime errors while in case of others, it has to assume
> some class so it does class casting. You can check in logs what sort of
> error happens. But in any case, like Jason pointed out, that is a problem
> that is just waiting to happen somewhere and the only way to make sure it
> does not happen is to do full reindexing or to create a new field (with a
> new name) and stop using the one that is wrong. Different field types are
> indexed in different structures and with different defaults (e.g. for
> docValues) and I would not rely on some features working after field type
> changed.
> >
> > Thanks,
> > Emir
> > --
> > Monitoring - Log Management - Alerting - Anomaly Detection
> > Solr & Elasticsearch Consulting Support Training - http://sematext.com/
> >
> >
> >
> >> On 23 Oct 2019, at 08:18, Shubham Goswami <shubham.goswami@hotwax.co>
> wrote:
> >>
> >> Hi Jason
> >>
> >> Thanks for the response.
> >> You are right that re-indexing is required after making any changes to
> >> Schema even i am re-indexing the docs in which i have
> >> changed the fieldtypes, but here Emir is talking about full re-indexing
> >> i.e. deleting the existing/core and creating new one that is
> >> time consuming i think. My doubt is that i am not able to change the
> type
> >> which has implementation classes like LongPointField/IntPointField to
> the
> >> type with implementation classes LongPointField/IntPointField.t
> >>
> >> But i am able to change into Text related fields like TextFields
> >> and from TextFields to any other Int/Long type fields.
> >> So i just want to know that what is exact dependency on these classes so
> >> that iam able to change types of some fields ?
> >>
> >> Thanks
> >> Shubham
> >>
> >> On Tue, Oct 22, 2019 at 6:29 PM Jason Gerlowski <gerlowskija@gmail.com>
> >> wrote:
> >>
> >>> Hi Shubbham,
> >>>
> >>> Emir gave you accurate advice - you cannot (safely) change field types
> >>> without reindexing. You may avoid errors for a time, and searches may
> >>> even return the results you expect. But the type-change is still a
> >>> ticking time bomb...Solr might try to merge segments down the road or
> >>> do some other operation and blow up in unexpected ways. For more
> >>> information on why this is, see the documentation here:
> >>> https://lucene.apache.org/solr/guide/8_2/reindexing.html.
> >>>
> >>> Unfortunately there's no way around it. This, by the way, is why the
> >>> community strongly recommends against using schema-guessing mode for
> >>> anything other than experimentation.
> >>>
> >>> Best of luck,
> >>>
> >>> Jason
> >>>
> >>> On Tue, Oct 22, 2019 at 7:42 AM Shubham Goswami
> >>> <shubham.goswami@hotwax.co> wrote:
> >>>>
> >>>> Hi Emir
> >>>>
> >>>> As you have mentioned above we cannot change field type after indexing
> >>> once
> >>>> and we have to do dull re-indexing again, I tried to change field type
> >>> from
> >>>> plong to pint which has implemented class solr.LongPointField and
> >>>> solr.IntPointField respectively and it was showing error as expected.
> >>>> But when i changed field types from pint/plong to any type which
> >>>> has implemented class solr.TextField, in this case its working fine
> and i
> >>>> am able to index the documents after changing its fieldtype with same
> and
> >>>> different id.
> >>>>
> >>>> So i want to know if is there any compatibility with implemented
> classes
> >>> ?
> >>>>
> >>>> Thanks
> >>>> Shubham
> >>>>
> >>>> On Tue, Oct 22, 2019 at 2:46 PM Emir Arnautović <
> >>>> emir.arnautovic@sematext.com> wrote:
> >>>>
> >>>>> Hi Shubham,
> >>>>> No you cannot. What you can do is to use copy field or update request
> >>>>> processor to store is as some other field and use that in your query
> >>> and
> >>>>> ignore the old one that will eventually disappear as the result of
> >>> segment
> >>>>> merges.
> >>>>>
> >>>>> HTH,
> >>>>> Emir
> >>>>> --
> >>>>> Monitoring - Log Management - Alerting - Anomaly Detection
> >>>>> Solr & Elasticsearch Consulting Support Training -
> >>> http://sematext.com/
> >>>>>
> >>>>>
> >>>>>
> >>>>>> On 22 Oct 2019, at 10:53, Shubham Goswami <
> shubham.goswami@hotwax.co
> >>>>
> >>>>> wrote:
> >>>>>>
> >>>>>> Hi Emir
> >>>>>>
> >>>>>> Thanks for the reply, i got your point.
> >>>>>> But is there any other way to do like one field could have two or
> >>> more
> >>>>>> different types defined ?
> >>>>>> or if i talk about my previous query, can we index some data for
> the
> >>>>> same
> >>>>>> field with different unique id after replacing the type ?
> >>>>>>
> >>>>>> Thanks again
> >>>>>> Shubham
> >>>>>>
> >>>>>> On Tue, Oct 22, 2019 at 1:23 PM Emir Arnautović <
> >>>>>> emir.arnautovic@sematext.com> wrote:
> >>>>>>
> >>>>>>> Hi Shubham,
> >>>>>>> Changing type is not allowed without full reindexing. If you do
> >>>>> something
> >>>>>>> like that, Solr will end up with segments with different types for
> >>> the
> >>>>> same
> >>>>>>> field. Remember that segments are immutable and that reindexing
> some
> >>>>>>> document will be in new segment, but old segment will still be
> there
> >>>>> and at
> >>>>>>> query type Solr will have mismatch between what is stated in schema
> >>> and
> >>>>>>> what is in segment. In order to change type you have to do full
> >>>>> reindexing
> >>>>>>> - create a new collection and reindex all documents.
> >>>>>>>
> >>>>>>> HTH,
> >>>>>>> Emir
> >>>>>>> --
> >>>>>>> Monitoring - Log Management - Alerting - Anomaly Detection
> >>>>>>> Solr & Elasticsearch Consulting Support Training -
> >>> http://sematext.com/
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>> On 22 Oct 2019, at 09:25, Shubham Goswami <
> >>> shubham.goswami@hotwax.co>
> >>>>>>> wrote:
> >>>>>>>>
> >>>>>>>> Hello Community
> >>>>>>>>
> >>>>>>>> I have indexed some documents for which solr has taken its
> >>>>> type="plongs"
> >>>>>>> by
> >>>>>>>> auto guessing but i am trying to change its type="pint" and
> >>> re-indexing
> >>>>>>> the
> >>>>>>>> same data with the same id and indexing the data with different id
> >>>>> where
> >>>>>>> id
> >>>>>>>> is unique key but it is showing error.
> >>>>>>>>
> >>>>>>>> Can somebody please let me know if it is possible or not, if not
> >>>>> possible
> >>>>>>>> then why it is not possible as i am using different id as well ?
> if
> >>>>>>>> possible then how we could achieve it ?
> >>>>>>>> Any help will be appreciated. Thanks in advance.
> >>>>>>>>
> >>>>>>>> --
> >>>>>>>> *Thanks & Regards*
> >>>>>>>> Shubham Goswami
> >>>>>>>> Enterprise Software Engineer
> >>>>>>>> *HotWax Systems*
> >>>>>>>> *Enterprise open source experts*
> >>>>>>>> cell: +91-7803886288
> >>>>>>>> office: 0731-409-3684
> >>>>>>>> http://www.hotwaxsystems.com
> >>>>>>>
> >>>>>>>
> >>>>>>
> >>>>>> --
> >>>>>> *Thanks & Regards*
> >>>>>> Shubham Goswami
> >>>>>> Enterprise Software Engineer
> >>>>>> *HotWax Systems*
> >>>>>> *Enterprise open source experts*
> >>>>>> cell: +91-7803886288
> >>>>>> office: 0731-409-3684
> >>>>>> http://www.hotwaxsystems.com
> >>>>>
> >>>>>
> >>>>
> >>>> --
> >>>> *Thanks & Regards*
> >>>> Shubham Goswami
> >>>> Enterprise Software Engineer
> >>>> *HotWax Systems*
> >>>> *Enterprise open source experts*
> >>>> cell: +91-7803886288
> >>>> office: 0731-409-3684
> >>>> http://www.hotwaxsystems.com
> >>>
> >>
> >>
> >> --
> >> Kind Regards,
> >> Shubham Goswami
> >> Enterprise Software Engineer
> >> mobile: +91 7803886288
> >> email: *shubham.goswami@hotwax.co*
> >> *www.hotwax.co <http://www.hotwax.co/>*
> >
>
>
--
Kind Regards,
Shubham Goswami
Enterprise Software Engineer
mobile: +91 7803886288
email: *shubham.goswami@hotwax.co*
*www.hotwax.co <http://www.hotwax.co/>*
[prev in list] [next in list] [prev in thread] [next in thread]
Configure |
About |
News |
Add a list |
Sponsored by KoreLogic