[prev in list] [next in list] [prev in thread] [next in thread] 

List:       bacula-devel
Subject:    Re: [Bacula-devel] How Bacula gets StorageId ?
From:       Kern Sibbald <kern () sibbald ! com>
Date:       2013-04-07 13:05:49
Message-ID: 51616F2D.5010708 () sibbald ! com
[Download RAW message or body]

Hello,

In the bug report you cite below, there was a problem with
StorageIds, but the user was using multiple catalogs and what
he was doing was not very clear.  What appeared to be the case
was that he was operating Bacula well out of the design
parameters by mixing data/volumes from several catalogs.

The solution you propose sounds like it might work.  However,
I recommend that you make a copy of your catalog prior to
modifying it.  In general, I don't recommend modifying the catalog
manually since you can get records out of sync and thus create
even greater problems later.  If you are running Bacula the way
it was designed (e.g. not using multiple storage devices that
point to the same autochanger, but which Bacula considers
to be different devices, you should never have any problems,
and manually updating the catalog should never be necessary.

As far as I can tell, this problem occurs only in cases where you
are running rather sophisticated configurations, and until this moment
no one has been able or wanted to produce a case that is reproducible
so that I can fix it.


Best regards and good luck,
Kern

On 04/07/2013 01:10 PM, Bugbear wrote:
> Hi,
>
> Thank you for your answer and time spended.
>
> Sadly it's not easy for me to make you a precise and few step test
> case, in which going through predefined steps will cause the problem to
> show. The main reason is because I'm not so familiar with some of the
> internal procedures in Bacula (the procedure of
> calculating/recalculating .. storage id's) and I cannot say the exact
> circumstances in which the problem rise.
>
> In fact I saw similiar problem in the bugtracker, which was marked as
> "won't fix" in this thread:
> http://marc.info/?l=bacula-bugs&m=132248738530236
>
> Anyway what I'm goin to do is that:
> - After delete/add client(respectively Storage resource) I will truncate
>    the Storage tables on all of my catalogs, then restart bacula (which
>    will reinsert the storages with the new ID's), then update all Media
>    tables with the appropriate Id's (the new ones). Hope this works well
>    and not break anything;
>
> Best regards
>
> On Fri, 05 Apr 2013 15:50:05 +0200
> Kern Sibbald <kern@sibbald.com> wrote:
>
>> Hello,
>>
>> Well, I am not sure I can answer all your questions. I usually
>> don't give much user support any more (too big a workload), but
>> this subject is coming up more and more as users have more
>> complicated setups.  It sounds like you have a pretty large
>> and sophisticated setup, which makes things even more
>> difficult.
>>
>> Basically Bacula was not designed to handle changing Storage
>> devices.  Doing so can get the Media table out of sync with
>> the Storage table, then certain functions don't work too well.
>>
>> If you do delete Storage devices and perhaps add them, that is
>> generally where problems come in.  Also, if you try to use multiple
>> Storage definitions in the SD that point to the same Device
>> or which are part of the same Autochanger (possibly shared),
>> you are going to run into a lot of problems that are simply due
>> to the fact that the original design did not envision handling those
>> cases.
>>
>> You should be able to add new Storage devices with no major
>> problems providing you restart Bacula after doing so.  If you
>> do delete a Storage or change its name, you will need to
>> delete all records associated with volumes written on that
>> device or somehow manually update them to point to some
>> new Storage device.
>>
>> Changing the code is a lot of work, and it is probably not
>> even possible for Bacula to "figure out" what new storage device
>> should be used in place of an old one.  Also,
>> I am mostly concentrating on the Enterprise code where
>> we have implemented multiple shared Autochangers and such.
>> That required a pretty big rewrite.
>>
>> It is hard to give you advice, because first, I have never really been
>> able to reproduce any of the problems that are coming up for users,
>> and second the exact advice depends on your setup, and understanding
>> that take a lot more time than I have.
>>
>> If you can create a simple case where something goes wrong, I don't
>> promise anything except to say I will look at it.  I would need a very
>> precise step by step process that shows how to create the problem.
>> The best way to do that would be to send  Dir and SD config files,
>> along with a script that runs Bacula, possibly modifies some config
>> parameters, restarts Bacula and clearly shows the problem.  With
>> something like that (i.e. something reproducible) I can hopefully
>> find the problem and find a solution that either fixes it or prints
>> much better messages telling you how to fix it.
>>
>> I recommend making any such submission in a bug report as emails
>> get buried in my Inbox very quickly and thus effectively lost :-(
>>
>> Best regards,
>> Kern
>>
>> On 04/05/2013 02:55 PM, Bugbear wrote:
>>> Hello,
>>>
>>> Thanks for your answer.
>>>
>>> Let me explain the situation. My english is not native (and maybe
>>> you have alreday noticed that) so please excuse me.
>>>
>>> Currently I'am using bacula with multi-catalog, multi-client,
>>> multi-storage setup. Bacula is configured to have dedicated:
>>> storage/catalog/volume set/file set.... for every client. I am using
>>> only Files as a media. Also I'm adding or deleting clients very
>>> often (when that's needed).
>>>
>>> The problem I'm facing is that sometime bacula decides to change the
>>> order of StorageId's (maybe after client deletion or something
>>> else ..I'm not very sure) and when this happens my new Backup jobs,
>>> are writing the new StorageId  in the client_catalog->Media table.
>>>
>>> When this happens and bacula change the StorageId for a storage
>>> resource, if I try to restore a job which is older (before the
>>> storageID change), then bacula get's my OLD StorageID from
>>> catalog->Media table (which contains job history) then searches the
>>> id in catalog->Storage table(where nothing has changed) and tries
>>> to find a volume in that Storage which of course is unsuccessfull
>>> because that ID doesn't belong anymore to this client/storage
>>> resource/ and bacula gets wrong Storage + missing Volume.
>>>
>>> In that case, my only choice is to manually update all
>>> StorageID's in table Media(on all client catalogs) seting them to
>>> the new StorageID, because that way I will be able to make a
>>> restore job from all of the jobs (including older ones).
>>>
>>> I tried to manually edit the values in Storage table for a custom
>>> client and then run another backup job, but it seems that Bacula
>>> doesn't get the ID from this table and also I couldn't find any SQL
>>> queries related to this, so I decided that bacula uses some
>>> different mechanism to calculate the StorageIDs for the Stroage
>>> resources.  That was the reason I asked for information about the
>>> way Bacula gets/calculates StorageID's for a storage resources.
>>>
>>> You mention "update slots", can you clarify what exactly this
>>> command does ? (truncate / re-insert all Storage tables of all
>>> catalogs or ..? )
>>>
>>> Regards
>>>
a

------------------------------------------------------------------------------
Minimize network downtime and maximize team effectiveness.
Reduce network management and security costs.Learn how to hire 
the most talented Cisco Certified professionals. Visit the 
Employer Resources Portal
http://www.cisco.com/web/learning/employer_resources/index.html
_______________________________________________
Bacula-devel mailing list
Bacula-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-devel
[prev in list] [next in list] [prev in thread] [next in thread]