[prev in list] [next in list] [prev in thread] [next in thread] 

List:       bacula-users
Subject:    Re: [Bacula-users] A general concept
From:       Arno Lehmann <al () its-lehmann ! de>
Date:       2008-02-28 23:08:25
Message-ID: 47C73EE9.2010308 () its-lehmann ! de
[Download RAW message or body]

Hi,

28.02.2008 22:37, Peter Much wrote:
> Hallo Arno!
> 
> <al@its-lehmann.de> aka Arno Lehmann  schrieb
> mit Datum Thu, 28 Feb 2008 17:38:55 +0100 in m2n.bacula.users:
> 
> |> The only thing that has changed is two values in the catalog
> |> which mark the old job as migrated and the new job as the target
> |> of that migration. Now if one would someway change these two values
> |> back to default, then one would have two jobs with different
> |> job-ids but the same content. Certainly, some piece of the Bacula
> |> code might be incompatible with such an action - as I said, it seems
> |> currently not supported. But there is not much work needed to get
> |> it implemented.
> |
> |Restores - currently, Bacula doesn't know how to handle restores where 
> |the needed data is available in more than one place.
> 
> Yes... could be. Have You tried it? 

No... but, last time I looked and asked, that was the situation. In 
fact I have been discussing this with Kern, and he stated that it 
would be rather simple to make sure the relevant data to have several 
usable job copies would be in the catalog, but choosing the right set 
of volumes for a restore would need work (hint: table Location, row Cost).
Keeping the original data (partly) in the catalog is a first step 
towards implementing copy jobs, I think.

> And as I said concerning the concept: the copy instance of the backup
> will not have file recods in the database - it will only have the
> job record, with a different jobid. There will not be a choice - 
> restores will always be done from the primary instance.

For now, and for migration jobs. I expect to see copy jobs some day.

> The copy
> instance comes only to play if we loose the primary instance, and 
> then it will only be migrated back to the replacement media of the
> primary instance. 
> And such a setup should be suitable if you want a second copy for
> emergency (cartridge broken etc.) - but if you want a second copy for
> load-sharing etc., well that's a different matter.

Right... that's the difference between what you want now, and what 
(eventually) is possible with Bacula.

> 
> |> So, if I were in the need of such functionality, and as the Bacula
> |> license allows us to make modifications to it as we seem appropriate,
> |> I would just make it working - on my own risk, on my own
> |> responsibility. (Respectively one could hire somebody who is willing
> |> and able to take the responsibility.)
> |
> |I hope you would submit it as a patch, as that would save Kern and 
> |others from doing things again in the near future :-)
> 
> Aaaahhhh!!! :-)))
> Now, the point is: I do not need it currently. I am perfectly happy
> with just running two backups and having the "since" values tuned
> in a way that they will always overlap and never gap.

I absolutely understand that. But you won't mind if I try to find out 
how likely it is you'll put some work into that, right? ;-)

> And, to be clear: for me this here is pure hobby, it is sports. 
> Or maybe it is even worse: maybe I am contractually prohibited to do
> development work on a piece like Bacula. So I will do exactly as is
> mandatory within my Code of Honour: I will share those patches and
> fixes that I have to implement in order to get Bacula back up my own
> private home computers the way I like it. And that's all.

And that's perfectly ok... I even suspect that's more than some other 
Bacula users do.

> |> No - I'm speaking about the problem of getting the daily amount of
> |> data thru some drives R/W heads within 24 hours. There are sites
> |> ...
> |I would even recommend to keep Bacula in mind if you need to handle 
> |such a scenario... I don't think we're far from being reliable enough 
> |for such a beast.
> 
> You would? Well, then maybe I see problems where there are none -
> or maybe I have just seen way too many sig-11s...

That's the difference between our approaches - I use what's documented 
and exists officially, and you started pushing Bacula to its limits to 
see where it needs work done to suit your needs. Obviously, you're mor 
likely to see failures. (And, to point that out - I appreciate the 
effort you put into it!)

> |Just out of curiosity - which backup solutions do you know that handle 
> |this with resonable performance?
> 
> TSM?

Ok, that I believe... TSM is, in my opinion, the most complete... erm, 
backup solutions seems to be a bit of an understatement :-)

The other big commercial ones are, as far as I can tell, not much 
better there than Bacula.

> |I mean, the disk throughput problem is a universal one, and my (or 
> |rather, my customer's) experience is that you can build really fast 
> |disk arrays, given enough resources.
> 
> Maybe I didn't make myself very clear - I am not concerned about
> the physical throughput - that can be considered later. 
> I am concerned about the logic. But maybe I am just missing the
> the right idea, so just give me a clue, please:
> When I create a disk storage object, I define a "Storage Device" as
> the name of a directory on disk. Within this directory, there will
> be "Volumes" created. But it is always only possible to access one
> Volume at a time - and *either* for reading *or* for writing.

There are two approaches to this: Either set a 'fake' autochanger for 
file based volumes - see scripts/disk-changer - or create several 
storage devices all using the same set of volume files. The former 
approach is more robust, I believe.

If the SD stumbles over the fact that it doesn't want to use the same 
archive device twice, that can be solved using soft links. Note that I 
haven't tried this myself under heavy load, but I saw a setup where 
one SD handled several storage devices, sharing one storage partition 
among them.

> So it
> is never possible to read from any disk storage pool at the same
> time while it is written. To make it more clear: I do *not* want
> to write and read to the *same Volume*! I want to fill one Volume,
> then close it and write to the next Volume, and at that time read
> the previous Volume. 
> Now imagine, we have a tape library, and an autochanger with two
> drives. So it should be perfectly legal to write to one Volume
> in one drive, and at the same time read another Volume in the other 
> drive. But Bacula will not let me do such: in the Director the
> Autochanger gets defined as only one storage device, and as soon
> as I try to read and write at the same time on this device (even
> to different StoragePools within the Library), I will get this:
> 
>> Fatal error: Job canceled. Attempt to read and write same device.
> 
> And that is why I am asking: how can a disk storage object be
> designed to overcome this bottleneck?

Add storage devices :-)

If you have your actual storage area in /var/bacula/storage, create 
soft links as /var/bacula/storage2, ...storage3 and so on and use 
these as archive devices.

Bacula itself can handle autochangers with more than one drive 
operating simultaneously - there are many reports about that working, 
and I've got customers using such a setup - and it is possible to 
create an autochanger device which doesn't handle tapes but disk files.

Such a setup should get you further, I believe...

Arno

> 
> rgds,
> PMc
> 
> -------------------------------------------------------------------------
> This SF.net email is sponsored by: Microsoft
> Defy all challenges. Microsoft(R) Visual Studio 2008.
> http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
> _______________________________________________
> Bacula-users mailing list
> Bacula-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bacula-users
> 

-- 
Arno Lehmann
IT-Service Lehmann
www.its-lehmann.de

-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
_______________________________________________
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users
[prev in list] [next in list] [prev in thread] [next in thread]