[prev in list] [next in list] [prev in thread] [next in thread] 

List:       bacula-devel
Subject:    Re: [Bacula-devel] [Bacula-users]  feature request
From:       Kern Sibbald <kern () sibbald ! com>
Date:       2008-07-30 16:29:11
Message-ID: 200807301829.12340.kern () sibbald ! com
[Download RAW message or body]

On Wednesday 30 July 2008 18:04:08 John Drescher wrote:
> On Wed, Jul 30, 2008 at 11:45 AM, Kern Sibbald <kern@sibbald.com> wrote:
> > Hello,
> >
> > On Tuesday 29 July 2008 15:22:03 T. Horsnell wrote:
> >> What:
> >> That Bacula be modified to enable the simultaneous use of multiple tape
> >> drives, either as a bunch of freestanding units or as multiple drives in
> >> an autochanger.
> >>
> >> Why:
> >> For me personally, this would permit faster simpler backup of a large
> >> single filesystem. I have a two-drive LTO4 tapechanger and at present,
> >> in order to utilise both drives at once, I have to create two jobs each
> >> of which backs up part of the filesystem. This is not optimal as the two
> >> subparts can change size radically, meaning that one drive may spend
> >> much of its time idle. I also have to split my tapes into two pools, one
> >> for each job.
> >> I have seen one other similar request to the users list recently,
> >> whereby a user had a bynch of freestanding drives which he wanted to
> >> preload with a set of tapes once a week, and then have Bacula
> >> automatically organise his backup over the tapes as it saw fit.
> >> This may make a cheap alternative to a tapechanger.
> >>
> >> A device pool maybe?
> >
> > Technically I don't know how to do this project.   The SD has blocks of
> > data coming in from the FD, and sending those blocks to multiple drives
> > would be extremely difficult to track -- I am not even sure how. 
> > Currently, they are either sequential (if spooling is on) or at least
> > written in order to a single drive, so it is very easy to track and find
> > them for restores.
> >
> > In addition, I don't think it is possible for any existing network
> > connection to run fast enough to drive an LTO4 tape drive at full speed,
> > so this project seems to add a lot of complexity to Bacula to give no
> > improvement in performance.  If I am wrong about the network connection
> > and an LTO4, please show me the math :-)
> >
> > For the moment I cannot accept the Feature Request.  If you can give
> > complete details of a design for tracking the blocks written to multiple
> > tapes and how to send them back to the FD in the same order, and if you
> > can show with a simulation that we would actually gain something, then
> > you can resubmit it, and I will reconsider adding it to the projects. 
> > After than, either someone must submit a patch or the Bacula users vote
> > and it becomes a top rated project, in which case one of the developers
> > would surely work on it.
>
> I totally agree that sending data blocks to more than 1 device at a
> time for a single job at this time will be a very difficult task but
> when I read this I see that there are two basic feature requests and
> the second is not so bad. The second is to be able to continue a job
> on a different tape than the job started on. I can see this is useful
> if you have more than 1 drive of the same type.

Continuing the job on a second tape is something that is going to take a 
significant set of changes to the SD, because you need the ability to change 
the device pointer while any number of jobs are simultaneously writing to the 
device.  This is non-trivial to implement.  I currently have implemented it 
for reading, but that is relatively easy since only one job can read a tape 
at a time, thus you change one pointer in one thread and away you go.  For 
writing tapes, it needs to change the pointer in as many threads as are 
writing simultaneously, which requires ensuring that they are all blocked -- 
non-trivial.  I am currently slowly but surely adding the structures and the 
code to permit this, but it will not be something that works in the immediate 
future.  It is similar to problems encountered in making the changes to the 
SD in going from one job writing a tape at a time to multiple simultaneous 
jobs.



-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
Bacula-devel mailing list
Bacula-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-devel
[prev in list] [next in list] [prev in thread] [next in thread]