[prev in list] [next in list] [prev in thread] [next in thread]
List: bacula-bugs
Subject: [Bacula-bugs] [bacula 0001762]: performance problem in version
From: Mantis Bug Tracker <nobody () bugs ! bacula ! org>
Date: 2011-09-17 15:54:38
Message-ID: c5c823929db9b060b04db0a82d9081e8 () bugs ! bacula ! org
[Download RAW message or body]
The following issue has been CLOSED
======================================================================
http://bugs.bacula.org/view.php?id=1762
======================================================================
Reported By: jstarek
Assigned To:
======================================================================
Project: bacula
Issue ID: 1762
Category: bat
Reproducibility: always
Severity: minor
Priority: normal
Status: closed
Resolution: reopened
Fixed in Version:
======================================================================
Date Submitted: 2011-09-03 23:09 BST
Last Modified: 2011-09-17 16:54 BST
======================================================================
Summary: performance problem in version browser with big file
sets (3.5 mio files)
Description:
When trying to visualize a fileset with about 3.5 mio files, bat takes forever
to build/display the tree.
(tried with mysql and sqlite databases...)
Steps to Reproduce:
- backup a huge filesystem
- go to the version browser and hit refresh on the fileset/client
- wait for ever... (or crash, depending on your memory)
Additional Information:
Reason:
a) the query to retrieve the paths does an "ORDER BY Path" which is totally
unneccessary (as the build code does not rely on the order).
This ORDER BY causes long processing times in the director.
I simply removed it completely (see patch below).
b) the code to build the QTreeWidgetItems is suboptimal and not written for
performance. I simply used the caches a bit better and got to a much better
speed. (see patch below)
With this change, selecting files for restore is now possible.
I tested this on debian wheezy.
But I would need a windows-build with this patch...
(don't know how to do this yet - can anybody help me?)
======================================================================
----------------------------------------------------------------------
(0005969) ebollengier (administrator) - 2011-09-04 08:28
http://bugs.bacula.org/view.php?id=1762#c5969
----------------------------------------------------------------------
Thanks for your patch, unfortunately, the version browser has also some problems
to display files in certain circumstances (see other bugs in this tool), so
making it faster is nice, but we still think about deprecate it.
The 5.2.0rc1 version has a new restore tool that permits to do more or less the
same thing using other technologies (brestore panel), perhaps you can try it
first and give us feedback. (it uses a cache in the DB, and the build process
can take also some time depending on your site and the DB backend that you are
using)
If you still want to see your patch integrated with the current version, you
need to test it against the 5.2.0rc1 source. We no longer add features to 5.0.x
tree, and we usually don't add such kind of patch in a RC cycle (but we can
probably make an exception).
As it's not really a bug, but more a development issue, I advise you to use
bacula-devel mailing list to discuss your ideas (which are good, IMHO), and I
will close this bug.
----------------------------------------------------------------------
(0005971) jstarek (reporter) - 2011-09-04 13:31
http://bugs.bacula.org/view.php?id=1762#c5971
----------------------------------------------------------------------
Since you have a 5.0.4 release still pending, I don't understand why there is no
way to improve the 5.0 line with this patch.
I have another one in the queue that speeds up the job preparation (after you
hit restore) and fixes some crashes on termination on 5.0.3, too.
I'm willing to help - also on 5.2 when the current problems are fixed as far as
I need it.
I once tried a .bvfs_update on the 5.0.3 line and decided that this takes too
much time with the jobs we have...
----------------------------------------------------------------------
(0005972) kern (administrator) - 2011-09-04 20:16
http://bugs.bacula.org/view.php?id=1762#c5972
----------------------------------------------------------------------
Well, we don't really have a 5.0.4 pending. There are a few patches in
the repo for those who want to download them.
We are planning to deprecate the old version browser as Eric indicated, but
until is is removed in the next version, I do think it is reasonable to apply
patches
that are submitted.
However, in general, we prefer not to accept more patches for 5.0.3 code,
because
they will not be released in any 5.0 code, and it is more work for us to
integrate them into 5.2.0, so we would appreciate it if you
would make your future submissions based on 5.2.0.
You should probably test 5.2.0 as we have made *many* improvements to
bat stability (including bvfs).
----------------------------------------------------------------------
(0005975) ebollengier (administrator) - 2011-09-17 16:54
http://bugs.bacula.org/view.php?id=1762#c5975
----------------------------------------------------------------------
Patch applied.
Issue History
Date Modified Username Field Change
======================================================================
2011-09-03 23:09 jstarek New Issue
2011-09-03 23:09 jstarek File Added: bat-restore-tree-performance.patch
2011-09-04 08:28 ebollengier Note Added: 0005969
2011-09-04 08:28 ebollengier Priority high => normal
2011-09-04 08:28 ebollengier Severity major => minor
2011-09-04 08:28 ebollengier Status new => closed
2011-09-04 08:28 ebollengier Resolution open => no change
required
2011-09-04 13:31 jstarek Note Added: 0005971
2011-09-04 13:31 jstarek Status closed => feedback
2011-09-04 13:31 jstarek Resolution no change required =>
reopened
2011-09-04 20:16 kern Note Added: 0005972
2011-09-17 16:54 ebollengier Note Added: 0005975
2011-09-17 16:54 ebollengier Status feedback => closed
======================================================================
------------------------------------------------------------------------------
BlackBerry® DevCon Americas, Oct. 18-20, San Francisco, CA
http://p.sf.net/sfu/rim-devcon-copy2
_______________________________________________
Bacula-bugs mailing list
Bacula-bugs@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-bugs
[prev in list] [next in list] [prev in thread] [next in thread]
Configure |
About |
News |
Add a list |
Sponsored by KoreLogic