[prev in list] [next in list] [prev in thread] [next in thread] 

List:       koffice-devel
Subject:    Re: KTarGz and streams
From:       David Faure <david () mandrakesoft ! com>
Date:       2001-07-29 22:52:14
[Download RAW message or body]

On Monday 30 July 2001 00:12, Pascal A. Niklaus wrote:
> 
> I'd like to save KCite database files 'tar | zipped'. I had a look at 
> KTarGz, but the only function I could find to add files takes a buffer 
> containing the whole file as argument. In my case, however, the files 
> contains a data base which can potentially be huge so that trying to load 
> it into memory is probably not a good idea. I think what I am looking for 
> is a way to bind an output stream to KTarGz.

This is what KOffice does, more or less, with KoStoreDevice, but this isn't really
helping. KoStore sort of buffers the data into memory, and writes it into the archive
all at once when the file is complete. This was done in order to avoid changing
KTar while still offering a QIODevice to kofficecore. But we ought to improve that
indeed :)

> - Is that possible? Or does KTarGz require that all files are fully loaded 
> into memory?

Not all files! Only the current file you are writing, using writeFile(). After
writeFile is done, the data is on disk, you can destroy the buffer.
But if a single file can be huge, then this is indeed not good enough.

> - If KTarGz will not work as I need it, is there another solution you could 
> suggest?  For the moment I try to run the systems tar over a KProcess, but 
> this involves other problems.

I'd avoid doing that, I'd rather suggest we work on improving KTar :-)

Let's see... exactly the kind of hack I was looking for right now :)
Done. The one thing this brings forward is that we need to know the
size of the data before hand, it's needed in the TAR header. Well, otherwise
we have to implement "going back in the stream and writing the size", a bit
of a problem with a compressed stream...
Hmm, I wanted to use this "write step by step" feature in koffice, but there
we don't know the size of the file before hand :(((

What do you think about the attached patch ?
(Untested, except that it compiles ;-)

-- 
David FAURE, david@mandrakesoft.com, faure@kde.org
http://perso.mandrakesoft.com/~david/, http://www.konqueror.org/
KDE, Making The Future of Computing Available Today

["ktar.diff" (text/x-diff)]

Index: ktar.cpp
===================================================================
RCS file: /home/kde/kdelibs/kio/ktar.cpp,v
retrieving revision 1.32
diff -u -p -r1.32 ktar.cpp
--- ktar.cpp	2001/06/26 20:22:37	1.32
+++ ktar.cpp	2001/07/29 23:50:26
@@ -373,18 +373,18 @@ void KTarBase::writeDir( const QString& 
   m_dirList.append( dirName ); // contains trailing slash
 }
 
-void KTarBase::writeFile( const QString& name, const QString& user, const QString& \
group, uint size, const char* data ) +bool KTarBase::prepareWriting( const QString& \
name, const QString& user, const QString& group, uint size )  {
   if ( !isOpened() )
   {
     qWarning( "KTarBase::writeFile: You must open the tar file before writing to \
                it\n");
-    return;
+    return false;
   }
 
   if ( m_mode != IO_WriteOnly )
   {
     qWarning( "KTarBase::writeFile: You must open the tar file for writing\n");
-    return;
+    return false;
   }
 
   // In some tar files we can find dir/./file => call cleanDirPath
@@ -433,14 +433,27 @@ void KTarBase::writeFile( const QString&
 
   // Write header
   write( buffer, 0x200 );
+  return true;
+}
+
+void KTarBase::writeFile( const QString& name, const QString& user, const QString& \
group, uint size, const char* data ) +{
+  if ( !prepareWriting( name, user, group, size ) )
+      return;
 
   // Write data
   write( data, size );
 
+  doneWriting( size );
+}
+
+void KTarBase::doneWriting( uint size )
+{
   // Write alignment
   int rest = size % 0x200;
   if ( rest )
   {
+    char buffer[ 0x201 ];
     for( uint i = 0; i < 0x200; ++i )
       buffer[i] = 0;
     write( buffer, 0x200 - rest );
Index: ktar.h
===================================================================
RCS file: /home/kde/kdelibs/kio/ktar.h,v
retrieving revision 1.16
diff -u -p -r1.16 ktar.h
--- ktar.h	2001/06/26 20:22:37	1.16
+++ ktar.h	2001/07/29 23:50:26
@@ -80,6 +80,19 @@ public:
   void writeFile( const QString& name, const QString& user, const QString& group, \
uint size, const char* data );  
   /**
+   * Here's another way of writing a file into a tar archive:
+   * Call @ref prepareWriting, then call write as many times as wanted,
+   * then call @ref doneWriting( totalSize )
+   * You need to know the size before hand, it is needed in the header!
+   */
+  bool prepareWriting( const QString& name, const QString& user, const QString& \
group, uint size ); +
+  /**
+   * Call @ref doneWriting after writing the data, @see prepareWriting
+   */
+  void doneWriting( uint size );
+
+  /**
    * If a tar file is opened for reading, then the contents
    * of the file can be accessed via this function.
    */


_______________________________________________
Koffice-devel mailing list
Koffice-devel@master.kde.org
http://master.kde.org/mailman/listinfo/koffice-devel


[prev in list] [next in list] [prev in thread] [next in thread]