[prev in list] [next in list] [prev in thread] [next in thread]
List: kde-core-devel
Subject: Reducing the number of conflicts in libraries
From: Lubos Lunak <l.lunak () sh ! cvut ! cz>
Date: 2001-09-17 18:33:02
[Download RAW message or body]
Hello,
until there's a gcc/binutils/whatever based solution to reduce the bloat
caused by conflicts, we could probably do some changes in KDE to reduce the
number of conflicts manually (especially if some of the things probably might
be needed anyway, and also, who knows when a gcc version doing something with
it will be available). In the following text, there are described some ways
how that could be done. Some of the changes described there are fairly
simple, while other (templates) may be a bit complicated (I'm not sure how
much actually). So, the two important questions are :
- Are some of the things below worth the trouble, and if yes, which ones?
- The second question is just like the first one, only aimed at folks from TT.
The following text is also available at
http://dforce.sh.cvut.cz/~seli/en/linking2/whatnow.txt , the list of things
causing conflicts ( c++filt-ed, sorted, uniq-ed ) is at
http://dforce.sh.cvut.cz/~seli/en/linking2/conflicts.txt . The list of
conflicts may be also obtained by running an executable linked to KDE
libraries with LD_TRACE_PRELINKING set (and glibc patched for prelinking) and
grepping the output for 'conflict', or by running
nm -D <library> | sed -n 's/^[^ ]* W \(.*\)$/\1/p' and checking which lines
appear in more than one library (any idea how to do this with some script or
something?).
Questions, opinions, flames ? ( volunteers ? :) )
Lubos Lunak
llunak@suse.cz ; l.lunak@kde.org
http://dforce.sh.cvut.cz/~seli
============
In case you haven't read http://dforce.sh.cvut.cz/~seli/en/linking2 ,
do so now. In the following text, I'll try to describe how to reduce
the number of conflicts in KDE libraries and applications until
a gcc/binutils based solution is available (actually, I it might be
that some of the problems won't be solvable in gcc/binutils, so some
of the things described below will be probably needed anyway).
Conflicts in KDE libraries are caused by these things:
- virtual tables, and all other virtual class related things like
thunks, out-of-line copies of inline virtual methods, typeinfo data
- inline functions
- templates
(I everywhere explicitly say if something is a template or belongs
to a class. In other words, 'inline functions' above means global
non-template functions that aren't members of any class. Also, 'virtual
class' means a non-template class that has at least one virtual method.)
Virtual classes :
===================
Conflicts here are caused by multiple copies of vtables and other virtual
class related things. As described in the linking2 document, gcc in most
cases generates only one copy, however there are cases where gcc doesn't
try hard enough, and there are also cases where gcc simply can't generate
only one copy. As explained in gcc info documentation, vtable is generated
only in the module where the first non-inline virtual method of the class
is implemented. This is often not enough. One example :
class KStaticDeleterBase {
public:
virtual void destructObject() = 0;
};
In the example, there is one virtual method, so vtable is needed, however,
because the method is pure virtual, it's not implemented anywhere. So gcc
has no other choice but to create the vtable whenever it's needed. Here,
the problem can be solved by changing KStaticDeleterBase to this :
class KStaticDeleterBase {
public:
virtual void destructObject() = 0;
private:
virtual void put_vtable_only_here();
};
and in kstaticdeleter.cpp :
void KStaticDeleterBase::put_vtable_only_here()
{
}
Now this makes things much easier for gcc - it can create the vtable only
in kstaticdeleter.o . Note that this hack breaks binary compatibility,
a solution that wouldn't have this disadvantage is to implement empty
KStaticDeleterBase::destructObject() in the source instead of adding
a new virtual method.
Second example :
class Q_EXPORT QGArray
{
// unimportant
virtual ~QGArray();
// unimportant
};
template<class type> class Q_EXPORT QArray : public QGArray
{
// only non-virtual methods here
~QArray() {}
// only non-virtual methods here
};
typedef QArray<char> QByteArray;
class Q_EXPORT QCString : public QByteArray
{
// no virtual method here, not even ~QCString()
// only non-virtual methods
};
QCString has a virtual method, its destructor (people, why are you so lazy
to repeat the 'virtual' keyword in derived classes? Writing the QArray
destructor as 'virtual ~QArray(){}' makes things easier to read, and it's
only few more keypresses). But ~QCString() is not implemented anywhere,
it will be generated automatically by the compiler. The situation is similar
with QArray, where the destructor is implemented, but it's inline (methods
implemented in the class body are treated as inline), so there's again no
.cpp file implementing it. Here gcc could be smarted and realize it can
generate the vtable in the module implementing the first non-virtual method
of the class, but it doesn't do so (feel free to send mails to your
favourite g++ developer asking when it will do so). Here again the hack
with private put_vtable_only_here() can help, the other choice is to
really implement the destructor as non-inline.
Out-of-line copies of inline virtual methods and typeinfo data are generated
together with the vtable, so this approach helps here too. But this doesn't
seem to apply to thunks these days.
Also, note that when the first virtual method in a class is not explicitly
marked as inline, but is later implemented as inline, this may lead to
conflicts. But this doesn't seem to be a common technique in KDE.
Inline functions :
====================
Inline functions are usually compiled directly into the code that calls them,
so they actually don't really exist in the code like normal functions,
unless this is needed for some reason (a so-called out-of-line copy
of inline function). And out-of-line copy is needed e.g. when compiling
without optimizations (i.e. without -O), or when a pointer to the function is
needed. Again, the compiler here must create a copy whenever an out-of-line
copy of an inline function is used. The only solution is not to create inline
functions to which pointers are taken. One example of such function is
endl(kdbgstream &) . A pointer to this function is taken when calling
kdbgstream& kdbgstream::operator<<(KDBGFUNC f).
Inline non-template global functions are only rarely causing conflicts.
Templates :
=============
Templates are causing a high number of conflicts in KDE. Example :
a.cpp :
void foo1()
{
// ...
QValueList< QString > some_list;
// ...
}
b.cpp :
void foo2()
{
// ...
QValueList< QString > some_other_list;
// ...
}
Here both in a.cpp and in b.cpp an instance of template QValueList< T > needs
to exist for T = QString. By default such instance is generated automatically
by compiler (implicit instantiation) whevener it's needed. So in the example
both in a.o and in b.o QValueList< QString > is created. But it already exists
in libqt, and therefore it's causing conflicts. The solution of course is
to create every instance of every template only once, so if
QValueList< QString > exists already in libqt, there shouldn't be another one
created elsewhere. This leads to three problems :
- where and how to create an instance of a template
- how to prevent other places from creating another (same) instance
- instances of templates may not disappear from libraries, otherwise binary
compatibility would be broken
Let's try to do something with QValueList< QString > in the example. Since it
already exists in libqt, there's no need to create it again elsewhere. One
way how to prevent creation of template instances is to use
'extern template QValueList< QString >;' at the beginning of both a.cpp and
b.cpp . This will prevent instantiation of QValueList< QString > in a.o and
b.o, and code that uses it will only reference it (i.e. it's similar
to 'extern void foo();' or 'extern int bar;' ). But declaring templates as
extern is a GCC extension (it gives a warning with -pedantic), so it might
be better to split template definitions into two parts : Declarations and
inlines in one part (i.e. things that don't create any real code) and
the rest in the other part.
If you look in qvaluelist.h, you'll notice that all template classes
in this header file have their methods implemented in the class body, which
automatically makes them inline, even if they aren't explicitly marked so.
Some of the methods are quite large and this causes larger executables.
Moreover gcc won't inline some of them (compile with -Winline to see),
because they are simply too large or for other reasons. Instead inlining
them, an out-of-line copy for them will be created, and such multiple
copies will cause conflicts. All large methods in templates should be moved
out of the class body, so they won't be treated as inline. Also, they
should be either enclosed by an #ifdef block, or they should be
in a separate source file. Since the definitions of non-inline template
methods won't be available, they won't be instantiated anywhere.
That should answer how to prevent other places from creating duplicate
copies. The question where and how the one single instance of template
should be created still remains. An instance can be created by putting
'template QValueList< QString >;' in one source file, after #including
a file with complete definition of QValueList< T >, including non-inline
methods. With older C++ compilers that can't handle this construct, one
uses 'typedef QValueList< QString > QValueListQString;' instead. The place
where one specific template instance should be created is of course
the library where it's first used, in this QValueList< QString > case,
it's libqt.
There are still two problems remaining. One of them is the fact that
QValueList< QString > may one day disappear from a newer Qt version
(e.g. because QMap< QString > is used instead it). This would break
binary compatibility for KDE libraries relying on QValueList< QString >
being present in libqt - after upgrading to this newer Qt version,
KDE libraries would cease to work. A solution would be to run a script
on every library version that would list all such symbols and make
sure that every new version contains all the symbols from the older version.
( nm -D <library> | sed -n 's/^[^ ]* W \(.*\)$/\1/p' | c++filt | grep '<' |
LC_ALL=C sort should print such list. I'm not actually sure what exactly
it does, but it seems to work). All missing symbols could be then explicitly
instantiated using the template keyword, e.g.
'template QString& QValueList< QString >::operator[](size_type);'. Just
in case you ask what the second problem is, the second problem would be
maintaining it. One would have to check all those (probably many) symbols
with every new library version. When simply using
'template QValueList< QString >;', this would lead to creating many probably
unused instances of template methods, which would lead to increasing
libraries sizes, on the other hand, if not all methods would be instantiated,
they might be needed in other libraries. Also, for every instance
of QValueList< T >, there must exists also one instance
of QValueListPrivate< T >, QValueListNode< T > and usually also
QValueListIterator< T > and QValueListConstIterator< T >.
There may be also another ways of handling templates that I'm not aware of,
e.g. using the 'export' keyword (which is not implemented in gcc yet). I'm
not exactly sure how it is supposed to work, and I had only a C++ draft dated
2 December 1996 available, so there may be some differences to the final
ISO C++ document.
----------------------------------------------------------------------
Lubos Lunak <llunak@suse.cz> <l.lunak@kde.org>
[prev in list] [next in list] [prev in thread] [next in thread]
Configure |
About |
News |
Add a list |
Sponsored by KoreLogic