[prev in list] [next in list] [prev in thread] [next in thread] 

List:       wine-devel
Subject:    Re: Elfdlls vs. built-in DLLs
From:       Bertho Stultiens <bertho () panter ! soci ! aau ! dk>
Date:       1999-08-31 21:39:40
[Download RAW message or body]

Ulrich Weigand wrote:
[snip]
> I'm not sure I follow this.  Sure, the parser in build is not as
> flexible as a yacc-generated one, but as long as you don't need any
> syntactical constructs more complex than those currently implemented,
> it should do just fine ...   

True, it does the job. It also does a lot of other things, which have
nothing to do with .spec-files. Build is a historic program which has
evolved into many things. Dinosaurs die(d)...

>The reason why I'd prefer a better parser
> is not because the old one doesn't work for what it does, but that a
> better one would allow a lot of interesting things, like having much
> more detailed argument type information, e.g. to do automatic structure
> conversion, or to have automatic generation of callback thunks ...

Serious syntactical change to .spec-files has been rejected by Alexandre
with the comment that it was a fix searching for a problem. I have no
clue why you want to have more detailed argument type info. All
automatic argument conversion (i.e. thunking compiler implementation)
has been rejected by Alexandre and said that you should do your own
thunking (I also wanted to implement this in dllglue once, a long time
ago).

> > Most is described in the white-paper from last December (elfdll.txt),
> > although some internals have undergone changes. I think that the
> > built-in dlls are an awfull hack and should vanish, but this is not an
> > option yet because it is too complex at the moment (see below).
> 
> Elfdlls and built-in DLLs are at the core exactly the same thing.
> The difference is mostly one of implementation details.

True and false.
The effects may be the same superficially, but the design strategy is
completely different. Built-in dlls are an integral part of the core of
wine without any possibility of other use. OTOH, elfdll are designed to
be seperate entities with the *option* of being compiled into the core.

I have made this clear (I hope), from day one, that I designed elfdlls
to do exactly what they are supposed to do: behave like seperate
entities.

This strategy has consequences how you integrate it into current wine.
Main consequence is that monolithic compilation is to become the
exception and not the rule. Now, we cannot expect this to happen
overnight and I have suggested a transitional path to get there
*without* compromising the elfdll structure in itself. You must not
denote (nor implement) a dll as elfdll when it in fact is a hacked
built-in dll. This is my main objection of modifying the tool in favour
of the code just to use the new tool.

[snip: multiple export tables]
> It is definitely *not* possible, for exactly the reason stated above
> (several export tables).  Note that I am saying every dlls/ subdirectory
> should get its own .so file, *not* every single built-in DLL.  This is
> not the same thing.

This is possible under the condition that you do not use importtables.
Another problem here is that you must keep the complete dll in the core
to function properly.

In fact, you are using dllglue to implement built-in dlls; and this is
exactly what I want to prevent.

[snip dependencies]
> Yes, the dependency of COMMDLG on USER is the single most difficult
> point to resolve (all other dependencies on USER either have already
> been resolved or are trivial).  This is why we should do it *after*
> everything else, not before.  (Even in COMMDLG, most of the dependencies
> are simple, solved by switching to 32-bit dialog procedures and using
> 32-bit API routines.  A few difficult points remain, like calling
> 16-bit hook procedures from 32-bit code with native USER.)

This is the core of the problem. This should be solved! And yes, it is
painfull, but absolutely necessary to start working on.


snip]
> > I know that it is problematic to update multiple places all the time,
> > but, as I said before, I rather live with some overhead now than that I
> > have to wait until everything is reorganized so that dllglue can replace
> > the parsing part of build.
> 
> This is why I am suggesting that dllglue should replace the parsing part
> of build *now* if ever.  The very point is exactly *not* to wait until
> everything is reorganized!

I agree. See below.

> [snip]
> > Note 2: was the elf-loader adapted to use the double linked modref list?
> > If so, why not all of the code?
> 
> As far as I can reconstruct it from the logs, I've switched the
> modref list to be doubly-linked on Mar 27.  The first version of
> ELFDLL_CreateModref in the tree dates from Apr 18, and this (and
> all subsequent versions) use a singly-linked modref list throughout.
> I don't know why this is so.  Maybe you copied ELFDLL_CreateModref
> from an old version of PE_CreateModref?

Probably I took a very old version. I always said that the code was a
poor hack and was only implemented to show the basic functionality
(actually to show the example of elfdll in winelib IIRC). The loader
needs more work to unify the different loaders and streamline it
completely (also necessary for unloading modules).

[snip]
> > Well, you are welcome to hack build so that it uses the front-end of
> > dllglue, but I have a feeling that you will have some trouble fitting it
> > in. OTOH, it would solve the problem of a double parser.
> 
> Having the *new* parser code duplicated in two tools is just as bad!
> If we decide to place the parser in 'build' this will mean that
> everything else will also be placed in 'build' and no separate 'dllglue'
> tool will ever exist.  If we decide to move the parser to 'dllglue'
> then this 'dllglue' will have the name 'dllglue' but at that point
> will be *different* from the dllglue existing now, as it will
> generate output identical to what build outputs now (i.e. a
> WIN32_DESCRIPTOR structure, and not a PE header).  With the very
> same patch that the new tool is introduced, the parser will be
> removed from build.

Argh, ever heard about a feature called code-sharing and the like? The
spec-parser is a seperate front-end which can be used in *both* build
and dllglue. Just make the proper makefile bindings and it works.

But please wait until the parser is *stable*.

> > > - If necessary and/or useful, switch to generation of PE headers
> > >   instead of DLL descriptors, both in the new tool (modified build
> > >   or new dllglue) and the built-in loader.
> >
> > This would be unnecessary because this will happen automatically when
> > you switch to dllglue. Build will never need this and I strongly doubt
> > that it is worth the effort to hack build and built-in dlls when you
> > know that they are about to be flushed.
> 
> There will be no 'switch'.  The very *point* of the suggested
> modification path is that there is never ever a single 'switch' point
> between an 'old' and a 'new' implementation!  Instead, there will be
> a sequence of modifications, each of which addresses one single
> problem to be fixed or feature to be added.

My point is that build != dllglue. The 'switch' (in my understanding) is
the transition of a dll from built-in -> elfdll.

> [snip the rest of your comments, they all apparently show the
>  same fundamental misunderstanding of what I was trying to say ...]

...

> [snip]
> > There is another problem with this transition in that it requires
> > 'objcopy' which I came to understand does not work for a.out systems
> > (*Bsd). Here you can imagine to make no importtables at all an link
> > everything statically because you cannot generate localized
> > indirections. This approach also prevent native user/user32 and others
> > form working properly, but that is just as far as we are now.
> 
> (Reasonably recent versions of) both FreeBSD and OpenBSD use ELF.
> I'm not sure about NetBSD, but then again, we don't support NetBSD
> anyway at the moment because of lack of thread support.

That is no excuse to throw away the fundamental idea. I don't know
whether there are a.out systems out there, but I can surely imagine that
someone at some stage wants to compile the stuff as one monolithic piece
of code for whetever reason. I want to be prepared now because I see it
coming.

> [snip]
> > > Note that this path has the IMO important advantage that after every
> > > single step, all the code modified/introduced by this step is
> > > immediately in main use, so that any bugs will be immediately
> > > noticed, and all advantages of the new mechanisms can be immediately
> > > used to their full extent; furthermore at no point will duplicate
> > > code have to be maintained ...
> >
> > Well, my proposal is slightly less ambitious.
> 
> These features are not all that ambitious, they are just the IMHO most
> basic requirements for any modifications to a huge project like Wine.

The point where you go too fast (IMO) is from using build to using
dllglue. This is much more work because it requires work on an
individual dll (piece of code).

> Plus, they are in fact reasonably easy to achieve, as I had hoped to
> point out by explicitly describing one possible migration path.

dito.

> [snip]
> > Much of the code is there, yes, but certainly not all of it and I doubt
> > that it can be done in a couple of weeks. I'd rather take a couple of
> > months and do it right than hack fast now and have to redo it again (and
> > again...).
> 
> I just cannot understand why you are that much opposed to doing
> development *inside* the Wine tree.  You are not supposed to put
> a new feature into the tree only after is completely finished,
> after years of development.  You are supposed to place it into
> the tree at the *start* of development, so that development can
> take place inside the tree.  *Of course* that means that the code
> is changed over and over again.  That's what development is all
> about. (With 'inside the tree' I mean of course *not* only 'well,
> it's in there, but it is never used in a default compilation',
> but that it is in active use throughout the complete development.)

Eh, you do not place half a patch in the queue, do you? The tool is not
doing too well at the moment. Example: it fails on every exportname with
invalid C identifiers... Seems very basic to me.

I have been providing these snapshots to give some insides and ask for
comments. I really want to implement the basics before I submit it.

BTW, you do not integrate your x86 emulator in the tree? I wonder why?

[snip]
> > (- adapt build to use the lex/yacc frontend .spec-parser; something for
> > you Ulrich?);
> 
> Sigh.  What exacly is the point of putting the parser into build if
> it is in dllglue at the same time?  Putting it into build makes sense
> only if build remains the only tool to be used, and there is never a
> dllglue.

Sigh, see above.


[snip]
> > So, when is this finished? My guess is around June next year when I look
> > at the amount of work.
> 
> ... and this is exactly the reason why it is *bad* if development
> takes place off-line (either completely out of the tree or in a
> not-used mode).

I was refering to "when is it finished after I place dllglue into the
tree". Well, several months, not weeks. Otherwise you would have to
clone yourself one more time :-)

[snip]
> You might have noticed that the differences between Wine and WineLib
> have almost completely vanished.  This was one of the goals of the
> various modifications to the 16-bit glue code and startup code I've
> been doing over the last few months.

Yes, it has been improving a lot and we are getting there.

> It appears that we are talking past each other.  

Partly. Basically you want a compromise in the tool which I don't want
in.

> To maybe show
> better what I mean, I'll try and create a version of dllglue that
> can be used as a drop-in replacement for 'build -spec'; this is
> the first step of the path I was talking about.

Well, I am not making a drop-in replacement for build. That is the
point: build != dllglue. See comments above. The parser is still a
moving target and I am still not sure that it is parsing all things
correctly all the time or in a way that is `smartī.

Please be patient.

Greetings Bertho


=========================================================================

[prev in list] [next in list] [prev in thread] [next in thread]