[prev in list] [next in list] [prev in thread] [next in thread]
List: kdevelop-devel
Subject: Re: On Memory consumption of KDevelop-PG
From: David Nolden <zwabel () googlemail ! com>
Date: 2009-12-19 17:45:03
Message-ID: 200912191845.03718.zwabel () googlemail ! com
[Download RAW message or body]
Am Samstag 19 Dezember 2009 17:36:59 schrieb Alexander Dymo:
> субота, 19-гру-2009 11:49:47 Andreas Pakulat ви написали:
> > IMHO its no big deal if the parser needs some 200M for parsing a single
> > file as long as afterwards the memory is free again for re-use. In
> > particular it must not be fragmented (which should already work due to
> > usage of the memory pool). For how to make the AST smaller see further
> > down on your "compress the AST" question.
>
> Well, but now huge parser memory consumption is an issue (at least for
> C++). I have 3.5M C file which is parsed into 550M AST.
> 550M isn't that small amount of memory...
>
> And that's just one file. Try parsing Ruby language sources - you'll end up
> using 3G of memory for parsing. That's definitely wrong. Not everybody has
> this much of memory.
>
> There're actually bugs in c++ parser where it creates AST nodes which
> aren't used (added to the tree). I have a patch (not public yet, still
> breaks some duchain tests) for that which reduces the memory consumption
> in my test case from 550M to 400M. But that's all I can do without doing
> AST compression or removing members from AST classes.
There is a few things we can do in future that would have a huge impact:
- Replace size_t with uint everywhere within the parser
- Move the "AST* parent" and "DUContext* ducontext" members out of the AST,
into a separate map in ParseSession, as they are not needed in most cases.
- Do the same thing with the "comments" member of CommentAST
Together, these would probably reduce the size of the AST by 2/3.
However I consider this all that important atm, because the performance needs
to be considered carefully when doing some of these changes, and KDevelop4
right now anyway needs a fairly modern machine, which makes the memory-
requirements not all that fatal. The more important issues right now are
stability and usability.
And btw. how did you measure those 550MB? It sounds very much..
--
KDevelop-devel mailing list
KDevelop-devel@kdevelop.org
https://barney.cs.uni-potsdam.de/mailman/listinfo/kdevelop-devel
[prev in list] [next in list] [prev in thread] [next in thread]
Configure |
About |
News |
Add a list |
Sponsored by KoreLogic