From kdevelop-devel  Sun Dec 20 23:33:22 2009
From: Hamish Rodda <rodda () kde ! org>
Date: Sun, 20 Dec 2009 23:33:22 +0000
To: kdevelop-devel
Subject: Re: On Memory consumption of KDevelop-PG
Message-Id: <200912211033.22360.rodda () kde ! org>
X-MARC-Message: https://marc.info/?l=kdevelop-devel&m=126135214420881

On Sun, 20 Dec 2009 06:19:36 am Alexander Dymo wrote:
> субота, 19-гру-2009 19:45:03 David Nolden ви написали:
> > And btw. how did you measure those 550MB? It sounds very much..
> 
> I've attached preprocessed.c file (it's not the actual source file, but
>  rather the output of the preprocessor because the source file is just a
>  not understandable huge collection of #defines).
> 
> Running duchainify on that will consume 551M memory. See attached massif
> output in ms.txt.
> 
> If you apply my patch (attached) that will prevent parser from storing
>  unused AST nodes, you'll get 400M (see ms_patched.txt). But that's all I
>  could get.
> 
> The patch currently breaks some duchain tests, but I haven't looked at that
> yet.
> 
> 
> PS: main reason for this huge memory consumption is that the file has a lot
>  of integer numerals inside.
> This gives about 1 000 000 primary expression ast's, roughly 88 bytes
>  teach. That's already about 90-100M.
> But each primary expression ast is nested into another ast's... Therefore
>  the number of ast nodes required for that file is multiplied (roughly) by
>  4 and that gives us 400M.
> 

Given the c++ parser is hand-written, we should probably just add a flag to 
the parser that prevents it from saving expression nodes to the AST, for most 
of the duchain processing - as I understand it is not required?  That should 
result in a dramatic saving for >90% of the processing we do... might also 
speed up the duchain creation...

Cheers,
Hamish.

-- 
KDevelop-devel mailing list
KDevelop-devel@kdevelop.org
https://barney.cs.uni-potsdam.de/mailman/listinfo/kdevelop-devel