[prev in list] [next in list] [prev in thread] [next in thread] 

List:       koffice-devel
Subject:    Re: profiling loading of office documents
From:       Jos van den Oever <jos.van.den.oever () kogmbh ! com>
Date:       2010-08-09 9:05:24
Message-ID: 201008091423.24843.jos.van.den.oever () kogmbh ! com
[Download RAW message or body]

On Monday, August 09, 2010 14:06:13 pm Ariya Hidayat wrote:
> Very cool stuff!
> 
> > Yes, this is something I have been thinking about but have not actually
> > suggested because there are some tough issues with this. The measurement
> > will never be exact and the profiling will have to be run on exactly the
> > same machine each time.
> 
> One idea is to do it like WebKit's SunSpider runner script (Perl): run
> each test 5 times (I know it may take a long time) and find out the
> 95% confidence interval. If it is beyond a certain threshold, e.g. > 5
> %, then mark the test as "not confident enough". Thus, only comparing
> two confident outcomes will be more or less conclusive.

Yes, I guess that is easy enough to do. Alternatively, one outputs the timings 
of all 5 runs, so one can calculate appropriate statistical measures, e.g. the 
interquartile range [1]. If the interquartile ranges of the reference run and 
this run overlap, then no change is reported.

The requirement to always use the same machine stands though, since I do not 
think clock() reports accurate numbers across machines.

Cheers,
Jos

[1] http://en.wikipedia.org/wiki/Interquartile_range

-- 
Jos van den Oever, software architect
+49 391 25 19 15 53
http://kogmbh.com/legal/
_______________________________________________
koffice-devel mailing list
koffice-devel@kde.org
https://mail.kde.org/mailman/listinfo/koffice-devel
[prev in list] [next in list] [prev in thread] [next in thread]