[prev in list] [next in list] [prev in thread] [next in thread] 

List:       webkit-dev
Subject:    Re: [webkit-dev] Iterating SunSpider
From:       Mike Belshe <mike () belshe ! com>
Date:       2009-07-08 2:04:19
Message-ID: 2a10ed240907071904y67f4a7adkedbef945e0ba9154 () mail ! gmail ! com
[Download RAW message or body]

[Attachment #2 (multipart/alternative)]


On Tue, Jul 7, 2009 at 7:01 PM, Maciej Stachowiak <mjs@apple.com> wrote:

> 
> On Jul 7, 2009, at 6:43 PM, Mike Belshe wrote:
> 
> 
> > (There are other benchmarks that use summation, for example iBench, though
> > I am not sure these are examples of excellent benchmarks. Any benchmark that
> > consists of a single test also implicitly uses summation. I'm not sure what
> > other benchmarks do is as relevant of the technical merits.)
> > 
> > Hehe - I don't think anyone has iBench except apple :-)
> > 
> 
> This is now extremely tangential to the original point, but iBench is
> available to the general public here: <
> http://www.lionbridge.com/lionbridge/en-US/services/software-product-engineering/testing-veritest/benchmark-software.htm
> 
> > 


Thanks!


> 
> 
> A lot of research has been put into benchmarking over the years; there is
> > good reason for these choices, and they aren't arbitrary.  I have not seen
> > research indicating that summing of scores is statistically useful, but
> > there are plenty that have chosen geometric means.
> > 
> 
> 
> I think we're starting to repeat our positions at this point, without
> adding new information or really persuading each other.
> 
> If you have research that shows statistical benefits to geometric mean
> scoring, or other new information to add, I would welcome it.


Only what is already on this thread or google for "geometric mean
benchmark".

Mike


> 
> 
> 
> Regards,
> Maciej
> 
> 


[Attachment #5 (text/html)]

<br><br><div class="gmail_quote">On Tue, Jul 7, 2009 at 7:01 PM, Maciej Stachowiak \
<span dir="ltr">&lt;<a href="mailto:mjs@apple.com">mjs@apple.com</a>&gt;</span> \
wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px \
#ccc solid;padding-left:1ex;"> <div class="im"><br>
On Jul 7, 2009, at 6:43 PM, Mike Belshe wrote:<br>
<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc \
solid;padding-left:1ex"> <br>
(There are other benchmarks that use summation, for example iBench, though I am not \
sure these are examples of excellent benchmarks. Any benchmark that consists of a \
single test also implicitly uses summation. I&#39;m not sure what other benchmarks do \
is as relevant of the technical merits.)<br>

<br>
Hehe - I don&#39;t think anyone has iBench except apple :-)<br>
</blockquote>
<br></div>
This is now extremely tangential to the original point, but iBench is available to \
the general public here: &lt;<a \
href="http://www.lionbridge.com/lionbridge/en-US/services/software-product-engineering/testing-veritest/benchmark-software.htm" \
target="_blank">http://www.lionbridge.com/lionbridge/en-US/services/software-product-engineering/testing-veritest/benchmark-software.htm</a>&gt;</blockquote>
 <div><br></div><div>Thanks!</div><div> </div><blockquote class="gmail_quote" \
style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;"><div \
class="im"><br> <br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc \
solid;padding-left:1ex"> A lot of research has been put into benchmarking over the \
years; there is good reason for these choices, and they aren&#39;t arbitrary.  I have \
not seen research indicating that summing of scores is statistically useful, but \
there are plenty that have chosen geometric means.<br>

</blockquote>
<br>
<br></div>
I think we&#39;re starting to repeat our positions at this point, without adding new \
information or really persuading each other.<br> <br>
If you have research that shows statistical benefits to geometric mean scoring, or \
other new information to add, I would welcome \
it.</blockquote><div><br></div><div>Only what is already on this thread or google for \
&quot;geometric mean benchmark&quot;.</div> <div><br></div><div>Mike</div><div> \
</div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc \
solid;padding-left:1ex;"><br> <br>
<br>
Regards,<br><font color="#888888">
Maciej<br>
<br>
</font></blockquote></div><br>



_______________________________________________
webkit-dev mailing list
webkit-dev@lists.webkit.org
http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev


[prev in list] [next in list] [prev in thread] [next in thread]