[prev in list] [next in list] [prev in thread] [next in thread] 

List:       wikitech-l
Subject:    Re: [Wikitech-l] [RFC] performance standards for new mediawiki features
From:       George Herbert <george.herbert () gmail ! com>
Date:       2013-03-27 3:28:25
Message-ID: CAK__Kzt_egCeVOczfUU7zP8Z8dd4VcBh6J_hUWp=YS0UxZByvw () mail ! gmail ! com
[Download RAW message or body]

On Tue, Mar 26, 2013 at 8:15 PM, Ryan Lane <rlane32@gmail.com> wrote:
> On Tue, Mar 26, 2013 at 3:58 PM, Asher Feldman <afeldman@wikimedia.org>wrote:
>
>> There are all good points, and we certainly do need better tooling for
>> individual developers.
>>
>> There are a lot of things a developer can do on just a laptop in terms of
>> profiling code, that if done consistently, could go a long way, even
>> without it looking anything like production.  Things like understanding if
>> algorithms or queries are O(n) or O(2^n), etc. and thinking about the
>> potential size of the relevant production data set might  be more useful at
>> that stage than raw numbers.  When it comes to gathering numbers in such an
>> environment, it would be helpful if either the mediawiki profiler could
>> gain an easy visualization interface appropriate for such environments, or
>> if we standardized around something like xdebug.
>>
>> The beta cluster has some potential as a performance test bed if only it
>> could gain a guarantee that the compute nodes it runs on aren't
>> oversubscribed or that the beta virts were otherwise consistently
>> resourced.  By running a set of performance benchmarks against beta and
>> production, we may be able to gain insight on how new features are likely
>> to perform.
>>
>>
> This is possible in newer versions of OpenStack, using scheduler hinting.
>
> That said, the instances would still be sharing a host with each other,
> which can cause some inconsistencies. We'd likely not want to use beta
> itself, but something that has limited access for performance testing
> purposes only, as we wouldn't want other unrelated testing load to skew
> results. Additionally, we'd want to make sure to avoid things like
> /data/project or /home (both of which beta is using), even once we've moved
> to a more stable shared storage solution, as it could very heavily skew
> results based on load from other projects.
>
> We need to upgrade to the Folsom release or greater for a few other
> features anyway, and enabling scheduler hinting is pretty simple. I'd say
> let's consider adding something like this to the Labs infrastructure
> roadmap once the upgrade happens and we've tested out the hinting feature.

I am concerned in this discussion with insufficient testbed load
generation and avoidance of confounding variables in the performance
analysis...



-- 
-george william herbert
george.herbert@gmail.com

_______________________________________________
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
[prev in list] [next in list] [prev in thread] [next in thread]