[prev in list] [next in list] [prev in thread] [next in thread]
List: pgsql-performance
Subject: Re: [PERFORM] load / stress testing
From: Jerry Champlin <jchamplin () absolute-performance ! com>
Date: 2009-07-31 14:04:45
Message-ID: 4A72F9FD.9020908 () absolute-performance ! com
[Download RAW message or body]
Chris:
There are a number of solutions on the market. A company called Soasta
has a cloud based testing solution, my company has a product/solution
that's highly customizable called StressWalk. On another computer I
have a larger list with some ranking info I can send if you are interested.
Having been involved in a number of stress tests for customers, I think
the most important questions in picking a service to help you get this
done are:
* How much value am I likely to get by doing the testing? (If the
app fails in production because your team missed something
fundamental and it takes a day to fix via EIP, how much will that
cost your company in lost revenue and intangibles like reputation?)
* How much money can I afford to spend?
* Do I have good instrumentation in place for my application,
database, infrastructure, etc that I can easily correlate with the
graphs produced by the stress testing application.
* Am I looking for a rubber stamp or real analysis of what's going
on -- do I or my team have the time, expertise and experience in
application, database and infrastructure performance optimization
to do the analysis ourselves?
* Is my application UI difficult to stress test? (ie. Active X
controls, Flex, Java Applet, SilverLight, Cross-site AJAX)
* To create a realistic load profile, do I need to simulate one user
scenario or several?
* Does my application have user session and permission caching
code? If yes, you may need to utilize a solution that can use
large number of user credentials with different permission sets.
If you do not have the users created already and it's a large
enough population, you may want to use the stress testing
service/tool to create and configure the users for your test in
an automated way.
This should get you started. However, if you are leaning toward a
service -- my analysis is that there are a few services priced under
$1000 but they do not look very useful. The services we have found that
look like they can provide some good value to their customers are in the
$2500 - $15,000 range. For really complex stuff like application
components interacting with mainframes and synchronized data extract and
load to facilitate the work, you can expect things from the top end of
that range up to $25-30K.
I'd be happy to have a conversation if you are serious about solving
this problem.
-Jerry
Jerry Champlin
Absolute Performance Inc.
O: (303) 443-7000 x501
C: (303) 588-2547
jchamplin@absolute-performance.com
Chris wrote:
> Hi,
>
> Everyone says "load test using your app" - out of interest how does
> everyone do that at the database level?
>
> I've tried playr (https://area51.myyearbook.com/trac.cgi/wiki/Playr)
> but haven't been able to get it working properly. I'm not sure what
> other tools are available.
>
> TIA.
[Attachment #3 (text/html)]
<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
<meta content="text/html;charset=ISO-8859-1" http-equiv="Content-Type">
</head>
<body bgcolor="#ffffff" text="#000000">
Chris:<br>
<br>
There are a number of solutions on the market. A company called Soasta
has a cloud based testing solution, my company has a product/solution
that's highly customizable called StressWalk. On another computer I
have a larger list with some ranking info I can send if you are
interested.<br>
<br>
Having been involved in a number of stress tests for customers, I think
the most important questions in picking a service to help you get this
done are:<br>
<ul>
<li>How much value am I likely to get by doing the testing? (If the
app fails in production because your team missed something fundamental
and it takes a day to fix via EIP, how much will that cost your company
in lost revenue and intangibles like reputation?) <br>
</li>
<li>How much money can I afford to spend?</li>
<li>Do I have good instrumentation in place for my application,
database, infrastructure, etc that I can easily correlate with the
graphs produced by the stress testing application.</li>
<li>Am I looking for a rubber stamp or real analysis of what's going
on -- do I or my team have the time, expertise and experience in
application, database and infrastructure performance optimization to do
the analysis ourselves?</li>
<li>Is my application UI difficult to stress test? (ie. Active X
controls, Flex, Java Applet, SilverLight, Cross-site AJAX)</li>
<li>To create a realistic load profile, do I need to simulate one
user scenario or several?</li>
<li>Does my application have user session and permission caching
code? If yes, you may need to utilize a solution that can use large
number of user credentials with different permission sets. If you do
not have the users created already and it's a large enough population,
you may want to use the stress testing service/tool to create and
configure the users for your test in an automated way.</li>
</ul>
This should get you started. However, if you are leaning toward a
service -- my analysis is that there are a few services priced under
$1000 but they do not look very useful. The services we have found
that look like they can provide some good value to their customers are
in the $2500 - $15,000 range. For really complex stuff like
application components interacting with mainframes and synchronized
data extract and load to facilitate the work, you can expect things
from the top end of that range up to $25-30K.<br>
<br>
I'd be happy to have a conversation if you are serious about solving
this problem.<br>
<br>
-Jerry<br>
<pre class="moz-signature" cols="72">Jerry Champlin
Absolute Performance Inc.
O: (303) 443-7000 x501
C: (303) 588-2547
<a class="moz-txt-link-abbreviated" \
href="mailto:jchamplin@absolute-performance.com">jchamplin@absolute-performance.com</a>
</pre>
<br>
<br>
Chris wrote:
<blockquote cite="mid:4A72941D.7000907@gmail.com" type="cite">Hi,
<br>
<br>
Everyone says "load test using your app" - out of interest how does
everyone do that at the database level?
<br>
<br>
I've tried playr (<a class="moz-txt-link-freetext" \
href="https://area51.myyearbook.com/trac.cgi/wiki/Playr">https://area51.myyearbook.com/trac.cgi/wiki/Playr</a>)
but haven't been able to get it working properly. I'm not sure what
other tools are available.
<br>
<br>
TIA.
<br>
</blockquote>
</body>
</html>
[prev in list] [next in list] [prev in thread] [next in thread]
Configure |
About |
News |
Add a list |
Sponsored by KoreLogic