Estimate, Testimate, Guesstimate
Estimating the test effort is often one of the first tasks one has to face when joining a new project. The project manager wants to know how much time and effort of the test periods (sometimes all from Unit to Acceptance Test), and your job is to find these numbers for her/him.
The most common estimating method is the Sinatra method, also known as “I did it my way”. I prefer to use some known methologies for estimation. It’s important to use more than one method, so that you can compare and see if you get (more or less) the same results. Also, remember that you estimate the number of hours, not the duration. There are lot of dependencies in a project, and if you have to wait one week for a programmer who has other priorities, you may allready be too late if you estimated the duration.
It is important to include estimating assumptions. If you write “testing product A will take 500 hours”, you may have promised more than you wanted to… Product A may change during development time, or perhaps someone forgot to give you the entire architecture, so there was more to test than you thought? Thus, when describing your estimate, write down which parts are included.
Prediction is very difficult, especially about the future
The first two methods are probably the most common for estimating. The important thing about these two, is to use both!
Bottom-up and Top-down
In Bottom-up estimation, you “split as much as possible”. Separate the software into as tiny bits as possible. The smaller the bits, the easier it is to see how much time it will take to test them. You should also take into consideration how complex each part is, which may add to testing effort. Integration points increase the amount of testing needed. When estimation has been done for each bit, these are summed to see the overall estimate. This method provides a good start for a burndown chart, by the way…
In Top-down estimation, experience/comparison is often used. Comparing to other, similar projects Using Work Breakdown Structure (WBS), you consider the various parts of the system which is due for testing, and then estimate each part. Remember to include documentation, demo, defect handling, retesting, regression test etc.
“One of the most robust findings in forecasting, human judgment, and software estimation studies is that “combination works.”5 Apparently it doesn’t matter whether the combination involves a simple average of estimates from different methods or a sophisticated weighting algorithm. A simple average offers a robust combination method unless one estimation method or expert is obviously more reliable than another. As shown elsewhere,6 though, an expert’s technical skill level can be a poor indicator of accuracy, and it’s rarely obvious, in advance, which expert will be the better estimator. This is one reason a simple average of outputs from different estimation experts and methods frequently offers the most robust and accurate combination method.” (Jørgensen, M., Simula Research Laboratory, “Practical Guidelines for Expert-Judgment-Based Software Effort Estimation”, 2005).
With Parametric estimating, you use the relationship and similarities which exists for each part subject to the estimation. First, come to an agreement regarding how many units it will take to complete a particular task. Then, compare similar parts using the technique, to find number of units needed. One exampler of parametric sizing, is the Planning Poker used in Scrum.
Expert Judgment (Delphi)
- Used to assess input to the estimating process Subjective opinion.
- Expertise is provided by any group or individual with specialised knowledge or training.
- Watch out for consensus and arguments.
Three point estimation
The expected completion times are found by using the three time estimates: Optimistic/minimum, pessimistic/maximum and most likely.
First, Make three estimates: Optimistic + Pessimistic + Expected. Then use the formula, to find the expected completion time: M = (min + 3x exp + max) / 5
This method can be used with common sense, but it is nicely combined with Expert Judgment (Delphi).