Greater Danbury Section – The “Flaw of Averages” — How Our Methods Doom Our Estimates And What We Can Do About It

Dave Northcutt, ASQ Mid-Hudson Section

ASQ Senior Member & ASQ Certified Quality Engineer, INFORMS Certified Analytics Professional

Estimates are critical to most, if not all, businesses.  Despite the critical nature of estimates, most of the techniques do not give good results.  As a result, we jeopardize schedules, profits, customer relationships, and employee morale far too frequently.  Why are our estimates so bad?

Many of the estimates that we rely upon are generated by using “average” assumptions to give expected outcomes.  Average inputs should give average results, right?  Wrong!  In almost any system of reasonable complexity, average inputs will not generate average outputs.  As a result, many of our estimates are overly optimistic and hence overly risky.  Even in those rare situations where average inputs do generate average results, the actual outcome will almost never be at the average that was estimated.  Therefore, it is critical for good decision making to generate estimates that include a view of the likely variability of the outcome.  Decisions made without knowledge of likely variability are inherently risky and doomed to fail.

There is a better way.  Dave Northcutt has successfully used relatively simple techniques to dramatically improve the quality of estimates over a 35-year career.

In this presentation, Dave will discuss:

• Why properly understanding historical data is critical for building good estimates
• Why using averages and static analyses to produce estimates often leads to incorrect results and provides no insight into the range of possible outcomes
• Why understanding the likely variability in our assumptions—and how that variability interacts—is critical to building a realistic model for estimating
• How relatively simple techniques such as Statistical Process Control and Monte Carlo Simulation can dramatically improve both our estimates and our insight
• How these techniques can help us understand and plan for the inevitable risk that exists in every estimation exercise

Dave Northcutt is a retired Industrial Statistician with over 35 years of experience driving improvements in the IT industry.  Dave was most recently an IBM Distinguished Engineer specializing in data analysis, predictive modelling, and continual improvement in IBM’s services business.  Dave has worked with teams around the world to help them improve their businesses by helping them to understand their data.  Throughout his career, Dave has always sought to make the teams he worked with self-sufficient, not simply give them the answers.  Dave holds Master’s degrees in Economics (Northwestern), Computer Science (Univ. of Illinois at Chicago), and Statistics (Rutgers).