Beware the Flaw of Averages in Asset Investment Planning

Beware the Flaw of Averages in Asset Investment Planning
Published Sunday, October 8, 2000, in the San Jose Mercury News

By D. Wayne McIntyre, August 2018

The commonplace problem of average assumptions often results in inaccurate Asset Investment Plans (AIPs). But there is a solution.

The drowning statistician

The cartoon of the statistician drowning while crossing a river that he calculated as being, on average, three feet deep is an excellent way to summarizing the problem facing so many AIPs. His death certificate would list the cause of death as the “flaw of averages”.

The flaw is that plans based on average assumptions will be wrong on average. This basic but almost always unseen flaw shows up everywhere you can find spreadsheets and financial models, undermining forecasts, under-estimating risks and resulting in apparently well-considered asset investment plans that drive bad investment decisions.

The Seven Deadly Sins of Averages

I only recently became aware of the Flaw of Averages, when I started working with a team of modeling experts.  Until then, I’d been relying on flawed models to make decisions. What’s more, I was often causing the problem by asking for my team to give me a firm number that I could manage to.

Apparently this is not uncommon, based on an HBR article by Sam Savage:

“Executives’ desire to work with “a number,” to plug in an average figure, is legendary. But whenever an average is used to represent an uncertain quantity, it ends up distorting the results because it ignores the impact of the inevitable variations. Averages routinely gum up accounting, investments, sales, production planning, even weather forecasting. Even the Generally Accepted Accounting Principles sanction the “flaw,” requiring that uncertainties such as bad debt be entered as single numbers. (To its credit, the SEC has proposed new rules that would begin to address this problem.)” You can find the article here.

The problem even has its own book, which is also written by Sam Savage, called the “The Flaw of Averages: Why We Understand Risk in The Face of Uncertainty”.

In the book, author Sam Savage states that “…when we use single numbers to estimate uncertain future outcomes…” such as assumptions based on historical equipment failure rates “…we are not just usually wrong, but we are consistently wrong.”

According to the book, there are eleven ”Averaging Sins” that are regularly committed in decision modeling, and the financial models that underpin most asset investment plans are equally guilty of these sins.

The Averaging Sins are:

  1. The family of 2.4 children: This average scenario does not exist. No one needs a 3.4 bedroom house. Unfortunately, most AIPs regularly use average scenarios.
  2. Why everything is behind schedule: Consider a construction project with 10 parallel tasks that each averaged six months. Setting each task at its average results in project completion in 6 months, but the chance that all 10 are completed at their average or sooner is like flipping ten heads in a row, so the chance of finishing by six months is less than 1 in 10,000. Sound familiar asset managers?
  3. The egg basket: If eggs in baskets have a 10{76a10d78d04e112156734d2d636dc893ae0a5fbdefde79b043bd28411ac9e222} chance of being broken, the average result of for a basket of 10 eggs is the same as 10 baskets containing 1 egg each. The risk of a total loss is 1 in 10 for the first scenario and 1 in 10,000,000,000 for the second. This is a big difference and the benefit of diversification. Risk is at the core of AIP, and while most plans have a risk scoring methodology, that mythology is fatally flawed by this “sin”.
  4. The risk of ranking: When planning capital investment projects, it is common to rank them from best to worst, then start at the top of the list when selecting projects. The projects at the bottom of the list don’t get selected when budgets are inevitably constrained. This flies in the face of Modern Portfolio Theory, which is based on the interdependence of investments. According to the ranking rule, fire insurance is a ridiculous investment because it loses money on average (that’s why the insurance business is so profitable). But is it really a good idea not to insure your house for fire?
  5. Ignoring restrictions: Suppose you plan your infrastructure maintenance budget to meet average forecasted deterioration or failures. If there are fewer problems than forecasted, you may find yourself with surplus maintenance capacity or reduced profitability. But if there are more problems than expected in a given time period than the average (and above your capacity to address) minimum service levels may not be met, capacity to produce may be lost or revenue generation may be constrained. Thus, there is some downside without a corresponding upside, and the average upside or profit is less than what would be forecasted based on averages.
  6. Ignoring optionality: if you are producing a product (e.g. energy) with an uncertain price, if the price drops below the marginal cost, you have the option to stop production to limit losses. If the price increases, you have the option to increase production to generate revenue and profit. These are upsides without a corresponding downside. If the value of the business is a function of price, the average value will be more than the value associated with the average price.
  7. The double whammy: if your goods are perishable and your production capacity is based on average demand, the cost of inventory management should be minimal. There is simply no waste. But below average demand will result in spoilage losses, and excess demand will result in additional costs such as sourcing alternative supply or the need to FedEx product to meet contractual commitments. So the cost associated with average demand is zero, but the average cost is positive.
  8. The Flaw of Extremes. Sandbagging is a classic example of the law of extremes. What rational investment planner or asset manager would plan for the best case? Typically, each individual puts in a risk buffet to ensure they can deliver, or even over-deliver. The impact becomes a very flawed budget when layers of unnecessary sand-bagging become increasingly thick as the budget moves through the organizational hierarchy.
  9. Simpson’s Paradox. This is a situation where a trend appears in several different groups of data but disappears when the groups are combined. This “averaging sin” is difficult to illustrate without some math, so I suggest that you click here to learn more (https://en.wikipedia.org/wiki/Simpson{76a10d78d04e112156734d2d636dc893ae0a5fbdefde79b043bd28411ac9e222}27s_paradox).
  10. Scholtes’ Revenue Fallacy: Most of us have committed this flaw without knowing it. Put simply, average percentages cannot be added…its just math. It is also something that DIREXYON regularly sees in asset investment plans. The fallacy is well explained in this blog if you want to learn more (http://blog.xuite.net/ecyyiu/hkblog/95376404-Research+Method+-+The+Flaw+of+Average{76a10d78d04e112156734d2d636dc893ae0a5fbdefde79b043bd28411ac9e222}3A+Scholtes+Revenue+Fallacy).
  11. Taking credit for chance occurrences: You would probably be surprised if someone tosses a coin 10 times and gets heads every time. The chances of this are 1-in-1024. This person must have some special coin tossing abilities, right? In reality, this is just chance, and should be expected when 1,000+ people are tossing coins.

The Solution To Better AIP

Fortunately, there is a solution to the Flaw of Averages. There are widely accepted data science techniques, such as Monte Carlo simulation, that address these problems by accounting fully for uncertainty. While these methods are complex, there are software solutions that allow you to drive better decision-making without getting a PhD in data science. My company, DIREXYON, develops this type of software and there are others in the market as well.

In a nutshell, software solutions like ours solve this problem by using powerful computers to simulate multiple possible futures and employ a wide range of inputs instead of single average values. Today, this technique, known as simulation, is at the center of such diverse activities as Wall Street investing and military defense planning. It is also best practice for AIP.

In an upcoming blog, we will describe how simulation, in particular Monte Carlo simulation and Markov Chains, is being used today by several leading electric utilities to drive better asset investment plans and decision-making.