Why the simple average of forecasts is hard to beat

Note · Statistics, forecasting · October 2026

What estimated combination weights cost, when they start to pay, and what breaks them.


Question

Combining forecasts lowers the error, and theory says how: weight each forecast by what it adds, given how its errors move with the others (Bates and Granger, 1969). In applied work the weights that theory recommends keep losing to the plain average. Stock and Watson (2004) named this the forecast combination puzzle, and Smith and Wallis (2009) and Claeskens, Magnus, Vasnev and Wang (2016) traced it to the error in the estimated weights.

This note puts numbers on that explanation. Estimation error has an exact price, and for the sample sizes of macro forecasting the price is often lower than the puzzle suggests. What does more damage in the simulations is a covariance of forecast errors that moves between the sample and the period of use.

Setup. There are K unbiased forecasts of the same variable, with errors of covariance Σ. A combination with weights w that sum to one has mean squared error w'Σw. The best weights and their error are

w* = Σ−11 / (1'Σ−11),   MSE* = 1 / (1'Σ−11)

and the simple average has MSEavg = 1'Σ1 / K2. Call g = MSEavg / MSE* the gain on offer: g = 1.10 means the average carries 10% more squared error than the optimum. With equal variances and one common correlation, g is exactly 1 and the average is the optimum.


The cost of estimating the weights

In practice Σ is replaced by the second moments of T past errors. With Gaussian errors the expected out-of-sample error of the resulting weights is

E[MSE(ŵ)] = MSE* × (T − 1) / (T − K)

The cost depends on the number of forecasts and the number of observations and on nothing else. No feature of Σ enters. It is the same algebra as the estimated minimum-variance portfolio, with forecast errors in the place of returns.

Exhibit 1 Extra squared error of estimated weights over the true optimum, by sample size. Simulation.
Table 1 Extra squared error over the optimum: simulation against the formula, and inverse-MSE weights

20,000 samples per cell. Errors have a common correlation of 0.6 and variances spaced from 1 to 2. Inverse-MSE weights are proportional to 1 over each forecast's own mean squared error and ignore correlations.

Inverse-MSE weights cost almost nothing to estimate, and their error barely changes with T. They also stop short of the optimum for good, because they ignore the correlations. In this design they stay about 8% above it with four forecasts however long the sample.


When estimated weights start to pay

Estimated weights beat the simple average in expectation when (T − 1)/(T − K) is below g. Solving for the sample size:

T* = K + (K − 1) / (g − 1)

Table 2 Break-even number of observations T*, by number of forecasts and gain on offer

Below T* the simple average is expected to win. A quarterly series gives 40 observations in ten years and 120 in thirty.

Table 3 Simulation check: estimated weights at T*, where their extra error should equal the gain on offer

How large is g? It is set by how unequal the forecasts are and how correlated their errors.

Table 4 Gain on offer g − 1, by design. In brackets, the smallest optimal weight.

Variance ratio is the error variance of the worst forecast over the best, with the others spaced geometrically between.

Two readings. When forecasts are of similar quality (a variance ratio of 1.5) the gain is a few percent: with four forecasts T* is 61 quarters at a correlation of 0.6 and 134 with uncorrelated errors. When errors are highly correlated and quality differs, the gain is large, and it comes from negative weights on the weaker forecasts: the optimum is a spread between models. That is where a wrong Σ hurts most.


Shrinking toward the average

Take a share λ of the estimated weights and 1 − λ of the simple average. Estimated weights are unbiased for w*, so the two sources of loss add:

E[MSE(λ)] / MSE* = 1 + λ2c + (1 − λ)2(g − 1),   c = (K − 1)/(T − K)
λ* = (g − 1) / (g − 1 + c)

Exhibit 2

The best mix is never at either end: as long as g is above 1 and the sample is finite, some shrinkage beats both pure choices. The catch is that λ* needs g, which is as unknown as Σ. The curve is flat near its minimum, so a fixed λ of one half gives up little.


What breaks the estimate

The formula assumes Gaussian errors and a Σ that holds from the estimation sample into the period of use. Table 5 relaxes each in turn for four forecasts.

Table 5 Extra squared error over the optimum of the period of use

Student t errors are scaled to the same covariance. In the third case the four forecasts have equal error variance in the period of use. In the fourth the best and worst forecasts trade places. Correlation stays at 0.6 throughout.

  1. Fat tails raise the cost of estimated weights (from about 8% to about 14% at T = 40) and leave inverse-MSE weights where they were.
  2. When the forecasts converge in quality the simple average is the optimum, and estimated weights carry the full cost of a pattern that is gone.
  3. When the ranking reverses, estimated weights are the worst choice at any sample size, and a longer sample does not help: the extra error is still about 68% at T = 120. A long sample measures the old Σ precisely.

Conclusions

  1. Estimation error alone is a weak explanation at moderate K. With four forecasts and ten years of quarterly errors it costs 8%, so under a stable Σ estimated weights win whenever the true gain is larger than that.
  2. It becomes a strong one as K grows against T. With eight forecasts and the same forty quarters the cost is 22%.
  3. The simple average is safest where the optimum is most tempting: large gains come from correlated errors and negative weights, which are the cases a shift in Σ turns into large losses.
  4. A half-and-half mix of estimated weights and the average is never the worst rule in Table 5, and at T = 40 it is the best one while Σ holds. It still loses to the simple average once Σ moves.

Limits


References

Personal page. Views are my own and do not represent my employer. Nothing here is investment advice.