Macroeconomic forecasts drive many of the investment decisions made by industry professionals. It is important for analysts and portfolio managers to understand how these forecasts are created and what inputs are used. Most forecasts are based on econometric models, which are manually adjusted by experts. These adjustments are often unknown and undisclosed. Experts can add value through industry manual adjustments to underlying forecasting models.
How Is This Research Useful to Practitioners?
Analysts must be aware of and up to date with macroeconomic forecasts. Many do not know how forecasts are created, what assumptions are used, or what adjustments are made. The authors study a methodology that explains the effectiveness of adjustments made by experts to the original econometric forecasting models.
They find that experts’ inputs and adjustments to underlying econometric models are beneficial in one out of four quarters. Specifically, experts tend to add more value to forecasting models in times of economic recovery, as evidenced by Q3 2009, Q4 2009, Q2 2012, and Q3 2012. Moreover, experts’ forecasts are more accurate in years of economic recovery. During these periods, the experts provide more accurate forecasts than do the benchmarks.
This finding is important for analysts and portfolio managers to understand. Knowing the underlying metrics used in any forecasting model is vital in order to understand the model forecasts.
Many experts do not disclose the adjustments they make; the authors find that these adjustments can both benefit and hurt the overall model forecast. It is important for industry professionals to realize these facts and to do their own due diligence when dissecting and interpreting forecasted data.
How Did the Authors Conduct This Research?
Given that a final forecast is created from both the original model-based formula and the adjustments made by the expert, the authors lay out a methodology to discover the expert’s contribution to the overall forecast in terms of the adjustments. They present two benchmarks: a “no-change” forecast and a “best-model-based” forecast. The difference between these two benchmarks is the benefit added by the expert’s adjustments.
To determine the outcomes for the datasets, the authors use the total least-squares (TLS) technique and regression analysis. They focus on two cases in their analysis: annual International Monetary Fund (IMF) forecasts for US real GDP growth and quarterly IMF forecasts for Dutch GDP growth.
The authors develop a methodology to study and analyze the effectiveness of expert adjustments. Their study may serve as a steppingstone to further research in this area in order to better understand how experts’ adjustments to models benefit the overall forecast.
The main limitation is the study’s small sample size. Although the authors focus on a multi-year timeline, they gather data from only two countries—the United States and the Netherlands. If their datasets were expanded across different geographic regions, we might be able to better understand whether expert adjustments are more or less beneficial, depending on the area of the world where they occur.
Abstractor’s Viewpoint
The authors provide a foundation for a deeper dive into forecast modeling and the effect of experts’ adjustments on the overall forecast. As their results demonstrate, experts’ adjustments, though sometimes beneficial, are not always for the better. It is important for industry participants to keep that in mind when making investment decisions on the basis of economic forecasts and models.
It is also important for decision makers to understand the underlying models and the changes made by experts. The knowledge that experts add more value during times of economic recovery is helpful, but expert adjustments need to be more consistently accurate—an area in need of improvement.