Building on his own paper written in 1982, the author discusses the usefulness of and limits to models in economic forecasting and risk management.
What’s Inside?
The conclusions about risk management and econometric modeling that the author reached in his 1982 paper hold true today. Models should be dynamic, rather than static, and cannot substitute for good judgment.
How Is This Research Useful to Practitioners?
Economic analysis runs along a continuum from development and verification of an accepted body of knowledge subject to further inquiry to an intuitive approach that is derived from experience and judgment in the review and interpretation of political and economic data. Econometric models capture changes after the fact, but to be useful, they need to do so beforehand. An ideal approach to forecasting and risk management would capture elements at both ends of the aforementioned scale.
Models demand a rigorous and structured thought process, which is their strength. Although models do a good job of capturing stable relationships, they prove their value when they encounter unstable ones because such situations give the experienced practitioner an opportunity to question previously held assumptions. In the author’s estimation, model results should be viewed as suggestive rather than inclusive.
The concluding remarks of the original paper are telling because they presaged the very problems to which practitioners were subject in the run-up to the Great Recession. No amount of technology can replace diligence and thoroughness in the interpretation of model results. Accordingly, econometric models cannot reliably foretell watershed moments in the economy because these moments are triggered by such structural changes as the oil embargoes of the 1970s, the US savings and loan crisis of the 1980s, and the subprime mortgage debacle of the late 2000s. Similarly, risk models are necessarily limited and are better suited to gauging shorter-term rather than longer-term market risk.
Chance would tend to favor the prepared mind. The author suggests overlaying quantitative skills with qualitative judgment. Additionally, risk officers should be mindful of early warning indicators of potentially larger issues, consider contingency arrangements in the event that such issues materialize, avail themselves of current and actionable data to respond to such events, and diversify business exposures beforehand.
Chief risk officers and policymakers would do well to heed the author’s conclusions, which have stood the test of time. An understanding of models’ limits as well as their capabilities is critical.
How Did the Author Conduct This Research?
Observations from psychology, econometrics, and other social sciences inform the author’s discussion and analysis. He considers risks in interpreting three aspects of statistical relationships.
First, drawing simplistic inferences by fitting so-called well-behaved theoretical distributions to a model’s parameters is dangerous in that it can mask possible outcomes of actual probability distributions where extreme events or black swans may reside.
Second, drawing conclusions from information not contained in the available data is folly. It is important to evaluate the quality of the data to understand what information actually exists in the data rather than what one wishes exists in the data. The author cites rating agencies’ overly optimistic appraisal of the risks embedded in tranches of subprime mortgage securities. Interpretations of historical data failed to account for the contemporaneous poor underwriting standards that engendered an explosion in these low-quality credits, which resulted in defaults on a grand scale that the models, based on outdated information, failed to recognize.
Finally, cyclical patterns experience reversals that can be as abrupt as the circumstances that preceded them. Continuing the previous example, loose underwriting standards brought about a precipitous increase in lower-quality mortgages that increased housing demand, which exceeded supply. Rising home prices erroneously validated aggressive lending in the form of lower loan-to-value ratios. When rising markets began to slow and experience price decreases, defaults followed. Subsequent lender foreclosures and sales of the once-expensive properties at distressed prices only reinforced downward pressure on home prices. The virtuous cycle quickly turned vicious.
Abstractor’s Viewpoint
The author’s discussion of the roles and limits of models is as timely today as it was in 1982. Econometric and risk models often exist in a symbiotic relationship that can prove fatal. Inaccurate forecasts may inform questionable decisions on risk management. Modeling has frequently attempted to fit a square peg into a round hole. Qualitative judgment must accompany quantitative analysis. Social science is not hard science; its problems do not always submit to rigid numerical interpretation.