Market participants and regulators care about the magnitude, as well as the number, of extreme losses. But current backtesting methodologies focus only on the number of value at risk exceptions and disregard the magnitude. The Risk Map, a new tool to validate risk models, jointly accounts for the number and the magnitude of extreme losses.
What’s Inside?
Current backtesting methodologies focus only on the number of value at risk (VaR) exceptions and disregard the magnitude of these exceptions. The authors present a new tool, called the “Risk Map,” for validating risk models. The approach accounts for both the frequency and the magnitude of extreme losses. The Risk Map can be used to validate market, credit, operational, or systemic risk estimates or to assess the performance of the margin system of a clearing house.
How Is This Research Useful to Practitioners?
The authors propose a validation framework, the Risk Map, that allows risk managers and regulators to assess the validity of a risk model by accounting for both the number and the magnitude of extreme losses. The Risk Map relies on the concept of a super exception, in which the loss exceeds both the standard VaR (exceptions) and VaR defined at a much lower probability (super exceptions). The authors design a testing procedure that combines information about both.
The main advantages of the Risk Map are that it is as simple to use as standard validation techniques, it is a formal hypothesis testing framework, and it can be applied to any tail-risk model.
The Risk Map approach may prove particularly effective in banking. It could help banking regulators detect mis-specified risk models and penalize banks that experience VaR exceptions that are too large as well as those that are too frequent.
The authors present a variety of applications for the Risk Map, including market risk modeling, systemic risk measurement, and margins for derivatives uses. Particularly, the authors believe the Risk Map is the first method that allows one to backtest a systemic risk measure.
How Did the Authors Conduct This Research?
Typically, the return or profit and loss of a portfolio on a particular day would be compared with the ex ante VaR forecasted one day ahead. If the VaR model is adequate, the probability of the former (the loss) exceeding the latter (forecasted VaR) should be equal to the alpha coverage rate. For example, for 500 daily observations with alpha equal to 1%, if the VaR model is adequate, there should be five days that the ex post return exceeds the VaR forecast.
A key limitation of this approach is that it is unable to distinguish between a situation in which losses are below but close to the VaR and a situation in which losses are considerably below the VaR.
To address this limitation, the authors propose a model validation methodology that is based on the number and the severity of VaR exceptions. The approach exploits the concept of a super exception, which they define as a loss greater than VaR(α′), with α′ being substantially smaller than α (e.g., α = 1% and α′ = 0.2%). So, for 500 daily observations with α = 1% and α′ = 0.2%, if the VaR model is adequate, there should be five days that the ex post return exceeds the VaR(α) forecast and one day that the ex post return exceeds the VaR(α′) forecast.
The authors propose an approach to jointly test the number of VaR exceptions and super exceptions using a simple likelihood ratio test, which tests whether the empirical exception frequencies significantly deviate from the theoretical ones. The test is different from the standard validation test, which would reject a 10 exceptions model in a 500 day sample at the 95% confidence level; the new approach would accept it as long as the super exceptions are within 1 to 3. The authors’ approach is to test whether both exceptions and super exceptions are at an acceptable level. The rejection zones for different confidence levels can be used to construct a graphical presentation, which then creates the Risk Map.
A natural application of the Risk Map is to backtest the VaR of a financial firm. As an example, the authors use the actual VaR and profit and loss for a large Spanish bank. They use daily one-day ahead VaR(1%) and daily profit and loss for that bank during 2007–2008. Over the period, there were 13 VaR exceptions and 3 super exceptions. Based on the Risk Map, the model is rejected at the 95% confidence level but not at the 99% level. Finally, the authors conduct a Monte Carlo experiment on the Risk Map, and the result is generally satisfactory.
Abstractor’s Viewpoint
Both the frequency and magnitude of tail risk are the concerns of risk managers. The Risk Map could help risk managers and regulators to jointly backtest the VaR model at two confidence levels. Particularly, the Risk Map may be the first tool to backtest systemic risk measures. It validates the marginal contribution of systemically important financial institutions to the overall risk of the system, which could be very helpful in preventing a future financial crisis.