Mind the GAP: Questioning the Investment Manager’s Stated Benchmark (Digest Summary)

Most investors use a fund manager’s stated benchmark for fund analysis. But many managers’ benchmark choices are misleading. To identify both the weak and more optimal benchmarks, the author introduces the concept of “GAP” analysis to evaluate the appropriateness of a stated benchmark. An anecdotal example is followed by a detailed case study illustrating her methodology for determining the appropriateness of a stated benchmark.

Identifying the appropriate benchmark is necessary for the detection of alpha. But when an inappropriate benchmark is used, beta can be disguised as alpha. The most appropriate benchmark is one that captures the same systematic risk exposure and uses the most similar universe of securities as the fund manager’s investments. Although holdings- and factor-based analyses can be used to determine the appropriate benchmark, the author introduces a seemingly more direct returns-based method for analysis that is useful for any manager or investor selecting and monitoring fund investments.

Illustrating the need for benchmark skepticism, the author points out that within the particular demonstration group of funds claiming the multi-cap MSCI Emerging Markets Index as their benchmark, a good number suspiciously specify a distinctly narrow market-cap approach (e.g., large cap or small cap). The empirical data support the author’s suspicions: The performance for many of the funds is more highly correlated with another benchmark. This measurement of the “GAP” is introduced as the difference between a fund’s correlation (R²) with the stated benchmark and that with another benchmark. Although larger GAP values indicate that a more optimal benchmark exists, it is consistent differences across successive time periods that provide evidence that the stated benchmark is misleading. Results are presented graphically for the proportions of funds falling into GAPs over one-, three-, and five-year periods and successive one-year periods and those repeatedly falling into GAPs during those successive periods. Of the 114 funds with a five-year history, 25% had wide GAPs during at least two of the five-year periods.

The author demonstrates a returns-based methodology for determining the appropriateness of a benchmark by selecting 187 open-ended funds that had the MSCI Emerging Markets Index as their stated benchmark. The weekly returns over one-, three-, and five-year periods (as of the end of June 2013) for these actively managed funds with at least one year of historical data (Bloomberg) are regressed against the stated benchmark returns, as well as returns from four sets of other MSCI indexes with distinct geographic, capitalization, style, and sector characteristics.

The correlation coefficient, R², for the stated benchmark is tabulated for each time period, illustrating its overall appropriateness for the group. The author then introduces the GAP measure—that is, the difference between each R² when measured against the stated benchmark and the highest R² measured from the other indexes. Larger, or wider, GAP measures indicate a more optimal benchmark exists. The GAP is analyzed across successive time periods; a consistently wide GAP is viewed as evidence that the stated benchmark is misleading.

This is a short, well-written article that illustrates a useful and simple approach to judging stated benchmarks. But the author omits discussion on the limitations of the methodology. For example, consistent and wide GAPs are likely to be a result of an inappropriate benchmark, but further analysis would be required to be certain.