Investors are wary of the robustness of the outperformance of smart beta strategies. The authors address this concern by providing measures of relative and absolute robustness. They examine the causes of a lack of robustness and propose remedies for these problems. Their conclusions focus on the dangers of data mining and a lack of transparency.
What’s Inside?
The authors seek to demonstrate failures in the robustness of smart beta indexes and to attribute these failures to a set of causes. They are critical of some specific examples of industry bad practice. They also propose solutions to address these problems and demonstrate how differently constructed indexes can perform better according to their definition of robustness. They address the problems of model selection, parameter estimation, weighting schemes, and factor dependencies.
How Is This Research Useful to Practitioners?
Investors in smart beta products need to be comfortable with the robustness of the performance of the products they buy. Smart beta is not protected by either the rigid simplicity of true passive investments or the fiduciary protection of active products. With the growth of this style of investing, the discussion around robustness is now more important than ever. The authors provide a summary of some of the key points of examination required for smart beta strategies and a framework for the quantitative evaluation of such strategies.
They outline sources of nonrobustness, including factor definitions, model specification, weighting schemes, and factor dependencies. To combat these, they propose avoiding uncompensated risks, diversifying across factors, and using simple tried-and-tested factors. The authors also emphasize the importance of transparency and of being consistent across factors and through time. These principles actually have relevance outside of smart beta; it is becoming less acceptable for any investor to be unfamiliar with factor-based investing (or at least factor-based performance attribution). The empirical evidence given relates only to equity indexes. Although less pressing because of the slower penetration of smart beta concepts into other markets, these principles are also applicable to other asset classes.
The authors’ work could be read as a literature review for robust methods, and the referenced works form a decent reading list for quantitative investors interested in building background knowledge of the quantitative space.
How Did the Authors Conduct This Research?
The authors illustrate each of their points with numerical examples. The time periods, frequency, and focus of the experiments change depending on the point to be made and the availability of data. It could be argued that this approach could indicate selection biases, but it produces simple illustrations rather than definitive empirical proof of ideas that are more conclusively supported in referenced materials. The authors compare smart beta methodologies almost exclusively using measures based on performance. They assess what they call “relative robustness” by comparing index performance with factor performance, which includes regressing against factors, comparing index performance in times of high/low factor performance, and using drawdown analysis. They also define absolute robustness, which they test using sophisticated extreme risk measures, such as semiparametric value-at-risk calculations.
The authors measure the robustness of single-factor smart beta indexes covering size, momentum, low volatility, and value as well as two multifactor indexes. They present charts of the excess returns of their multifactor indexes and analyze significant drawdowns. Because the authors are providers of smart beta solutions, their preference for their own approach is not surprising. As evidence of the benefits of multifactor construction, they compare it with the average of single-factor outperformance over a varying time horizon. Unfortunately, they neglect to show this point of comparison in their other exhibits and instead present comparisons with the standalone single-factor indexes.
Abstractor’s Viewpoint
The focus on return-based analysis misses a key point about robustness: The consistency of returns can be a target for data mining too. Only out-of-sample performance gives true evidence of robustness, and even then, fragilities may be hidden because of the rarity of tail events. It should be noted that relative robustness can often be assessed a priori from holdings or methodology alone. That said, scrutiny of the methods of smart beta indexes is much needed to illustrate critical points for all investors.