Alpha Wounds: Bad Adjunct Methodologies

Active management has taken a lot of body blows recently. The principal criticism: Active managers contribute no alpha once their fees are factored in. So is it time to write active management's obituary? Not quite. But active management certainly is feeling pain.

A few weeks ago, I argued that many of the alpha wounds plaguing active management are self-inflicted. But this month, I will discuss injuries to active managers that are not of their own making. Specifically, I will talk about those wounds resulting from the bad evaluative methodologies employed by the investment industry’s adjuncts: consultants, academics, research institutions, and so forth.

Volatility Is Not Risk

I began my analysis career working as an intern at Portfolio Management Consultants (PMC) in Denver, Colorado. Back then, among other measures, we utilized alpha, beta, Sharpe ratios, and Treynor ratios to quantify whether an investment manager was better than her peers.

Over time we started to notice something very interesting for which we could not account: namely, investment managers who beat their benchmarks handily in 17 of 20 quarters and had barely trailed their benchmark in the remaining three quarters. On a graph, this outperformance was especially dramatic. However, we would then look back to our list of quantitative measures — alpha, beta, and Sharpe and Treynor ratios — and see that these managers ranked poorly compared to those who had barely beaten the same benchmark over the same time period and who had led during fewer quarters. How can this be? This result — all too common in the investment management industry — is due to one simple fact: Volatility is not risk.

Yes, I have heard the argument that for investors entering and exiting funds, volatility is risk. But why is this the concern of the investment manager? Absent a lock-up period for an investor's funds, how is the investment manager (active or passive) supposed to account for the anxiety of the end consumer? This is like holding physicians responsible for whether or not their patients smoke, drink, or eat hot dogs covered in trans fats. Or blaming psychologists for whether or not their patients continue to talk to their exes or take cruise ship vacations with the in-laws.

A Definition of Risk

The discussion of whether volatility differs from risk also depends on a critical assumption overlooked by most of the industry: Only in finance do we define risk as volatility. Different dictionary definitions of risk all converge on something like the "chance of loss." Here is one such example:

noun

1. exposure to the chance of injury or loss; a hazard or dangerous chance
2. Insurance
   1. the hazard or chance of loss.
   2. the degree of probability of such loss.
   3. the amount that the insurance company may lose.
   4. a person or thing with reference to the hazard involved in insuring him, her, or it.
   5. the type of loss, as life, fire, marine disaster, or earthquake, against which an insurance policy is drawn.

verb

1. to expose to the chance of injury or loss; hazard
2. to venture upon; take or run the chance of

idiom

1. at risk
   1. in a dangerous situation or status; in jeopardy
   2. under financial or legal obligation; held responsible
2. take/run a risk: to expose oneself to the chance of injury or loss; put oneself in danger; hazard; venture.

Notice that not a single definition includes volatility as a part of its explanation. Dictionary definitions and popular understandings of risk might differ from a business definition, yet a popular business dictionary describes over a dozen different forms of risk, ranging from exchange rate risk to unsystematic risk, all of which focus on the chance of loss. The insurance business is an industry critically dependent on an understanding of risk, and an insurance licensing tutorial says that "Risk means the same thing in insurance that it does in everyday language. Risk is the chance or uncertainty of loss."

Only in finance is risk defined as volatility. Why? In the early days of investment management analysis in the 1950s, academics recognized that average and standard deviation and the entirety of hundreds of years of statistics research thinking could be borrowed to analyze the performance of investment portfolios — if some of the definitions could be bent to their aims. Also, it was very difficult to run calculations of any sort in an automated fashion when calculators did not even exist and all computations were done by slide rule. So a simple measure was needed. Here, of course, I am talking about standard deviation. With standard deviation transformed into “risk,” the complex work of analyzing portfolios could begin and theories could be developed. Surely the math would take care of itself as the analysis was improved.

Take a look at the calculation of standard deviation and you will see that it is essentially the weighted average variation from a mean. There are two interesting things to note here:

• Variations are both above and below the mean, so besting your benchmark handily is also called “risk” in finance.
• Because larger variations from the mean are weighted more, any very large outperformance above your benchmark is even “riskier.”

These two facts illuminate how, back in the day at PMC, we found investment managers consistently outperforming (and when they underperformed, it wasn’t by much), but who would look bad from the point of view of alpha, beta, and Sharpe and Treynor ratios. So we developed investment management analytics at PMC that used a more accurate definition of “risk.” For example, we considered the return on US Treasury securities with the same maturity as the investment manager’s preferred investment time horizon to be the appropriate comparison for whether there was risk. Specifically, a manager’s return in a quarter was compared with the US Treasury. If the manager did not beat the Treasury, then it was characterized as a “loss” and became part of the time series used to calculate a downside-only standard deviation. We also adjusted the time series for each of our benchmarks as well for comparability.

Not surprisingly, this adjustment (among others omitted for the sake of brevity) began to highlight the performance of those truly outstanding managers rather than punishing them for terrific outperformance to the upside that the old measures said was “risk.” If you dig into the mathematics of alpha, beta, and the Treynor ratio, you find that they also punish managers who beat their benchmarks by too much. This is because the mathematics for all of these measures is sensitive to variability around a mean or trend line, rather than about only examining the downside.

Do Active Managers Beat Their Benchmarks?

But what does all of this matter? Don’t active managers underperform passive managers? It depends. Again, back in the day, I made it the focus of my masters program in business school to look at the performance of investment managers both using the traditional measures of “risk” and those metrics that I preferred. Guess what I found? If I used the traditional risk-adjusted return measures, I got the traditional result: Active managers underperform passive managers. Yet, when I used measures more like risk (the chance of loss), the average active manager outperformed the benchmark. In other words, the outcomes were reversed. This result held for multiple time frames. Subsequently, I updated this research while I was a money manager in 2003 (these results are proprietary to my old employer), and I found the exact same result as before. It would be lovely if someone would please update this research, don’t you think?

The Same Tired Paper

From where came the refrain about active managers not beating passive managers? If you trace many of the threads to their origin, you discover something fairly interesting: Most of these articles and stories all point to the same paper — and that paper was published a long time ago.

Here is one such thread: State Street’s Center for Applied Research and the Fletcher School at Tufts University recently published a white paper, entitled “By the Numbers: The Quest for Performance,” that reported that only 1% of active money managers deliver alpha after fees. What was their source for this oft-repeated claim? It was a 2012 article written by Charles D. Ellis, CFA, in the Financial Analysts Journal, entitled “Murder on the Orient Express: The Mystery of Underperformance.” Bad: This paper is three years old. Worse: The paper itself was not the original source of the data; instead, the data was from a paper written in 2010, entitled “False Discoveries in Mutual Fund Performance: Measuring Luck in Estimated Alphas,” that was published in the Journal of Finance. Worst: The return data used in this piece was only from up until 2006. The research also uses Sharpe ratios as the basis for its evaluation.

In other words, researchers and journalists are quoting nine-year-old data as the definitive coroner’s statement on whether active managers are better than passive managers. Furthermore, they are using measures of performance that punish upside outperformance in contradiction of most people’s definition of risk. Lastly, I know that passive strategies (most of the money in which tracks a well-published index) tend to perform better in up markets since these strategies encourage $billions to buy the same list of assets. I also know they tend to perform worse in down markets for the same reason. Would the results be replicated if the research used returns through the end of 2009? Hard to say, because such research is not often quoted if it exists.

Remedies

So what can be done about this alpha wound, the one that is inflicted by the use of bad methodology on the part of the investment industry’s adjuncts?

• Research into new measures of risk needs to be done. Sortino ratios are a start. But what are the full statistical ramifications of a semi-beta or alpha calculated on downside performance?
• Once these measures are developed, then performance should be reevaluated for both active and passive managers — and after fees, of course.
• Investment industry adjuncts need to stop quoting the same outdated research papers. It is the modern era and this data should be assessed in an automated fashion.

If you liked this post, don't forget to subscribe to the Enterprising Investor.

All posts are the opinion of the author. As such, they should not be construed as investment advice, nor do the opinions expressed necessarily reflect the views of CFA Institute or the author’s employer.

Image credit: iStockphoto.com/CSA-Archive