A seemingly endless battle is waged between believers in the efficient market hypothesis, such as Eugene Fama, and believers in behavioral finance, such as Daniel Kahneman. Regardless of your perspective, an analysis of the S&P 500’s history of sigma events provides an interesting field for the battle to be waged.
For example, from 3 January 1950 through 31 July 2012, the average daily return of the S&P 500 was 0.03%, and the standard deviation was 0.98% (source: Yahoo Finance, CFA Institute). These results are remarkably similar to the mean and standard deviation of the normal distribution of 0 and 1, respectively. This suggests that daily returns for the S&P 500 closely approximate the normal distribution, and that returns follow a random walk.
What if you feel that mean and standard deviation are not the only way to describe a probability distribution? Then these data do not tell the entire story. Many researchers have noted anomalies in return data such as extreme positive and negative daily returns — the proverbial “fat” tails that characterize stock market returns.
Here is a look at the distribution of the S&P 500’s daily returns categorized by how extremely those returns deviated from the average daily return of 0.03%.
Number of S&P 500 Sigma Events (3 January 1950 – 31 July 2012)
Source: CFA Institute.
As you can see, the overwhelming majority of daily returns fall within one standard deviation, or sigma, from the mean return of 0.03% per day. This is actually a characteristic not discussed as frequently as the stock market’s “fat tails.” Namely, that daily returns are leptokurtic until you reach the tails. Yet, the normal distribution holds that ~68% of returns should occur within one standard deviation of the mean, yet the actual number is a gigantic 95.6%.
Here is the numerical breakdown of the graph above:
Source: CFA Institute.
Market observers have noted that financial markets have become more volatile over time. A look at the number of sigma events by decade makes that clear.
Source: CFA Institute.
For example, the number of normal trading days — as measured by the percentage of trading days that are a 1 sigma event — is sharply lower since the 2000s, with the decade of the 2000s having 1 sigma trading days only 89.54% of time, as compared to the average of 95.56% and the peak of 98.61% in the 1950s. That said, the most "normal" decade, as measured by the decade with the smallest deviations from the average, was a recent decade, the 1990s.
There has also been a doubling of two sigma events, a tripling of three sigma events, and so forth. However, careful scrutiny reveals something extraordinarily interesting: Just two years of daily market activity, 1987 and 2008, account for 56% of all five sigma and above events! In 1987 there were six events that were five sigma and above, and in 2008 there were 18 such occurrences. Wow! These numbers compare to the average number of five sigma and above events per year of 0.68. So, in addition to there being daily return sigma events to be cautious of, there are clearly high sigma years to be wary of as an investor, too.
What about the expected daily occurrence of sigma events? Here is the historical record:
Source: CFA Institute.
What the above chart shows is that there are, on average, 129.2 trading days per 251.62 trading days in a year in which your return is between 0.03% and 1.02%. Similarly, there are, on average, 3.85 days per year where your loss is between −0.98% and −1.99%, or between a one sigma and two sigma loss.
After the two sigma events, it becomes harder to tell what the expected frequency of a sigma event is, so here are the data rescaled by years, instead of days:
Source: CFA Institute.
Here you can see that a seven sigma up day can be expected once every 31.29 years, and a 10 sigma or greater down day can be expected once every 62.58 years.
One famous piece of oft repeated wisdom doled out by the buy-and-hold community is that missing just the top 10 up days results in a significantly lower total return. Consequently you should always stay invested, lest you miss these days. Indeed, when framed this way there is truth to the statement. One dollar invested on 3 January 1950 would have turned into $81.79 on 31 July 2012. Yet, if you had missed those top 10 performing days, you would only have $38.95 instead of $81.79.
But this is only half the story. For what if you were in fact a brilliant market timer, and you were able to miss just the 10 worst-performing days in market history? Your $81.79 would actually be a whopping $214.41. This result is clearly an example of brilliant market timing as investors would have experienced each of the top 10 performing days, yet missed all of the 10 worst trading days.
So what is the result of missing the top 10 and bottom 10 trading days? Investors’ $1 would have grown to $102.94. Because this result is much higher than the $81.79 earned with the buy-and-hold strategy, it does not make sense to justify a buy-and-hold strategy just on the premise that you make more money from employing it instead of market timing.
And just for giggles, what if you had perfect market timing and were only invested on up days? Your $1 investment would have grown to be:
$335,288,501,296,558,000,000,000.
For those of you not up on your large numbers, that is $335 sextillion (or a trillion trillion).
Last, the largest positive sigma event of all time occurred on 13 October 2008, when the S&P 500 surged upward registering as an 11.82 sigma event. Meanwhile, the largest negative sigma event was the famous 19 October 1987 crash, which was a whopping 20.98 sigma event!
If you liked this post, don’t forget to subscribe to the Enterprising Investor.
All posts are the opinion of the author. As such, they should not be construed as investment advice, nor do the opinions expressed necessarily reflect the views of CFA Institute or the author’s employer.
Image credit: ©Getty Images/Bloomberg Creative Photos
35 Comments
Hi Eden,
I have not performed the study for monthly returns, but may do so in the future.
Thank you for the feedback!
Jason
If i understand the concept correctly, I don't think it is about 1M or 6M return that matter. The point is returns tend to converge when you deal with higher and higher number of samples. Even with 6M return, the tendency of normal curve would appear. And that has to do with basic central tendency. The simplicity of 1D return is you get Mean of 0 and SD of 1 which is straight out of the statistics textbook. If you'r running 6M and comparing with 1D take care to adjust the number of sample-n.
Jason - do you reason similarly, in your experience with other industries.
Thanks in advance.
Regards
M Ashok, CFA
Hello Ashok,
In general, I feel that you are correct. I would add that just because the mean and standard deviation approximate normal does not necessarily mean that the entire curve approximates normal. After all, essentially mean and standard deviation are two point estimates of a two-dimensional curve. I can think of many geometrical shapes that would have similar points, but radically different shapes. In fact, that is what we have with the S&P 500's returns. I feel the most surprising thing of the above data is the extreme leptokurtic quality of the returns. Typically we only hear about the "fat tails" of market returns, but what about the giraffe head? This is strongly non-normal.
I would also point out that I feel differing the time scale will change the results, just as changing your sieve changes the texture of your refined spice. Yet, the spice will taste the same even if it does not look the same.
Ashok, you point out the importance of adjusting for dispersion (n) is making comparisons. Yet, this adjustment for Brownian motion is essentially a normal distribution/stochastic/random transform. But applying this transform to a curve that is certainly non-normal is probably tough to justify. Take a look at this other The Enterprising Investor piece about this topic: http://blogs.stage.cfainstitute.org/investor/2012/05/29/holes-in-some-o…
As always, thanks for your invaluable contribution!
Jason
Agreed, on the sampling part. Yes the leptokurtic distribution (over the long term) did take me by surprise. The fat tails are compensating as well -in shorter time frames. I mean a negative fat tail events is most often followed by positive fat tail events. The interval could vary, but its not very long. My point is one always hears about tail-risk, but they don't obviously discuss the upside-tail-risk, do they?
Nice piece of research here, and almost more so the generated discussion within the commentary.
If I may weigh in a few of the raised issues.
The term of your returns will most certainly have a large impact on the obtained return distribution. And, unfortunately, this need not approach normality. An oft quoted but somewhat misinterpreted stylized market fact a al Cont (2001) is that of aggregational guassianity, with the rule of thumb being that returns can be considered close enough to normality from 1M onwards (Bingham & Kiesel (2004)). This is definitely not true in certain markets, under both rolling and resampled x-month returns. In your case, your dataset is large enough to choose independent periods, thus alleviating any autocorrelation issues.
In addition, depending on the period chosen, one will find quite severely differing results, even if the chosen periods are somewhat overlapping.
That said, what one can posit is that returns are ergodic. However, ergodicity is very difficult to
In terms of non-linear risk measures, I would suggest coupling this type of analysis with your simple VaR and CVaR measures. The nice point here is that due to the size of your dataset, you can quite easily use a kernel density estimator to find the specified percentile and mean below that with without having to worry too much about estimation/sample size effects here.
I would also suggest considering Omega. If you are not aware, there is a great picture of two extremely different distributions superimposed with the statement: 'these distributions have the same mean and variance'. This would capture the potential differences between negative and positive 'fat tail' events, and would also allow you to quantify with a bit more rigour what 'fat tail' really means and the extent of its effects.
Finally, in terms of the statement that a negative fat tailed event is most often followed by a positive fat tailed event - I am not so sure. One is uniquely aware at a base level of the gain/loss asymmetry within returns which immediately points to there being more negative extreme events than corresponding positives. However, in order to properly analyse this type of statement, one should really make use of survival (reliability) analysis techniques. While typical survival analysis models the time until 'death' of a population for example, one can quite easily define survival as being within certain sigma bounds and 'death' being an extreme value. Thus one can accurately capture the dynamics of the recurrence times between extreme negative events or between extreme positive events and more importantly, the recurrence time between moving from a neg (pos) extreme to a pos (neg) extreme. In essence, one would focus on the probability of moving from one extreme event to the next extreme (fore example, down-to-up), conditional upon past survival (no extremes). The hazard function considers exactly this.
I am always surprised by how under-utilised this type of analysis is in financial research.
Yours in research,
Emlyn
Apologies on the unfinished lines in my previous comment. The correction lines below:
"That said, what one can posit is that returns are ergodic. However, ergodicity is very difficult to prove for dynamic systems, of which the financial world is most certainly one. Another confounding factor is that ergodicity is most usually associated with systems in statistical mechanics, where one's scale of observations is close to Avogadro's constant (6.023 x 10^23!), rather than only 13000 returns.
Hi Emlyn,
Thank you very much for taking the time to share your thoughts about the above data. We all have our favorite aspects of interesting data results. My favorite from the work I did above was suggested by my colleague, Ron Rimkus. He suggested that by creating a ratio of the value of one dollar invested in a buy-and-hold strategy, divided by the result of perfect market timing you could get a sense of the market's ability to predict the future. Essentially, and obviously that ratio is zero.
I did not report it in the above piece, but the result of absolutely perfect bad market timing (only buying on the down days) results in your one dollar turning into $0.02 x 10^-23!
Emlyn and Ashok, you both may be interested in my most recent "Fact File" piece published today on The Enterprising Investor: http://blogs.stage.cfainstitute.org/investor/2012/09/10/fact-file-the-s…
With smiles!
Jason
Interesting points Jason. I was convinced about the futility of market timing strategy after looking at the average of daily overlapping returns, which is very close to zero. If you analyse buy-and-hold strategies (by looking at increasing duration of buy and hold) you will be convinced that higher-duration returns statistically improve with time. Chance-of-loss or a VAR improve with increasing duration.
Wow, this discussion has taken another level with Emlyn's comments. I am a nobody in advanced Stats but this opens up thinking.
As for your comments on utilizing basic and adv statistics in financial research, I am gonna agree with you. The closer you reach towards self-actualization mode, the more disconnected you feel with the rest!
Thanks much for your inputs.
Regards
Ashok, CFA
Dear Jason: I wrote an article for my students based on your "Fact File:
S&P 500 Sigma Event" which includes data on monthly and annual returns.
I can send you a copy if you wish. Dr. Floyd Vest, Retired Professor of
Mathematics and Education, Mathematics Department, University of North
Texas, 940-387-2137, 1103 Brightwood, Denton, TX 76209, [email protected]