Morningstar, Forbes’ “Mutual Fund Honor Roll,” and mutual fund advertisements are some of the most well-known sources of investment information used by manager pickers. Examining these sources shows why superior performance cannot be correlated to past superior performance; because the future cannot be predicted.
The study also found
that at the end of 1990, after a long period of superior performance by
foreign-oriented mutual funds, only 32% (25 out of 77) of the total number
of highly rated stock funds were listed in the “international”
and “global” fund categories. Predictably, many investors
jumped into these funds, believing that their past superior performance
would be repeated in the future. Not surprisingly, every one of these
25 international and global funds subsequently underperformed the average
stock fund in the following 12 months. At the end of 1992, after foreign-oriented
mutual funds performed poorly for a year, no international or global funds
appeared on a highly rated funds’ list. Few investors were attracted
to these international and global funds because they were at the bottom
of the pile. The result: investors missed the superior performance of
international and global mutual funds that began at the end of 1992.
Morningstar, for its part, released a new star-rating system in July of 2002. The old system compared the historic risk and returns from a mutual fund with the risk and returns of a broad group of funds. The new star system attempts to compare a fund with a much smaller group of funds of a similar style. According to a March 2004 paper published in The Journal of Financial Planning by William Reichenstein, Ph.D., Morningstar’s decision to change rating systems reflects a decade of studies on the importance of investment style in explaining stock returns. As discussed several times in this book, a Fama and French study in 1992 concluded that stock returns vary systematically across two dimensions: size and value-growth. Actually, Fama and French came to this conclusion after completing two different studies. First, the researchers focused on returns from a period from 1963 to 1990. In a later study they conducted along with their colleague Jim Davis, the researchers looked at returns from a longer period—1929 to 1997. The researchers discovered that stock returns can best be explained when stocks are separated into portfolios based on size as measured by market capitalization and value-growth as measured by book/market ratios. Many studies over the past decade have confirmed and reinforced Fama and French’s conclusion that returns vary systematically across size and value-growth dimensions. This type of fund analysis was pioneered by Dimensional Fund Advisors (DFA), where Fama is the director of research. Fama also has written extensively on the random walk and efficient market theories and is one of the world’s most cited economists. DFA uses a Fama/French designed factor regression analysis to show that active managers’ returns are attributable not to skill, but to exposure to these risk factors. Consistent with the Fama and French research, DFA offers no actively managed funds, but has a complete assortment of passively managed index funds.
There is one other
point regarding the futility of attempting to identify skillful money
managers. An old investment proverb observes that “markets make
managers.” This means that if the market favors a money manager’s
particular investment style anyone can achieve outstanding performance.
Markets can make a money manager look good or bad — a factor that’s independent of their “skillful” stock picking or market-timing abilities. An active money manager that an investor selects will usually turn out to be a winning or a losing manager because of the behavior of the market itself, rather than the manager’s skill at picking stocks or timing markets. Active money managers play a game that’s almost entirely random in conferring long-term investment success among them.
There are at least
three other problems associated with manager picking. For one thing, investors
are seldom aware that active funds or separate portfolios that have good
performance histories are always riskier than the indexes they outperform.
According to Modern Portfolio Theory, any portfolio of investments
that hold fewer stocks than the index in which it is invested must be,
by definition, underdiversified relative to that index portfolio. It follows
then that any mutual fund or separate portfolio that has turned in a market-beating
performance achieved it by holding investments that somehow were different
in kind or amount from those of the relevant index. Any mutual fund or
separate portfolio that boasts a superior performance history must therefore
A mutual fund manager with recent performance success has bet money and concentrated it in specific stocks or bonds. The bet may pay off, but people are too blinded by the “brilliant investment insight” to understand that the bet was too risky in the first place. Peter Lynch, the legendary manager of Fidelity’s Magellan mutual fund, concentrated about 25% of the fund’s holdings in foreign stocks in the 1980s. These stocks turned out to be top performers, and Magellan widely outpaced the S&P 500. The irony is that these stocks weren’t even represented in the S&P 500.
Lynch’s performance was not measured against an appropriate benchmark comprised of a proportionately weighted mix of U.S. and foreign stocks. It was measured against the wrong benchmark, the S&P 500. Using an appropriate benchmark would have reduced, perhaps even eliminated, his successful performance during this period. Lynch’s bet was nevertheless deemed a winner by popular acclaim, and he was widely hailed as the leading investment guru of the decade.
Had Lynch’s bet turned out wrong and Magellan underperformed, Lynch would have been widely criticized as a fool for making such a risky bet. Right or wrong, it was still a risky bet because Magellan had a greater amount of diversifiable risk than was represented in the benchmark by which it was measured.
There are two lessons to be learned from this. First, any active investment strategy is inherently risky, but is not considered risky in hindsight if it turns out to be a winner. Second, a mutual fund’s outstanding performance history is nothing more than the market’s reward for exposure to excessive investment risk. Due to the unpredictable nature of the market, the same excessive risk that produces outstanding performance today can turn and produce miserable performance in the future. Once the market begins to favor sectors other than those a manager is invested in, his or her luck has run out.
Yet another problem with manager picking is that outstanding performance histories can be surprisingly fragile. Few investors realize that the most important factor separating a winning performance history from a losing one is the choice of starting and ending dates. Fidelity’s Magellan beat the S&P 500 for the decade ending in mid-1995. Lengthening the ending date by one year to mid-1996 would have painted a very different picture. Fidelity’s Magellan underperformed the S&P 500 for that 11-year period.
Lastly, outstanding performance histories don’t always reflect taxes or commission loads. Published mutual fund ratings are often pre-tax returns that disguise their true after-tax performance in taxable accounts. Fidelity’s Magellan generated an average annual pre-tax return of 18.3% over the 10-year period from mid-1985 to mid-1995. Once the taxes and commission loads were factored in, the net return dropped to 12.7%. At first glance, this fund appeared to widely outperform the market. A closer look reveals that Fidelity’s Magellan came very close to underperforming it. However, an investor may never know this because mutual fund advertisements often feature only pre-tax and/or pre-commission load returns. Tax-adjusted returns are now available from Morningstar on the Internet at www.morningstar.com. Morningstar’s tax-adjusted returns only account for federal income taxes, but not state income taxes. Investors should also consider that state income taxes need to be deducted in order to see a complete picture of how all taxes impact investment performance, especially relative to a tax-efficient index fund.
The inception date for the Magellan fund was May 1, 1963. For a 47 year, 11 month comparison of the Fidelity Magellan Fund to the IFA Indexes and Index Portfolios, see here.
Indexes such as the S&P 500 or Wilshire 5000 are often used to evaluate
the performances of active money managers. Given the Fama and French findings,
the use of such benchmarks is often misleading. Because these indexes
are weighted heavily towards large company stocks and high priced stocks,
the performances of managers investing more heavily in small company stocks
or low priced stocks won’t be accurately measured by them. Instead,
customized benchmarks are needed to provide accurate measurements of the
contributions to performances made by active money managers.
The Fama and French Three Factor Model is a superior way to evaluate the performances of active money managers. It shows whether a manager achieves returns in excess of index returns. After all, an active manager shouldn't be rewarded just for buying value stocks—that’s something that can be done inexpensively with an indexing strategy.
The place where a portfolio is positioned or structured on the cross hair map in Figure below determines the vast majority of its return. The cross hair map doesn't plot the market risk factor since all stock portfolios take similar market risk and are plotted relative to the stock market. So, there’s no need for a separate axis; instead, the stock market sits right at the cross hairs of the map. The cross hair map has two dimensions. The size dimension is plotted along the vertical axis, and the value (BtM) dimension is plotted along the horizontal axis. The axes represent exposure to these two risk factors. Portfolios that take on a lot of size risk appear higher along the size axis, and portfolios that take on a lot of value risk appear further along to the right on the growth/value axis.
Throughout the IFA Website, there are many assertions regarding the expected returns of different asset classes, and there are many warnings against attributing outperformance to active managers based on a few years of hot returns. One could easily make the mistake of thinking that IFA is merely expressing opinions without any mathematical support. In fact, however, nothing could be further from the truth. IFA has always relied on the scientific method of statistical analysis.
Perhaps the single most important application of statistics lies in the realm of hypothesis testing. Virtually all of the experimental and observational scientific studies (across all fields) take the approach of proposing a “null hypothesis” and then either rejecting or failing to reject the null hypothesis, based on the probability of making the recorded observations, assuming that the null hypothesis is actually true.
The specialty within economics known as “asset pricing theory” lends itself quite readily to hypothesis testing because for every asset (or asset class), there is usually a set of historical returns that can be tested. A common test would propose the null hypothesis that the expected return of the asset class is no different from the risk-free rate of return. Given a series of historical returns, we can calculate a parameter known as the “t-statistic” which will give us a quantitative indicator of the probability of observing these returns under the assumption of the null hypothesis. A higher value of the t-statistic indicates a lower probability of the occurrence of the observed returns, and vice versa. The t-statistic is commonly used to construct a 95% confidence interval around the observed mean, and if this confidence interval does not contain the value assumed in the null hypothesis, then we can reject the null hypothesis with a 95% level of confidence. Generally, a 95% confidence level is associated with a t-statistic of 2.
The value of the t-statistic depends on 3 separate parameters: The number of observations (N), the average of the observations, and the standard deviation of the observations. The t-statistic is directly proportional to both the square root of the number of observations and the average of the observations. It is inversely proportional to the standard deviation, so the more volatile the asset class, the less likely we will be able to draw firm conclusions.
To see exactly how the calculation of a t-statistic operates, we will use the returns for the last five calendar years of the top three mutual funds used in 401k plans (according to Brightscope.com )
The three funds to be evaluated are:
The results are summarized below: (1/1/2006 to 12/31/2010)
|T-Statistic||# of Years Needed to get a T-stat of 2|
For all three of these funds, we are unable to reject the null hypothesis that their expected returns are no higher than their benchmarks at a 95% confidence level. 401k Plan sponsors who have incorporated these funds based on short-term performance have no right to be surprised if the future returnscaptured by plan participants fall short of expectations.
The interactive chart below has the formula for a t-stat of 2, based on return and risk values built into the chart. As you roll your mouse to a coordinate of return and risk, the line that highlights represents the number of samples needed to obtain a t-stat of 2. A data box representing each point on the line provides the 3 values for that position.
In the next example, we will use the t-stat calculation to evaluate an asset class that has seen an enormous increase in popularity in the last few years, commodities. Using 19 years of calendar year returns data for the Dow Jones UBS Commodity Index, we will test the hypothesis that the expected return of commodities is no different than the risk-free rate (One-Month T-Bills).
1/1/1992 to 12/31/2010 (19 years)
|Average Excess Return||Standard Deviation||T-Statistic||# of Years Needed to get a T-stat of 2|
|DJ UBS Commodity Index||5.20%||18.94%||1.20||53|
Again, we are unable to reject the null hypothesis at a 95% confidence level, so those investors who have poured their hard-earned money into commodities (or commodity futures) might be setting themselves up for bitter disappointment.
Lastly, we will examine IFA’s statistical justification for tilting a portfolio towards the factors of small cap and value. Based on 83 years of data, we will test the hypothesis that the expected return of the IFA US Small Cap Value Index is no different than the IFA US Large Company Index.
1/1/1928 to 12/31/2010 (83 years)
|Average Excess Return||Standard Deviation||T-Statistic|
|IFA US Small Cap Value Index||5.29%||16.64%||2.90|
Since the t-statistic is greater than two, we are able to reject the null hypothesis and conclude that small value has a higher expected return than the large blend segment of the market. In this case, however, we can offer an explanation that small value stocks are riskier and thus should carry a higher expected return. In general, no conclusion should ever be drawn from data alone, because as we all know, if the data is tortured for long enough, it will confess to anything. It is crucial to have a sound explanation for the observed data.
Although it is possible to find actively managed funds that have shown outperformance with a t-statistic greater than 2, IFA strongly cautions investors against throwing their money at these managers even when there appears to be a statistical justification for doing so. The reason, quite simply, is that since there are thousands of active managers, by chance alone, we expect to see some that have outperformed their benchmark after expenses. The problem is that the number that we actually do see is, in fact, no higher than what we would expect from chance alone (i.e., it is no higher than what we would observe if all active managers were monkeys throwing darts at the Wall Street Journal) . This means that when an active manager appears to exhibit outperformance, there is no reliable way to determine if it was due to luck (i.e., a false positive) or skill. Two papers that elegantly address this point are:
IFA has always encouraged investors to obtain as much education as possible so that they can make informed decisions. Most investors will find that having a good understanding of statistics is incredibly helpful. Whenever they come across an advertisement such as “Fund XYZ beat its 5-year Lipper average”, they would do well to ask, “What is the t-statistic behind that number?” Odds are, it will not be included in the advertisement, and investors should not waste their time or their money on such spurious claims.
The solution for manager
pickers is to stop being fooled by randomness, stop believing in Santa
Claus, and give up the hope that a fund manager can be selected in advance
to consistently beat a market in the future.
stated their case saying they need at least 20 years worth of risk and
return data to establish skill in a manager. The real problem is choosing
those managers at the beginning of the period. Therefore, index funds
are a far better choice for investors because of their 80-year
1. Statisticians tell you that you need a minimum number of years
of performance data on mutual funds to draw conclusions about future risks
and returns. How many years are required?
a. 1 year
b. 5 years
c. 10 years
d. 20 years
2. The problem with picking a manager to beat the appropriate
index is that:
a. they can’t pick next year’s winning stocks
b. they can’t pick the best time to be in or out of the market
c. they can’t determine which style of investing is the best
d. there is no persistence in manager performance
e. all of the above
3. A Dalbar study found manager pickers changed their managers
a. 7 months
b. 5.3 years
c. 2.6 years
d. 15.5 years
4. There are overlooked factors when investors review the past
performance of managers. They include:
a. improper benchmarks
b. after-tax returns in taxable accounts
c. exact same time periods
d. commission charged on the purchase of the fund
e. all of the above
5. According to the mutual fund tracking service, Lipper, the
top 50 hottest selling mutual funds in March 2000 were reviewed again
in March 2005. On average, the top 50 funds had a total change in value
a. up 83%
b. down 42%
c. up 5%
d. down 10%
e. up 22%
Index Funds Advisors, Inc. — 19200 Von Karman Ave., Suite 150 — Irvine, CA 92612
Call Toll Free: 888-643-3133 — Local Phone: 949-502-0050 — Fax: 949-502-0048 — Email:
For several other offices and representative locations, see About Us.