Throughout the IFA Website, there are many assertions regarding the expected returns of different asset classes, and there are many warnings against attributing outperformance to active managers based on a few years of hot returns. One could easily make the mistake of thinking that IFA is merely expressing opinions without any mathematical support. In fact, however, nothing could be further from the truth. IFA has always relied on the scientific method of statistical analysis.
Perhaps the single most important application of statistics lies in the realm of hypothesis testing. Virtually all of the experimental and observational scientific studies (across all fields) take the approach of proposing a "null hypothesis" and then either rejecting or failing to reject the null hypothesis, based on the probability of making the recorded observations, assuming that the null hypothesis is actually true.
The specialty within economics known as "asset pricing theory" lends itself quite readily to hypothesis testing because for every asset (or asset class), there is usually a set of historical returns that can be tested. A common test would propose the null hypothesis that the expected return of the asset class is no different from the riskfree rate of return. Given a series of historical returns, we can calculate a parameter known as the "tstatistic" which will give us a quantitative indicator of the probability of observing these returns under the assumption of the null hypothesis. A higher value of the tstatistic indicates a lower probability of the occurrence of the observed returns, and vice versa. The tstatistic is commonly used to construct a 95% confidence interval around the observed mean, and if this confidence interval does not contain the value assumed in the null hypothesis, then we can reject the null hypothesis with a 95% level of confidence. Generally, a 95% confidence level is associated with a tstatistic of 2.
The value of the tstatistic depends on 3 separate parameters: The number of observations (N), the average of the observations, and the standard deviation of the observations. The tstatistic is directly proportional to both the square root of the number of observations and the average of the observations. It is inversely proportional to the standard deviation, so the more volatile the asset class, the less likely we will be able to draw firm conclusions.
To see exactly how the calculation of a tstatistic operates, we will use the returns for the last five calendar years of the top three mutual funds used in 401k plans (according to Brightscope.com )
The three funds to be evaluated are:
 AGTHX  American Funds Growth Fund of America (benchmarked against the S&P 500 Index)
 PTTAX  PIMCO Total Return Fund (benchmarked against Barclays Aggregate Bond Index)
 AEPGX  American Funds EuroPacific Growth Fund (benchmarked against the MSCI EAFE Index)
The results are summarized below: (1/1/2006 to 12/31/2010)

Fund 
Average Excess Return 
Standard Deviation 
TStatistic 
# of Years Needed to get a Tstat of 2 
AGTHX 
0.73% 
5.60% 
0.29 
235 
PTTAX 
1.82% 
3.38% 
1.20 
14 
AEPGX 
2.54% 
4.96% 
1.15 
15 
For all three of these funds, we are unable to reject the null hypothesis that their expected returns are no higher than their benchmarks at a 95% confidence level. 401k Plan sponsors who have incorporated these funds based on shortterm performance have no right to be surprised if the future returns captured by plan participants fall short of expectations. The chart below shows different sample sizes needed to get a Tstat of 2.
If you have your own set of returns data and you would like to see how many years you would need to get a tstatistic of 2, the calculator below can do that for you.
https://www.ifa.com/12steps/step3/step3page6.asp#tcalc1
Furthermore, if you wish to calculate the tstatistic for your returns data, plese use the calculator below.
https://www.ifa.com/12steps/step3/step3page6.asp#tcalc2
In the next example, we will use the tstat calculation to evaluate an asset class that has seen an enormous increase in popularity in the last few years, commodities. Using 19 years of calendar year returns data for the Dow Jones UBS Commodity Index, we will test the hypothesis that the expected return of commodities is no different than the riskfree rate (OneMonth TBills).
1/1/1992 to 12/31/2010 (19 years)


Average Excess Return 
Standard Deviation 
TStatistic 
# of Years Needed to get a Tstat of 2 
DJ UBS Commodity Index 
5.20% 
18.94% 
1.20 
53 
Again, we are unable to reject the null hypothesis at a 95% confidence level, so those investors who have poured their hardearned money into commodities (or commodity futures) might be setting themselves up for bitter disappointment.
Lastly, we will examine IFA's statistical justification for tilting a portfolio towards the factors of small cap and value. Based on 83 years of data, we will test the hypothesis that the expected return of the IFA US Small Cap Value Index is no different than the IFA US Large Company Index.
1/1/1992 to 12/31/2010 (19 years)


Average Excess Return 
Standard Deviation 
TStatistic 
IFA US Small Cap Value Index 
5.29% 
16.64% 
2.90 
Since the tstatistic is greater than two, we are able to reject the null hypothesis and conclude that small value has a higher expected return than the large blend segment of the market. In this case, however, we can offer an explanation that small value stocks are riskier and thus should carry a higher expected return. In general, no conclusion should ever be drawn from data alone, because as we all know, if the data is tortured for long enough, it will confess to anything. It is crucial to have a sound explanation for the observed data.
Although it is possible to find actively managed funds that have shown outperformance with a tstatistic greater than 2, IFA strongly cautions investors against throwing their money at these managers even when there appears to be a statistical justification for doing so. The reason, quite simply, is that since there are thousands of active managers, by chance alone, we expect to see some that have outperformed their benchmark after expenses. The problem is that the number that we actually do see is, in fact, no higher than what we would expect from chance alone (i.e., it is no higher than what we would observe if all active managers were monkeys throwing darts at the Wall Street Journal) . This means that when an active manager appears to exhibit outperformance, there is no reliable way to determine if it was due to luck (i.e., a false positive) or skill. Two papers that elegantly address this point are:
 "False Discoveries in Mutual Fund Performance: Measuring Luck in Estimating Alphas" by Barras, Scaillet, and Wermers which evaluated 2,076 fund managers over 32 years and found that 99.4% of active fund managers showed no genuine stockpicking ability.
 "Luck versus Skill in the Cross Section of Mutual Fund Alpha Estimates" by Fama and French which evaluated 819 actively managed funds over 22 years and found that 97% could not be expected to beat a riskappropriate benchmark.
IFA has always encouraged investors to obtain as much education as possible so that they can make informed decisions. Most investors will find that having a good understanding of statistics is incredibly helpful. Whenever they come across an advertisement such as "Fund XYZ beat its 5year Lipper average", they would do well to ask, "What is the tstatistic behind that number?" Odds are, it will not be included in the advertisement, and investors should not waste their time or their money on such spurious claims.
Index Fund Advisors, Inc. (IFA) is a feeonly advisory and wealth management firm that provides riskappropriate, returnsoptimized,
globallydiversified and taxmanaged investment strategies with a fiduciary standard of care.
Founded in 1999, IFA is a Registered Investment Adviser with the U.S. Securities and Exchange Commission that provides investment
advice to individuals, trusts, corporations, nonprofits, and public and private institutions. Based in Irvine, California, IFA manages
individual and institutional accounts, including IRA, 401(k), 403(b), profit sharing, pensions, endowments and all other investment accounts.
IFA also facilitates IRA rollovers from 401(k)s and 403(b)s.
To find out more about the value of IFA
click here.
To determine your risk capacity, take the Risk Capacity Survey.