Fingers Crossed Behind Back

Stock Pickers and the Illusion of Validity

Fingers Crossed Behind Back

To beat up an old cliché, if you ask 1,000 stock pickers for an explanation for why they chose to buy a particular stock, you are likely to get 1,000 different answers. They range the gamut from "Cramer likes it" to "it just hit a head-and-shoulders bottom" to an elaborate fundamental analysis considering dozens of variables and supported by hours and hours of research. We know that if we look at the records of stock pickers as a group, it is no better than if all of them had simply thrown darts at the Wall Street Journal, but with all these different approaches to picking stocks, one might expect that one group of stock pickers would consistently outrank their peers. Unfortunately, that is simply not the case, as numerous studies (e.g., the Standard and Poors Persistence Scorecard) have shown that stock pickers who rank in the top quartile of their peers in one year have no better than a 25% chance of being in the top quartile in the following year which is what we would expect from chance. One possible explanation of this result (in fact, the most likely explanation) is that markets are efficient, so the opinion of a single market participant is unlikely to prove superior to the judgment of the market as a whole. A somewhat amusing result occurs when these 1,000 stock pickers are asked to rank themselves relative to their peers; virtually all of them will claim to be above average. Although this Lake Wobegon effect is certainly not limited to stock pickers, the potential that it has to wreak havoc with their wealth (and the wealth of those who trust their investments with them) makes it worthy of further exploration.

Nobel laureate Daniel Kahneman, one of the founding fathers of behavioral economics, recently wrote an article, Don't Blink! The Hazards of Confidence, which helps us understand the prevalence of overconfidence. Kahneman recounts how during his time in the Israeli army he provided psychological assessments of the future potential of recruits based on how they conducted themselves on an obstacle course as part of a group. He would see which recruits assumed a leadership role and which ones proved to be a detriment to the group's completion of the task. Although he had every reason to think that his brief observations were strong indicators of a soldier's future prospects, he later discovered that there was zero correlation between his assessment and how the soldiers actually turned out. The underlying problem was the notion that several years of future accomplishments could be predicated on an hour of observation. As with stock pickers, his predictions were no better than random guesses, but even with this knowledge, it was difficult to stop conducting these assessments. Kahneman coined the term "the illusion of validity" to describe this cognitive fallacy. As Kahneman describes, "The confidence we had in judgments about individual soldiers was not affected by a statistical fact we knew to be true – that our predictions were unrelated to the truth. When a compelling impression of a particular event clashes with general knowledge, the impression commonly prevails."

Kahneman bore witness to the illusion of validity when he visited a Wall Street firm and was given the 8-year returns records for 25 of their "wealth advisors" (i.e., stock pickers). These records were used in the determination of the year-end bonuses received by these advisors, so there was a strong incentive to achieve the top ranking in the group. Unsurprisingly, Kahneman found that the rankings were no different than if they had been playing a game of pure chance like roulette. What happened next is best described by Kahneman, "What we told the directors of the firm was that, at least when it came to building portfolios, the firm was rewarding luck as if it were skill. This should have been shocking news to them, but it was not…I am quite sure that both our findings and their implications were quickly under the rug and that life in the firm went on just as before." When one of the advisors was presented with this finding, his defensive response was, "I have done very well for the firm, and no one can take that away from me." Kahneman managed to restrain himself from vocalizing what he was thinking, "Well, I took it away from you this morning. If your success was due mostly to chance how much credit are you entitled to take for it?"

It is clear how the illusion of validity applies to stock pickers. As Kahneman notes, "Nevertheless, the evidence from more than 50 years of research is conclusive: for a large majority of fund managers, the selection of stocks is more like rolling dice than playing poker. At least two out of every three mutual funds underperform the overall market in any given year." When confronted with this evidence, stock pickers will usually shrug their shoulders and continue in their misguided ways. The former Chairman of the Securities and Exchange Commission, Arthur Levitt, has some poignant commentary on Kahneman's article. You can listen to Levitt explain why investors are better off with index funds.


Listen to Bloomberg Surveillance. with TOM KEENE, KEN PREWITT