Often you will hear the voices on TV telling you they like an out-of-favor stock or sector but that they don’t see the catalyst that will turn the sentimental tide. They advise waiting until the market stabilizes, until others start buying, before you buy. “Don’t worry about missing the first 10 percent of the move because otherwise you are trying to catch a falling knife,” they assure you.

This line of reasoning has its benefits, but it has some very severe flaws as well. Chief among them is the notion that the market will go up calmly, gradually, rationally, leaving you plenty of time to get back in once it’s become abundantly clear that the water is once again safe and inviting. But ask yourself whether that makes sense. If it’s so clear to you that everything’s OK—and you are putting yourself in the position of trying to act in line with the crowd—why shouldn’t the same clarity or safety be evident to the person who might sell to you? Why should they, having taken the risk of buying or at least not selling 10 percent lower when you were too afraid to jump in, gladly part with merchandise that suddenly everyone wants to own?

The fact is, they won’t. They are greedy too, though perhaps not as greedy as they should be. Research has shown that market timing—the strategy of getting repeatedly in and out of the market—is an even trickier game than it seems. That’s because market performance doesn’t tend to come in the smooth, steady, a-little-bit-every-day fashion people would like. Instead, studies have shown that a very small number of trading days tend to account for the bulk of the market’s returns. So if you miss out on a couple of those days, you’ll miss out on most of the good things that happen in any given period.

What’s worse, it’s psychologically difficult and extremely painful to jump back in after you’ve bailed out, told yourself you’ll wait until the coast is clear, and then watched things turn violently higher. You may well stay out for a much longer period and end up damaging your long-term investment objectives.

But wait a minute, you say. This all applies to mutual funds—those amorphous blobs of retail investor money lumped together. But what about hedge funds? Aren’t those guys the real smart ones? Aren’t they the ones really digging beneath the numbers, aren’t they the ones who are really plugged in? If something’s so cheap, won’t they buy it? And if something’s so expensive, won’t they sell it short? Aren’t they the investors who are still keeping the markets efficient?

Well, you would think so. Hedge funds definitely have more latitude to do original thinking and investing than do most mutual funds. At the same time, for all the talk about the growth and power of hedge funds, they still represent only a small fraction of all the money that’s out there. It’s true that they tend to trade more actively than mutual funds, and that they have an impact on prices disproportionate to their size, but they are still simply not big enough to drive prices to the “correct” levels called for by the collection of ideas generally known as the efficient market theory (EMT).

Also, you must not forget that hedge funds get most of their money from our herd-following friends, the asset allocators, who don’t have much patience for a strategy that underperforms in the short term. So if its performance begins to suffer, even a hedge fund with an outstanding long-term record is usually on a very short leash. Thus, hedge funds that are inclined to be contrarians—inclined to buy when they sense that selling has taken on a life of its own and become derailed from reality—know that they had better be right quickly. But the real world is messy and often doesn’t cooperate with that timetable.

When I was at my second hedge fund, back in 2005, the world of convertible bonds was imploding. The years 2000-2002 had been the golden age for convertible arbitrage, buying convertibles and selling short the underlying stocks. Expensive stocks lost most of their value, but the companies stayed in business and paid off their debts and, thus, convertible hedgers made a lot more on their short-stock positions than they lost on the bonds they owned. But in the next several years those ideal results stopped playing out, and returns on the strategy became very ordinary, though not disastrous by any means. That is, not disastrous until 2005. The watchword of asset allocators became, “Sell convertible bonds! Get out—get out now!”

As a hedge fund manager, I would frequently meet with these asset allocators. Often they would bring in a checklist of questions, wanting to know the theoretical exposures of my portfolio and the likely outcomes in a variety of scenarios. In one conversation, my partner and I explained that we would typically buy a certain amount of “disaster protection” through put options on the QQQQ, an index of over-the-counter, largely technology stocks. The allocator asked us, “How much would you make on those put options (which increase in value as the underlying stocks fall) if QQQQ goes to zero?”

My partner and I looked at one another. It’s one thing for an individual stock to go to zero—it can happen and it does. It’s another altogether for a diversified index to lose all its value. This is essentially saying the entire stock market and, thus the whole economy, is essentially worthless. While the answer to the question could be computed easily enough, it’s one of those scenarios truly too horrible to consider because its implications would make the 2008 credit crunch look like nothing more than a gently stubbed financial toe.

“Please don’t take this the wrong way,” I replied, “but if QQQQ goes to zero, how much we make on those put options should be the least of your concerns.”

We didn’t get the business. Some people have no sense of humor. But the fact that someone could even ask that question helps you realize that the efficient market theory is pretty far removed from reality.

Another asset allocator story helps show how the investment process often has little to do with discovering value; it also happened during the convertible bond meltdown of 2005.

My marketing guy told me I’d be having a meeting with a prospective new investor. “These guys really understand converts, and they know there’s a lot of value in the market now,” he told me. “They’re going to want to talk about the trades you really like and why you like them.”

At last, I thought. I’ll get the chance to show why people should be buying, not selling. Anybody who understood anything—anything at all—about convertibles could see that the securities, relative to their underlying stocks, were more attractive than they’d been in years, perhaps decades.

So later that day the investors came in. We sat down, made some pleasant small talk about the spring weather for a few minutes. I began to mention that I had some trades that I was especially excited about. Suddenly their faces turned glum and gray. “Just tell us one thing,” one of them requested. “When is the selling going to stop?”

This was one of those moments when they tell you to count to 10—or perhaps to 100—before you say anything. I bit down as long as I could and then bit no longer. “I don’t know,” I replied, “I guess when guys like you stop selling.”

We didn’t get that business either.

I think that meeting had a bigger impact on me than any other one in my career. Here were individuals charged with trying to understand the guts of an opportunity. They chose instead to ask me to time the market for them—something nobody should ever be asked to do, because it’s something that, really, nobody can do.

Investors should ask questions similar to those Jason Zweig recommended succinctly:

Use “the five whys.” Rather than asking questions that have “yes” or “no” answers, ask “how” and “why” questions. Don’t ask if someone deserves a bonus or whether an adviser is confident that an idea will generate superior returns. Ask, instead, why the bonus is deserved or why the idea is superior. Then ask why the person knows that answer is right. If you ask five such “why” questions in a row, you are likely to expose any weak points in the advice.

I hope you can see that I’m trying to point out a deep problem in our financial system. It’s all about the herding instinct, the desire to belong to a club. It’s about the fear of thinking for oneself that is native to most people and especially to Wall Street’s legions of minor-leaguers getting paid major-league money. It’s about thinking that if you say something often enough, you’ll make it the perceived truth regardless of the objective reality.

Please check out another excerpt posted here, The Parable of the Classroom. Think about those times in school when the teacher confused pretty much everyone in the class, then asked if there were any questions. Remember how rarely anyone spoke up? It’s because people have an intrinsic fear of sticking out, of appearing to be the outlier who doesn’t get it. When enough people with this mind-set control enough money—and trust me, they do—the foundation of the efficient market theory crumbles to dust.

To read more on this topic, please see the columns I’ve written for Minyanville.com (free registration required):

What Do A-Rod and the Market Have In Common? (2/10/2009)

How Betting the Horses Makes Better Investors (4/6/2009)