I really hate arguing with Warren Buffett…

After all, the Oracle of Omaha can call “scoreboard” on just about any investor alive today. His cumulative returns at Berkshire Hathaway are a whopping 2,472,627% since 1965. No one in history has produced returns that high, over a period that long, and with a portfolio that large.

Regardless, Buffett has been giving what I consider to be dangerous advice as of late.

In a CNBC interview last year, Buffett mentioned that ordinary investors should “consistently buy an S&P 500 low-cost index fund,” adding “I think it’s the thing that makes the most sense practically all of the time.”

Buffett has made similar comments in other interviews, too. To his credit, he puts his money where his mouth is. In his will he instructed his executors to invest the money left for his wife as follows: 10% in short-term government bonds and 90% in an S&P 500 index fund.

The problem with Buffett’s comment isn’t that it’s completely wrong. The problem is that it’s almost right.

Over time, the stock market rises. At least it has over America’s history as an industrialized nation. I have no reason to doubt that, over the long term, the market will continue to rise. Barring nuclear armageddon or an environmental catastrophe, American companies are likely to continue generating wealth, and buying a diversified portfolio of stocks gives you a piece of the action.

The key here is “over the long term.”

If you’re 30 years old and you have a good 30-40 years until retirement, chances are good that you’ll make money following Buffett’s advice. Dollar-cost averaging — or adding to your investment in regular intervals over time — allows you to systematically buy the dips. Bear markets are fantastic buying opportunities.

But what if you don’t have 30 years? Or what if you’re already retired and thus don’t have fresh cash from a paycheck to buy the dips? Do you really want your entire nest egg subject to the wild swings of the stock market?

The Fault in the S&P 500

The S&P 500 is up well over 330% from its 2009 lows. But remember, those returns were only possible because we were coming off of generational lows following the 2008 meltdown. If you look at the annualized returns since the 2000 top, the returns are less than 4% per year. All of those gains happened in the past six years. Between 2000 and 2013, the S&P 500 had returns of exactly zero.

And this wasn’t an isolated incident. The 1970s were another lost decade. Stocks went nowhere between 1968 and 1982. If you were the unlucky one that bought at the top in 1929, you wouldn’t have broken even until 1954 — 25 years later. Looking overseas, the Japanese Nikkei is still 46% below its old 1989 high… 30 years later.

By some metrics — including widely followed ones like the price/sales ratio — the S&P 500 is as expensive today as it was at the peak of the 1990s tech bubble. This doesn’t guarantee that we’re looking at another lost decade in the stock market. But it would make me think twice before blindly putting my cash into an index fund and hoping for the best.

Since the 2009 bottom, the bull market has been led by a small core of large-cap tech names: Apple (Nasdaq: AAPL), Amazon (Nasdaq: AMZN), Alphabet (Nasdaq: GOOGL), Microsoft (Nasdaq: MSFT), and Facebook (Nasdaq: FB). But as I mentioned on Friday, all of these companies are facing potential antitrust action by the government that could wreck their business models.

Even if that doesn’t happen, it’s not reasonable to expect these companies to continue leading the market higher. Three are flirting with trillion-dollar market caps, and one — Microsoft — is already worth more than a trillion dollars.

The combined market caps of these five stocks are larger than the GDPs of every country in the world but the United States, China, and Japan. How much bigger can these companies realistically get?

If you’re buying an S&P 500 index fund these days, you had better like these large-cap tech names. Because, given their size, as go these stocks, so goes the S&P 500 index.

I’m not necessarily expecting a bear market to start now. But I do believe we should be prepared for one.