What is a stock index?

An index is a group of companies/stocks altogether, to help you understand what’s going on in the overall market. That’s why they were invented years ago- as guides and measurement tools.

But nowadays you need to understand that indices are very different to that humble intention, as the rise of passive investing has turned indices from an overall representation of the market, to a sellable investment product making firms billions every year in fees. Once something goes from a simple metric with no monetary impact to a vastly valuauble sellable product in a competitive market, clearly the incentives change enormously.

The first thing to understand is how they are constructed, and there are a variety of methods but two main ones that are worth considering:

Market cap (capitalisation) weighted

S&P 500, FTSE 100, DAX, NASDAQ, Hang Seng etc (most are)

Market cap weighting is arguably the most sensible. It basically adds up all the market caps of the constituent companies, and that’s your index level. So the higher the market cap, the higher the weight in an index. This means ‘larger’ companies have more weight, assuming market cap is a representation of size- which we know it isn’t. Market cap is a representation of popularity amongst investors, so market cap weighted indices are popularity indices, not size indices.

While market cap is arguably the most sensible of a number of terrible ways of creating an index, it does bring some problems. It results in an index becoming less diversified and more risky un-during a period of a speculative boom- as the stocks with wild performance and huge valuaitons end up making up more and more % of the index- the exact time when you’d rather have some risk reduction from diversification.

Price weighted

The fact that this even exists today is laughable. Yet one of the most often quoted indices in the world- the Dow Jones, uses an archaic and utterly meaningless methodology for construction- price weighting.

The Dow is a manually selected index of 30 companies, chosen manually by a committee. The weight of each company in an index is then determined by its share price in dollars.

What this means is that a stock with a share price of $1000 will have ten times more weight in the index than a stock with a share price of $100. So a company’s weight in the index is driven by how many shares that company decides to issue- not by anything related to actual company size. If I have a company worth $10billion, and I only issue 100 shares, each share is worth $100million. Compare that to a company worth only $1bn that has a million shares in issue, where the share price is just $1,000, and the first company (despite being with 100x more in reality) will have 0.00001 of the weight in the index. It’s ridiculous.

For a real-life example of how nonsensical this approach is; when Apple underwent a stock split issuing 4 new shares for every 1 existing in 2020, obviously the price of each share fell 75%, as you’ve now got 4 shares vs your original one, so your shareholding is worth exactly the same. The size of Apple as a business was unchanged but on the day of the change it’s weight in the Dow dropped by 75% instantly. Was Apple suddenly 75% less important or influential as a company? In real terms it was exactly the same company as it was the day before.

Equal weighted

There are often versions of the mainstream indices that are made available on an ‘equal weighted basis’. Which simply means of the chosen company constituents, each one has the same impact on the index level. If there’s 100 stocks, each is given a weight of 1%. That removes any size issues, or performance issues. It can provide a more useful representation of overall market performance, as this approach removes the over- allocation to stocks that are seeing huge price appreciation and overvaluation. The flipside is that it gives equal weight to tiny companies vs huge ones.

Why are indices less helpful now than they used to be?

Index providers are selling a product today- they have less interest in the index being representative of anything useful, and every interest in marketing a product. These indices generate billions of dollars in fees as passive investment vehicles, and as benchmarks for active funds. Index providers like FTSE, MSCI, Russell and S&P are competing- and the indices are their products. What was once a simple representation of the market is now one of the most profitable industries ever created.

A perfect example of how this change has totally warped how useful indices are is the invention of a new sector in 2018 to hide the lack of diversification in the S&P, which was putting off many investors. At that point the S&P was around 40% technology companies, so many investors were shying away from it saying it’s no longer offering diversified protection. That’s not good for S&P as a business, they need the S&P to be a popular investment product. So to hide the heavy tech weighting S&P invented a new sector called ‘Communication services’, moved half the tech stocks into that and boom, the index was now only 18% tech and investors were happy buying it again.

Absolutely nothing had changed but how they classified stocks. It was pure window dressing and a shameless cover up. But most investors (professional and retail) do little actual research work, so take the sector weights at face value and immediately assumed the S&P was now much more diversified. The real risks were identical. Not a single thing had changed about the index apart from how they presented it. Do not trust any financial firms- their incentive is to make money not to protect you.

These ‘passive’ index providers are very much making an ‘active’ decision on whether a company (or even country) should be included or excluded in their index. And those decisions move billions of dollars of real money.

Outside actors can also have outsized influence on an index’s construction – eg Chinese authorities lobbied MSCI for years to include Chinese stocks despite obvious the legal issues of Chinese stock ownership (as a Western investor you never actually own anything in China- we can talk about that later).

Indices change over time

Given all of the above and constantly changing stock prices, it’s fairly obvious that indices change a lot over time. The make up of an index today is very different to that same index five or 10 years ago.

Just 10 years ago, the S&P 500 was over 13% Energy, today it is less than 2% Energy. Over the same period Technology has doubled from 18% to around 40% (including “Communication services” – because it is still Technology). It is for this simple reason that whatever valuation methodology you choose to use as a yardstick (PE, Cyclically Adjusted PE etc etc), an index simply cannot be compared across time periods. While sector neutral versions of the index may come closer, they still fall down as the constituents of each sector also continually change.

This is a significant issue, and one often overlooked. You have probably seen many comments about the ‘S&P trading at all time highs’, ‘FTSE at record levels’, etc. All these comments are meaningless because of the simple fact that any index is not comparable across time- you’re not comparing apples with apples. (This same issue does exist with some individual companies also- Amazon today is a global technology company dominating cloud storage, but 10 years ago it was an online bookstore- are multiples or comparisons of those businesses really appropriate?)

So the next time someone in a newspaper or a research report is proudly displaying a chart of historic market multiples you can ignore it and smile, because it’s nonsense.