Back to Community
Historical Constituents of S&P Index? (or other indexes: Nasdaq, Dow, etc.)

+

Looking into basket strategies out of index constituents, thus need historical constituents of an index.
The most obvious is S&P500, for starters.
Is this type of data integrated into Quantopian? And if not, how to get it and incorporate?

Many thanks,
Zlatko

8 responses

Data vendors keep and charge for this kind of historical data and we don't currently provide that through Quantopian.

A recent thread might be of help on this though: https://www.quantopian.com/posts/feature-request-security-universe-with-constantly-updated-s-and-p500#557ebc9b9a496741b600048f

Disclaimer

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.

For historical components of Dow Jones, just check Wikipedia.

For components of S&P 500 and NASDAQ-100, check this: http://marketcapitalizations.com/historical-data/historical-components-sp-500/

Bothering about the constituents of an index over time is a futile and unnecessary exercise when Quantopian already gives you two better alternatives: the ability to back test on baskets of stock chosen by liquidity or market cap.

An index is much the same anyway and Quantopian gives you the ability to create your own. Use it would be my advice.

I don't disagree with with Anthony. It's pretty easy to create your own dynamic universe of stocks using fundamentals data.

That said, you might want to trade on the announcement of a stock being added to an index -- for that purpose, you can use data such as the EventVestor Index Change data.

A neat data consistency check is to take the S&P 500 constituents through time, calculate their daily returns, match to a time series of shares outstanding and recreate the index return. Comparing to the returns of SPY or the S&P500 total return is, IMHO, the first step in creating a truly unbiased universe of historic equities and some of the underlying data. This can be extended to the Russell 3000 and, viola!, you have a verifiable, as reported, universe of US equities that cover virtually every single equity you might invest in (ok, ok, no BRK.B before 2010). Not only that, you have a quantitative reason to believe that you've reduced survivorship bias and look ahead bias in the prices, dividends, other corporate actions (returns) and shares outstanding. For extra credit, compute the index earnings yield as a time series and compare to published reports to verify earnings data.

It works beautifully. I showed this trick to Standard and Poors and they promptly included an essential part of it in their Compustat Xpressfeed product. Needless to say, without giving any credit.

Try it on Q's data. See how close the data set here compares to reality.

A major gap in replicating the Russell indexes, at least, is that there is no indication in the Morningstar data of companies which are MLP/pass-throughs and/or report "unrelated business taxable income". This means that universes constructed on market cap in quantopian typically are substantially overweight energy partnerships.

You are correct, Simon, that the Russell index construction is somewhat contrived; no foreign subsidies, no multiple share classes, no Berkshire Hathaways. However, the rules are easily reproduced; see here for starters and even better, the index constituents, which only change once per year other than some corporate actions, are published.

I ran a fund that, in part, used index adds/deletes as a component and tried to front-run both the Russell reconstitution (easy) and S&P announcements (harder). There was a classic price response both before and after the announcement and actual index change.

Sally, I don't at all follow what you are saying. The issue/question here is are the historical "constituents" available. Secondly, if available is pricing and shares (market cap weighted) also available - not how you can "replicate" an index - that's easy. 10%-20% of the constituents in the S&P500 change every 5 years or so. Most probably due to M&A.

The question is ... does Quantopian have the historical constituents with pricing / shares?