This notebook calculates the weighted correlations of top US equities over time. Correlation here is the Pearson correlation, which is a measure of how similar two sets of returns are. For example, two stocks with returns that are exactly the same over the past 6 months will have a correlation of 1.0, and two stocks with returns that are completely unrelated will have a correlation close to zero.
For each day over the past 15 years it grabs the top 100, 500, etc. US equities, and calculates their correlations between each other. Then the notebook weights all the correlations by market cap, calculates the mean correlation each day, and plots it.
The interesting plots are at the bottom. We can see that the recent overall correlation is low. This means that recently, stock returns have been more diversified and unrelated to each other.
Feel free to look through the code and build upon it!