Python Pairs Trading - Z score and Correlation Coefficient for negative correlation (2)

Hi All, This is more conceptual than technical, as I can insert the logic into the below for loop if I know what I'm looking to do. I have a DataFrame, Df with zscores, correlation coefficients for time series (stocks) data.

The two columns being referenced accordingly are "pairs_zscore" and "ES_CC" (correlation coefficient between the S&P 500 index and US treasury futures). Here's the for loop to determine if I'm going to get long ES (S&P 500 futures) based on the relationship between S&P 500 futures and treasury futures:

for index, row in Df.iterrows():
if Df.loc[index, 'pairs_zscore'] < -.5 and Df.loc[index, 'ES_CC'] > .4:
Df.loc[index, 'Lead_Lag'] = "Long_ES"
elif Df.loc[index, 'pairs_zscore'] < -.5 and Df.loc[index, 'ES_CC'] < -.4:
Df.loc[index, 'Lead_Lag'] = "Long_ES"


So for instance, in the first if statement I'm saying if the z-score is < -.5 and the correlation coefficient is positive (>.4) between S&P 500 (ES) and Treasury futures, then the spread between ES and treasury futures is less than the mean and so I want to get long S&P 500 futures.

For instance, if S&P 500 is at 2300, and treasury futures are priced at 1000, but then treasuries go up to 1100 and S&P 500 stays at 2300, the spread has tightened and z-score will be negative. So I would get long S&P futures expecting them to go up and the spread to widen again back to the mean.

That's straightforward enough, I think the logic in the for loop makes sense for when the two assets correlate positively, but what about when they're negative. As shown above, my logic right now is if the z-score is negative (spread between S&P futures and treasury futures is less than average) and Correlation Coefficient (CC) is negative, then I would anticipate the spread reverting to the mean again.

For instance, if the correlation is negative between the two assets and treasuries move down, it makes sense that S&P 500 should move up (because they move in opposite directions). If the S&P 500 hasn't moved up then the spread is smaller than it should be. So it seems like z-score for negative correlating variables works entirely differently from z-score for positive correlating assets.

So the question is: How would I code the logic for negative correlation and z-scores to figure out when to get long the ES if treasuries have dropped in value?

edit: To clarify, what I'm trying to do seems straighforward...maybe z score just isn't the tool? If two assets correlate negatively, one asset moves up, you would expect the other to move down. If it hasn't moved down yet, it's "lagging" the other asset in absolute movement. I want to figure out how to identify that situation and short the asset that hasn't moved down yet.

3 responses

Hello Cole,

Many of your questions will likely be answered by going through the following two lectures.

Let me know if things still don't make sense when you've gone through these.

Disclaimer

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.

Hey Delaney,

Thanks for responding to my post. So I did find those lectures on pairs trading prior very useful for positively correlating assets. However, when two assets have a negative correlation, I don't think they account for pairs trading based on what is probably an ever-widening spread (for negative correlating assets). For instance, if ES has been trending up long-term like it has, and treasuries have been trending down, that negative correlation can't be captured by traditional pairs trading methods (at least I don't think it can...?). Because the relationship isn't mean-reverting, the spread just keeps getting wider and wider. But at the same time, one asset can lead the other where one will move up (like ES) and the other doesn't move down. I don't think this is covered in your lectures as pairs trading is usually based around positively correlating assets, but definitely feel free to correct me if I'm wrong there!
Thanks, Cole

Hey Cole,

Great question. Negative correlation doesn't imply an ever widening spread. Nor does cointegration imply positive correlation. The key differentiation is between correlation and cointegration. In negatively correlated assets, you can still see mean reversion. Negative correlation is symmetric with positive correlation, and just means that when one gets bigger the other gets smaller and vice versa. See the attached notebook for an example. Basically it doesn't matter if you move the same way or the opposite way, you can still have mean reverting behavior. Basically non-zero correlation is often linked with, but also can be independent of, mean-reversion.

I think what you're getting at is the difference between whether the spread is mean-reverting or trending. This is captured by the cointegration test, and should be thought of somewhat independently from whether the series are correlated. In pairs trading you do want to watch for whether the spread is starting to become a trending series rather than a mean reverting series. The Hurst exponent is a way of doing this, see here: