Back to Community
Difference between SPY and benchmark

I tried to mimic the benchmark algorithm by executing a simple buy and hold strategy of SPY. The same algorithm is used in this thread: https://www.quantopian.com/posts/backtester-change-updated-benchmark

As explained in the post of the link, there can be differences between both algorithms because it is not possible to buy partial shares of SPY and thereby use up all cash and because the benchmark is fully invested at market open, whereas SPY needs to be bought. So, in the following plot I understand that there is a discrepancy at the start of both algorithms. However, why is the difference suddenly increasing on 12/15/2016?

Cheers,
Kersten

Clone Algorithm
21
Loading...
Backtest from to with initial capital
Total Returns
--
Alpha
--
Beta
--
Sharpe
--
Sortino
--
Max Drawdown
--
Benchmark Returns
--
Volatility
--
Returns 1 Month 3 Month 6 Month 12 Month
Alpha 1 Month 3 Month 6 Month 12 Month
Beta 1 Month 3 Month 6 Month 12 Month
Sharpe 1 Month 3 Month 6 Month 12 Month
Sortino 1 Month 3 Month 6 Month 12 Month
Volatility 1 Month 3 Month 6 Month 12 Month
Max Drawdown 1 Month 3 Month 6 Month 12 Month
# Backtest ID: 5875b0c9e6ddb248ca298dc9
There was a runtime error.
3 responses

There is no disparagement between the graphs on day one. If there were, literally the first plot on the chart would be different. Here, the benchmark is purchasing the same amount as you at the same time as you are. I believe the issue is that the benchmark only uses intraday equity pricing and the order that you coded for the SPY long includes afterhours changes in value. I bet the graphs would match up if you open your position every morning at the open and exit your position right before the market closes. Quantopian does not include afterhours pricing and volume data, but it seems like, for some reason it dynamical calculates afterhours value change for your investments, but not for the benchmark. Now, I may be wrong about this, but ThinkOrSwim seems to suggest otherwise:
https://imgur.com/gallery/L7hWA
(The grey bars at the bottom of the chart are volume and when the volume bars drop to almost nill, that's when afterhours trading starts.)

-afterhours price increases between the 12th and 13th (accounting your your increase on datapoint #2)
-similar close and open on 14th, 15th and 19th
-significant price drop between 15th and 16th (accounting for your dip in price on the 16th)

Actually, if this is true, then it's a pretty significant issue and Quantopian support need to know about it.

Thanks for the quick response, James. You are absolutely right that this particular example does not show a difference between SPY and benchmark on the first data point. However, when choosing different time spans, you can observe a difference, e.g., when selecting 12/14/2016 to 12/16/2016.

I like your analysis that the difference might stem from different treatment of afterhours changes. However, I implemented your suggestion of opening the position in the morning and closing it in the evening. Every day, it purchases at 9.32am and sells at 4.00pm. Unfortunately, the graphs still don't match up (see below). I am wondering what I miss here.

Clone Algorithm
4
Loading...
Backtest from to with initial capital
Total Returns
--
Alpha
--
Beta
--
Sharpe
--
Sortino
--
Max Drawdown
--
Benchmark Returns
--
Volatility
--
Returns 1 Month 3 Month 6 Month 12 Month
Alpha 1 Month 3 Month 6 Month 12 Month
Beta 1 Month 3 Month 6 Month 12 Month
Sharpe 1 Month 3 Month 6 Month 12 Month
Sortino 1 Month 3 Month 6 Month 12 Month
Volatility 1 Month 3 Month 6 Month 12 Month
Max Drawdown 1 Month 3 Month 6 Month 12 Month
# Backtest ID: 5876fbd990ec025de709b25c
There was a runtime error.