It's time for an update on some backtester changes.
Several weeks ago we had a bundle of changes to the Sharpe ratio ready to go. Before we released the changes, one of our community members found a mistake in our calculations (thanks, Simon!) . We did another round of revisions to the Sharpe ratio, and we updated the alpha, beta, Sortino, and Information Ratio to use the same methodology. We finished that project last week and rolled it out on Friday evening. Backtests using the "old" calculation will need to be re-run if you want to see the new calculation. Shortly we'll be putting up a notice on each of the "old" backtests so you can tell which ones are which. I'll repeat what I said before about the effects of the changes:
"We're going to standardize our Sharpe calculation on daily returns that are annualized (with a few exceptions that will be obvious, like intra-day calculations on live trading). Today, our Sharpe is calculated using the absolute returns over the period specified. As an example, if we look at the Sharpe ratio for July, we're looking at the returns on July 31 minus the returns on July 1; if we're looking at the Sharpe for the last year, we subtract the result of 21-Aug-12 from 21-Aug-13. This method works well in a lot of situations, but it makes it difficult to compare the lifetime Sharpe ratio for algorithms if one algo has been running for longer than the other. When we roll out this change, we're going to fix that problem by taking the returns of each individual day over the period specified and then calculating an annualized Sharpe ratio.
"If the period in question is a day, the returns over that day will be taken and annualized over a 252-day year. Similarly, the Sharpe over July will be calculated starting with the sum of the returns of 22 individual trading days, and then annualized. The Sharpe for a year will be calculated with the sum of the returns of the 252 different trading days in the year.
"For the mathematically inclined, some more detail: yes, we are calculating the expected value of the daily returns divided by the square root of the daily variance of the returns as described in the formula in Wikipedia. That means to annualize a daily return, the result is multiplied by the square root of 252.
"One of the lessons we learned from this process is that we need to be more explicit about how we calculate our ratios and risk metrics. The calculation is performed in Zipline, so anyone can see the calculation there, but not everyone can understand the Zipline code. So, we started sharing this Excel file. This Excel file is the "answer key" that we test Zipline against. We think that more people will be able to understand the Excel file. If you see anything in our answer key you think isn't correctly calculated, we'd like to hear about it so that we can find and fix it. Our Sharpe ratio calculation was following our answer key, but the community prompted us to review our calculation and revise the answer key."
Separately, we fixed a bug in paper trading this morning. We found that paper trading algorithms weren't honoring the set_slippage() or set_commission() in initialize(). If you are affected by this bug you will notice a discontinuity in your algorithm results this morning. Accounts that connected to IB for paper trading were not affected.
As always, we try to avoid changes to the backtester. In backtesting, stability and correctness are virtues that are sometimes in conflict. We welcome feedback about how we're balancing them.