Sharpe ratio, other risk metrics change; Slippage in Live Trading

It's time for an update on some backtester changes.

Several weeks ago we had a bundle of changes to the Sharpe ratio ready to go. Before we released the changes, one of our community members found a mistake in our calculations (thanks, Simon!) . We did another round of revisions to the Sharpe ratio, and we updated the alpha, beta, Sortino, and Information Ratio to use the same methodology. We finished that project last week and rolled it out on Friday evening. Backtests using the "old" calculation will need to be re-run if you want to see the new calculation. Shortly we'll be putting up a notice on each of the "old" backtests so you can tell which ones are which. I'll repeat what I said before about the effects of the changes:

"We're going to standardize our Sharpe calculation on daily returns that are annualized (with a few exceptions that will be obvious, like intra-day calculations on live trading). Today, our Sharpe is calculated using the absolute returns over the period specified. As an example, if we look at the Sharpe ratio for July, we're looking at the returns on July 31 minus the returns on July 1; if we're looking at the Sharpe for the last year, we subtract the result of 21-Aug-12 from 21-Aug-13. This method works well in a lot of situations, but it makes it difficult to compare the lifetime Sharpe ratio for algorithms if one algo has been running for longer than the other. When we roll out this change, we're going to fix that problem by taking the returns of each individual day over the period specified and then calculating an annualized Sharpe ratio.

"If the period in question is a day, the returns over that day will be taken and annualized over a 252-day year. Similarly, the Sharpe over July will be calculated starting with the sum of the returns of 22 individual trading days, and then annualized. The Sharpe for a year will be calculated with the sum of the returns of the 252 different trading days in the year.

"For the mathematically inclined, some more detail: yes, we are calculating the expected value of the daily returns divided by the square root of the daily variance of the returns as described in the formula in Wikipedia. That means to annualize a daily return, the result is multiplied by the square root of 252.

"One of the lessons we learned from this process is that we need to be more explicit about how we calculate our ratios and risk metrics. The calculation is performed in Zipline, so anyone can see the calculation there, but not everyone can understand the Zipline code. So, we started sharing this Excel file. This Excel file is the "answer key" that we test Zipline against. We think that more people will be able to understand the Excel file. If you see anything in our answer key you think isn't correctly calculated, we'd like to hear about it so that we can find and fix it. Our Sharpe ratio calculation was following our answer key, but the community prompted us to review our calculation and revise the answer key."

Separately, we fixed a bug in paper trading this morning. We found that paper trading algorithms weren't honoring the set_slippage() or set_commission() in initialize(). If you are affected by this bug you will notice a discontinuity in your algorithm results this morning. Accounts that connected to IB for paper trading were not affected.

As always, we try to avoid changes to the backtester. In backtesting, stability and correctness are virtues that are sometimes in conflict. We welcome feedback about how we're balancing them.

Disclaimer

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.

2 responses

That means to annualize a daily return, the result is multiplied by the square root of 252.

Maybe with volatility (under some assumptions), but for converting daily returns to annualized this does most certainly not hold. But I guess you mean Sharpe Ratio here.

Bernd, the fuller quote is more instructive. Yes, this is the Sharpe Ratio.

For the mathematically inclined, some more detail: yes, we are calculating the expected value of the daily returns divided by the square root of the daily variance of the returns as described in the formula in Wikipedia. That means to annualize a daily return, the result is multiplied by the square root of 252.