Back to Community
Version 2, Proposal for Dividend-Adjusted Prices

Last week I posted a proposal for how to get dividend-ajdusted prices into Quantopian. The good news is that one of our developers, James, found a flaw in the proposal. I love it when we find the mistakes before we've done the hard work. . . . The flaw is that we were unintentionally creating the possibility of look-ahead bias. In the original proposal, at any given moment the backtest had access to both the dividend-adjusted price and the unadjusted price. That meant that the backtest could always know if there was a dividend scheduled in the future. That's not good.

Proposed Change (revised Apr-21-2015)

In the new proposal, the use cases and the actual calculation are the same as the April 13 proposal. The implementation and API access method are different than the original proposal. The API change is around the history call, not the data object. Here is some sample code:

open_history = history(bar_count=252, frequency='1d', field='open', dividend_adjusted=True)  
close_history = history(bar_count=252, frequency='1d', field='close_price', dividend_adjusted=True)  
low_history = history(bar_count=252, frequency='1d', field='low', dividend_adjusted=True)  
high_history = history(bar_count=252, frequency='1d', field='high', dividend_adjusted=True)  

The key concept in the new proposal is: throughout the course of a backtest, the dividend adjusted price and the current price are always the same ("current" in this context is the datetime within the backtest). This is perhaps best explained through example.

Company ABC puts out a $1 dividend quarterly with ex-dates of Jan 17, Apr 16, Jul 15, and Oct 15 in 2012. Company ABC is really boring and has the flattest stock price you've ever seen. It trades at 100 on the first day of the year and declines by a dollar on each ex-date as you'd expect, closing out the year at 96. In your algorithm you create a close_history dataframe as described in the example above. You want to run a backtest over 2012. During the course of your backtest, the prices look like this:

This can be a bit non-intuitive because the calculated dividend-adjusted price for a given day (i.e. Jan 3 2012) is different depending on what day the backtest is currently simulating. But when you think about it, it makes sense. At the end of 2012, the value of owning ABC's stock is exactly the same as it was on the first day of the year. The dividend-adjusted price for ABC is flat at all times. You can reconstruct this example with a more volatile stock and the principle is the same.

Most importantly, this implementation solves the algorithm writing use cases that we were targetting in the beginning.

Notes

  • The way that the Quantopian backtester handles dividends is unaffected. You still get dividends as cash payments. It's probably worth a minute of your time to review the current implementation:
  • With the change, your portfolio value is unaffected - still the same split-adjusted price times the same volume.
  • Even with the change, your portfolio overall returns will still have a "jag" during dividends. That's because your portfolio loses value on the ex-date when the price of the stock drops, and the portfolio recovers a few days later when the actual cash is delivered.

How it's calculated
Quantopian already provides split-adjusted prices. Our documentation describes it this way: "Our data uses adjusted close prices. That means that the effect of all stock splits and merger activity are applied to price and volume data. As an example: A stock is trading at $100, and has a 2:1 split. The new price is $50, and all past price data is retroactively updated 2:1 (volume data is correspondingly updated 1:2)."

In the same way, we will apply a dividend multiplier. When there is a dividend, all past price data will be updated by the dividend multiplier. (Yahoo's adjusted price is computed by the same process.

dividend_multiplier = 1 - (dividend_amount/close_price_pre_ex_date) = 0.nnnnn  

Use cases

  • You want to know the long-term economic return for a given security
close_history = history(bar_count=252, frequency='1d', field='close_price', dividend_adjusted=True)  
returns = close_history.iloc[[0, -1]].pct_change()  
  • You want to track the moving average, or perform any other calculations on a look-back window
close_history = history(bar_count=252, frequency='1d', field='close_price', dividend_adjusted=True)  
returns = close_history.mean  
  • You want to open or close a position (long or short) based on the upcoming dividend, or study price behavior around dividend surprises
    • Unfortunately, this is not supported still.
Disclaimer

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.

12 responses

Hi Dan - Any update on this, and when we can start using it?

Dan: The docs state: "Dividends are not relayed to algorithms as events that can be accessed by the API; we will add that feature in the future.". I understand you don't provide upcoming dividend information, but would it be useful to provide historical dividend events as you already have this information? I could see and algorithm making use of past trends without needing future events, or just wanting to ensure to hold the position until the dividend is paid out.

Simon - This feature is actively under development. I don't have a time frame for you yet. Hopefully a matter of weeks, but you never know until you do the testing and performance testing and such.

Paul - Unfortunately, this change doesn't solve that use case. I understand what you're looking for (at least I think I do), and it's a good feature request. It's just not in the scope of this improvement. It will be a different improvement, sometime in the future.

Hi Dan,

We are all still waiting on this. It would be really nice to have this before next month's contest start since I had yet another entry disqualified because of a intermittent fetcher file i/o error (used to load in the adjusted dividend data from yahoo finance instead). Without having historic dividend adjusted prices, it makes it hard to rely on relative historic alpha/beta calculations. Is there any hope to see this soon?

Thanks!

Rob

Rob, thanks for checking in. This was put on a slight pause while a couple other projects came up (for example), but it's still active and will be worked on. Once it's ready and available, we'll let you know.

Disclaimer

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.

Thanks Alisa, I completely understand the need to prioritize major bug fixes first - good to know that's it's still high up in the queue though.

Speaking of feature requests, I notice there have been many on this board and was wondering how you are keeping track of all of them! One nice way I've seen this handled on other sites is that there is a separate board just for "Ideas and Suggestions" with a way for the community to up vote those that are popular. This might give you a way of finding out which features would benefit the largest number of users, etc. For example check out: http://procreate.si/forums/index.php?board=10.0;sort=rating;desc

Hi Alisa,
Any updates regarding this new capability? Thanks.

The specific project we kicked off this spring ended up being a lot more difficult that we had anticipated (not the first time that's happened in software development, and not the last either!). We ended up shelving this particular implementation.

Instead, we're including this in the new Modeling API. We're working on that at full-speed.

Overall, this is why we generally resist giving release dates. We release new features when they're ready, not according to a calendar. And, sometimes we get a project to the finish line and then decide it's just not good enough. That's what happened here. We love the feature, and there is a lot of demand for it, but we're committed to doing it right.

Sorry for the delay. I'm betting you'll like it when it finally gets here.

Thanks for the update.

Hello Dan,
do you have any update on this 'close_history = history(bar_count=252, frequency='1d', field='close_price', dividend_adjusted=True)' ?
Any branch on zipline ?

Hey Dan,
I wanted to check in and see if you have any estimates on when there will be more dividend-related information and events available through the API?

Thanks!

Is it possible to get dividend-adjusted price history or was this feature abandoned?

It would seem the drop in stock price on exdate is counter-productive to any mean-reversion or momentum algo.