Back to Community
Computing the Fama-French Factors with Pipeline

The recently-released Pipeline API allows you to swiftly run computations on large universes of stocks. This creates a vast world of possibilities, one of which is the implementation of the Fama-French Three Factor Model. Computing these factors requires partitioning a large universe of stocks, which canonically involves thousands of equities: before Pipeline, this wasn't possible on the Quantopian platform. Now it is.

My implementation allows for computing rolling Fama-French factors over any time period. The accuracy of my model can easily be confirmed, because Ken French has published datasets of the Fama-French factors over various timespans. Below are some examples of Ken French's and my results:

July 2015

August 2014 - August 2015

There are, of course, discrepancies due to differing methodologies. For one, Ken French only considered data from the NYSE, AMEX, and NASDAQ exchanges, whereas Quantopian draws data from over twelve US exchanges. Arguably, my implementation offers a more holistic and complete view.

Furthermore, Ken French computed his factors strictly by calendar period (week/month/year). While it's possible to do so in Quantopian as well, it requires a little wrangling, as the native unit on Quantopian really is the business day. For the sake of simplicity, I left my script in terms of business days: augmenting it to handle particular periods is reasonably straightforward. Note that relatively small changes to the parameters of the Fama-French factors (e.g. computing them over 22, as opposed to 23 business days) can have relatively large impacts on the results, so be careful.

I hope that this algorithm is useful to you in two ways:
1. This implementation concretely illustrates a use for the Pipeline API.
2. When Pipeline is deployed to Quantopian Research, you'll be able to use variable-length Fama-French factors to regress against the returns of your algorithms, giving you further insight into your strategies.

Feel free to play around with this and share your findings if you come across anything interesting. I'm keen to see what you come up with.

One thing I'd be particularly interested in is weighting equities: in the canonical implementation, the universe is partitioned into six disjoint subsets, and equal weight is given to every equity in every subset. What's problematic with this is that you get equities that are very close to boundaries, but still carry equal weight for their categories. It might be interesting to look into weighting equities in their subsets according to distance from the center of the subset.

Clone Algorithm
443
Loading...
Backtest from to with initial capital
Total Returns
--
Alpha
--
Beta
--
Sharpe
--
Sortino
--
Max Drawdown
--
Benchmark Returns
--
Volatility
--
Returns 1 Month 3 Month 6 Month 12 Month
Alpha 1 Month 3 Month 6 Month 12 Month
Beta 1 Month 3 Month 6 Month 12 Month
Sharpe 1 Month 3 Month 6 Month 12 Month
Sortino 1 Month 3 Month 6 Month 12 Month
Volatility 1 Month 3 Month 6 Month 12 Month
Max Drawdown 1 Month 3 Month 6 Month 12 Month
# Backtest ID: 561cc737f1d47510e22ca221
We have migrated this algorithm to work with a new version of the Quantopian API. The code is different than the original version, but the investment rationale of the algorithm has not changed. We've put everything you need to know here on one page.
There was a runtime error.
19 responses

Hi John, great post, very clean code. I was investigating this kind of strategy with pipelines, however when I run the backtest it stalls. My own pipeline tests stall too. I mean they dont seem to start processing any data. Do you see this problem at all ? I'm wondering if the pipeline api has any glitches at the moment... cheers

I found the same, and later realized the warm-up period (all the pre-calc work) for pipeline takes some time. I later came back to it and it ran to completion. Try again and let it sit and confirm you also see it complete.

hmmm... just cloned it again and got this when running:

31 Error Runtime exception: AttributeError: 'module' object has no attribute 'valuation'

I've been getting that sporadically today as well, if you retry it may work. I think one of their backtesting nodes hasn't been updated or something.

Simon's right - some of our backtesting servers haven't updated quite yet. I'm forcing the update through now. The problem should be fully resolved in about 30 minutes. Thanks for the heads-up and apologies for the trouble!

Disclaimer

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.

Simon's right, we're fixing now.

Disclaimer

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.

thanks for the quick turnaround on this guys. I'm wondering if this returns based example could be a way of calculating the return of a portfolio from a universe with more than 500 stocks, without having to 'trade', so thanks for this idea John.

I'll update here if I get a good template working.

Tom, that should certainly be possible. You'll want to use the Returns factor and then write another factor similar to the CommonStock factor to filter down to your portfolio of choice.

I just cloned this and can't run the backtest, follows is the error... is there something obvious ?

Something went wrong. Sorry for the inconvenience. Try using the built-in debugger to analyze your code. If you would like help, send us an email.
TypeError: Don't know how to construct AdjustedArray on data of type object.
There was a runtime error on line 83

@all, this algo no longer works and fails on line 83, as stated by @Jehill . Can someone look at it and fix it? Thanks.

If i want to do exactly like Fama French data, how do i restrict the stock exchange within NYSE, AMEX, and NASDAQ?

BTW, the backtest now works. FYI

Hey everybody. The lectures on Fundamental Factors and Factor Risk Exposure now show how to construct and analyze Fama French portfolios in pipeline. Might be useful to people on this thread.

https://www.quantopian.com/lectures

Disclaimer

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.

@paul
Odd how it stopped working for a period of time...

Here is a 31 line solution for accessing any of the 100 cross sectional portfolios utilized by Fama/French in their paper. It only takes two code modifications to isolate any of the portfolios in the matrix:

  1. Alter the Market Cap filter on line 22
  2. Alter the Book to Market filter on line 30

The result will be a daily output of each cross sectional portfolio in the context.FF_Portfolio variable. The attached example outputs the portfolio depicted by the green cell in the matrix below:

Clone Algorithm
16
Loading...
Backtest from to with initial capital
Total Returns
--
Alpha
--
Beta
--
Sharpe
--
Sortino
--
Max Drawdown
--
Benchmark Returns
--
Volatility
--
Returns 1 Month 3 Month 6 Month 12 Month
Alpha 1 Month 3 Month 6 Month 12 Month
Beta 1 Month 3 Month 6 Month 12 Month
Sharpe 1 Month 3 Month 6 Month 12 Month
Sortino 1 Month 3 Month 6 Month 12 Month
Volatility 1 Month 3 Month 6 Month 12 Month
Max Drawdown 1 Month 3 Month 6 Month 12 Month
# Backtest ID: 57f15b93dcc04d104c6ecb29
There was a runtime error.

Here is a fun way to visualize returns by Fama-French standards:
(Coding for the visual was derived from Q tutorials)

https://quantonomist.com/FF_Heat_Map

Very cool visual, would it be possible to get some kind of 3D plot where you could see the returns of each decile portfolio through time?

@Delaney, that is a great idea, but a few of the dependencies needed to output 3D plots are restricted. I threw the code to make the 3D rendering as well as cluster the data in the last cell of this notebook. Will revisit in future...

References used:

Seong - https://www.quantopian.com/posts/research-do-you-want-parameter-optimization-click-here-to-get-started-heat-maps-included
Harrison - https://www.youtube.com/watch?v=3ERPpzrDkVg&list=PLQVvvaa0QuDfKTOs3Keq_kaG2P55YRn5v&index=39

Loading notebook preview...
Notebook previews are currently unavailable.

Ah that's too bad. Still a really great notebook that others can improve on their own.