Back to Community
Computing the Fama-French Factors with Pipeline

The recently-released Pipeline API allows you to swiftly run computations on large universes of stocks. This creates a vast world of possibilities, one of which is the implementation of the Fama-French Three Factor Model. Computing these factors requires partitioning a large universe of stocks, which canonically involves thousands of equities: before Pipeline, this wasn't possible on the Quantopian platform. Now it is.

My implementation allows for computing rolling Fama-French factors over any time period. The accuracy of my model can easily be confirmed, because Ken French has published datasets of the Fama-French factors over various timespans. Below are some examples of Ken French's and my results:

July 2015

August 2014 - August 2015

There are, of course, discrepancies due to differing methodologies. For one, Ken French only considered data from the NYSE, AMEX, and NASDAQ exchanges, whereas Quantopian draws data from over twelve US exchanges. Arguably, my implementation offers a more holistic and complete view.

Furthermore, Ken French computed his factors strictly by calendar period (week/month/year). While it's possible to do so in Quantopian as well, it requires a little wrangling, as the native unit on Quantopian really is the business day. For the sake of simplicity, I left my script in terms of business days: augmenting it to handle particular periods is reasonably straightforward. Note that relatively small changes to the parameters of the Fama-French factors (e.g. computing them over 22, as opposed to 23 business days) can have relatively large impacts on the results, so be careful.

I hope that this algorithm is useful to you in two ways:
1. This implementation concretely illustrates a use for the Pipeline API.
2. When Pipeline is deployed to Quantopian Research, you'll be able to use variable-length Fama-French factors to regress against the returns of your algorithms, giving you further insight into your strategies.

Feel free to play around with this and share your findings if you come across anything interesting. I'm keen to see what you come up with.

One thing I'd be particularly interested in is weighting equities: in the canonical implementation, the universe is partitioned into six disjoint subsets, and equal weight is given to every equity in every subset. What's problematic with this is that you get equities that are very close to boundaries, but still carry equal weight for their categories. It might be interesting to look into weighting equities in their subsets according to distance from the center of the subset.

Clone Algorithm
738
Loading...
Backtest from to with initial capital
Total Returns
--
Alpha
--
Beta
--
Sharpe
--
Sortino
--
Max Drawdown
--
Benchmark Returns
--
Volatility
--
Returns 1 Month 3 Month 6 Month 12 Month
Alpha 1 Month 3 Month 6 Month 12 Month
Beta 1 Month 3 Month 6 Month 12 Month
Sharpe 1 Month 3 Month 6 Month 12 Month
Sortino 1 Month 3 Month 6 Month 12 Month
Volatility 1 Month 3 Month 6 Month 12 Month
Max Drawdown 1 Month 3 Month 6 Month 12 Month
import pandas as pd
import numpy as np
from quantopian.algorithm import attach_pipeline, pipeline_output
from quantopian.pipeline import Pipeline
from quantopian.pipeline import CustomFactor
from quantopian.pipeline.data.builtin import USEquityPricing
from quantopian.pipeline.data import morningstar

# time frame on which we want to compute Fama-French
normal_days = 31
# approximate the number of trading days in that period
# this is the number of trading days we'll look back on,
# on every trading day.
business_days = int(0.69 * normal_days)

class Returns(CustomFactor):
    """
    this factor outputs the returns over the period defined by 
    business_days, ending on the previous trading day, for every security.
    """
    window_length = business_days
    inputs = [USEquityPricing.close]
    def compute(self,today,assets,out,price):
        out[:] = (price[-1] - price[0]) / price[0] * 100

class MarketEquity(CustomFactor):
    """
    this factor outputs the market cap of every security on the day.
    """
    window_length = business_days
    inputs = [morningstar.valuation.market_cap]
    def compute(self,today,assets,out,mcap):
        out[:] = mcap[0]

class BookEquity(CustomFactor):
    """
    this factor outputs the book value of every security on the day.
    """
    window_length = business_days
    inputs = [morningstar.balance_sheet.tangible_book_value]
    def compute(self,today,assets,out,book):
        out[:] = book[0]
                                        
class CommonStock(CustomFactor):
    """
    this factor outputs 1.0 for all securities that are either common stock or SPY,
    and outputs 0.0 for all other securities. This is to filter out ETFs and other
    types of share that we do not wish to consider.
    """
    window_length = business_days
    inputs = [morningstar.share_class_reference.is_primary_share]
    def compute(self,today,assets,out, share_class):
        out[:] = ((share_class[-1].astype(bool)) | (assets == 8554)).astype(float)                                     
        
def initialize(context):
    """
    use our factors to add our pipes and screens.
    """
    pipe = Pipeline()
    attach_pipeline(pipe, 'ff_example')
    
    common_stock = CommonStock()
    # filter down to securities that are either common stock or SPY
    pipe.set_screen(common_stock.eq(1))
    mkt_cap = MarketEquity()
    pipe.add(mkt_cap,'market_cap')
    
    book_equity = BookEquity()
    # book equity over market equity
    be_me = book_equity/mkt_cap
    pipe.add(be_me,'be_me')

    returns = Returns()
    pipe.add(returns,'returns')
    
def before_trading_start(context,data):
    """
    every trading day, we use our pipes to construct the Fama-French
    portfolios, and then calculate the Fama-French factors appropriately.
    """
    spy = sid(8554)
    
    factors = pipeline_output('ff_example')
    
    # get the data we're going to use
    returns = factors['returns']
    mkt_cap = factors.sort(['market_cap'], ascending=True)
    be_me = factors.sort(['be_me'], ascending=True)
    
    # to compose the six portfolios, split our universe into portions
    half = int(len(mkt_cap)*0.5)
    small_caps = mkt_cap[:half]
    big_caps = mkt_cap[half:]
    
    thirty = int(len(be_me)*0.3)
    seventy = int(len(be_me)*0.7)
    growth = be_me[:thirty]
    neutral = be_me[thirty:seventy]
    value = be_me[seventy:]
    
    # now use the portions to construct the portfolios.
    # note: these portfolios are just lists (indices) of equities
    small_value = small_caps.index.intersection(value.index)
    small_neutral = small_caps.index.intersection(neutral.index)
    small_growth = small_caps.index.intersection(growth.index)
    
    big_value = big_caps.index.intersection(value.index)
    big_neutral = big_caps.index.intersection(neutral.index)
    big_growth = big_caps.index.intersection(growth.index)
    
    # take the mean to get the portfolio return, assuming uniform
    # allocation to its constituent equities.
    sv = returns[small_value].mean()
    sn = returns[small_neutral].mean()
    sg = returns[small_growth].mean()
    
    bv = returns[big_value].mean()
    bn = returns[big_neutral].mean()
    bg = returns[big_growth].mean()
    
    # computing Rm-Rf (Market Returns - Risk-Free Returns). we take the 
    # rate of risk-free returns to be zero, so this is simply SPY's returns.
    # have to set an initial dummy value
    context.rm_rf = float('nan')
    if spy in returns.index:
        context.rm_rf = returns.loc[spy]
    
    # computing SMB
    context.smb = (sv + sn + sg)/3 - (bv + bn + bg)/3
    
    # computing HML
    context.hml = (sv + bv)/2 - (sg + bg)/2

def handle_data(context, data):
    # print the Fama-French factors for the period defined by business_days
    # ending on the previous trading day.
    print(context.rm_rf, context.smb, context.hml)
    pass
We have migrated this algorithm to work with a new version of the Quantopian API. The code is different than the original version, but the investment rationale of the algorithm has not changed. We've put everything you need to know here on one page.
There was a runtime error.
25 responses

Hi John, great post, very clean code. I was investigating this kind of strategy with pipelines, however when I run the backtest it stalls. My own pipeline tests stall too. I mean they dont seem to start processing any data. Do you see this problem at all ? I'm wondering if the pipeline api has any glitches at the moment... cheers

I found the same, and later realized the warm-up period (all the pre-calc work) for pipeline takes some time. I later came back to it and it ran to completion. Try again and let it sit and confirm you also see it complete.

hmmm... just cloned it again and got this when running:

31 Error Runtime exception: AttributeError: 'module' object has no attribute 'valuation'

I've been getting that sporadically today as well, if you retry it may work. I think one of their backtesting nodes hasn't been updated or something.

Simon's right - some of our backtesting servers haven't updated quite yet. I'm forcing the update through now. The problem should be fully resolved in about 30 minutes. Thanks for the heads-up and apologies for the trouble!

Disclaimer

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.

Simon's right, we're fixing now.

Disclaimer

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.

thanks for the quick turnaround on this guys. I'm wondering if this returns based example could be a way of calculating the return of a portfolio from a universe with more than 500 stocks, without having to 'trade', so thanks for this idea John.

I'll update here if I get a good template working.

Tom, that should certainly be possible. You'll want to use the Returns factor and then write another factor similar to the CommonStock factor to filter down to your portfolio of choice.

I just cloned this and can't run the backtest, follows is the error... is there something obvious ?

Something went wrong. Sorry for the inconvenience. Try using the built-in debugger to analyze your code. If you would like help, send us an email.
TypeError: Don't know how to construct AdjustedArray on data of type object.
There was a runtime error on line 83

@all, this algo no longer works and fails on line 83, as stated by @Jehill . Can someone look at it and fix it? Thanks.

If i want to do exactly like Fama French data, how do i restrict the stock exchange within NYSE, AMEX, and NASDAQ?

BTW, the backtest now works. FYI

Hey everybody. The lectures on Fundamental Factors and Factor Risk Exposure now show how to construct and analyze Fama French portfolios in pipeline. Might be useful to people on this thread.

https://www.quantopian.com/lectures

Disclaimer

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.

@paul
Odd how it stopped working for a period of time...

Here is a 31 line solution for accessing any of the 100 cross sectional portfolios utilized by Fama/French in their paper. It only takes two code modifications to isolate any of the portfolios in the matrix:

  1. Alter the Market Cap filter on line 22
  2. Alter the Book to Market filter on line 30

The result will be a daily output of each cross sectional portfolio in the context.FF_Portfolio variable. The attached example outputs the portfolio depicted by the green cell in the matrix below:

Clone Algorithm
26
Loading...
Backtest from to with initial capital
Total Returns
--
Alpha
--
Beta
--
Sharpe
--
Sortino
--
Max Drawdown
--
Benchmark Returns
--
Volatility
--
Returns 1 Month 3 Month 6 Month 12 Month
Alpha 1 Month 3 Month 6 Month 12 Month
Beta 1 Month 3 Month 6 Month 12 Month
Sharpe 1 Month 3 Month 6 Month 12 Month
Sortino 1 Month 3 Month 6 Month 12 Month
Volatility 1 Month 3 Month 6 Month 12 Month
Max Drawdown 1 Month 3 Month 6 Month 12 Month
from quantopian.algorithm import attach_pipeline, pipeline_output
from quantopian.pipeline.data.builtin import USEquityPricing 
from quantopian.pipeline import CustomFactor 
from quantopian.pipeline.data import morningstar 
from quantopian.pipeline import Pipeline
import pandas as pd
class MarketCap(CustomFactor):
    inputs = [USEquityPricing.close, morningstar.valuation.shares_outstanding]
    window_length = 1
    def compute(self, today, assets, out, close, shares):
        out[:] = close[-1] * shares[-1]
class Book_To_Market(CustomFactor):
    inputs = [morningstar.balance_sheet.total_equity, USEquityPricing.close, morningstar.valuation.shares_outstanding]
    window_length = 1
    def compute(self, today, assets, out, equity, close, shares):
        out[:] = equity[-1] / (close[-1] * shares[-1])   
def initialize(context):
    attach_pipeline(my_pipeline(context), 'my_pipeline')
def my_pipeline(context):
    pipe = Pipeline()
    ME_Factor = MarketCap()
    ME_Decile = ME_Factor.percentile_between(90, 100) # Adjust For ME Decile
    pipe.set_screen(ME_Decile)
    book_market = Book_To_Market()
    pipe.add(book_market, 'BE_ME')
    return pipe
def before_trading_start(context, data):
    context.output = pipeline_output('my_pipeline')
    context.output['BE_ME_Decile'] = pd.qcut(context.output['BE_ME'], 10, labels=False) + 1
    context.FF_Portfolio =  context.output.loc[(context.output['BE_ME_Decile'] == 2.0)] # Adjust For BE_ME Decile
    record(Portfolio_Size=len(context.FF_Portfolio))
There was a runtime error.

Here is a fun way to visualize returns by Fama-French standards:
(Coding for the visual was derived from Q tutorials)

https://quantonomist.com/FF_Heat_Map

Very cool visual, would it be possible to get some kind of 3D plot where you could see the returns of each decile portfolio through time?

@Delaney, that is a great idea, but a few of the dependencies needed to output 3D plots are restricted. I threw the code to make the 3D rendering as well as cluster the data in the last cell of this notebook. Will revisit in future...

References used:

Seong - https://www.quantopian.com/posts/research-do-you-want-parameter-optimization-click-here-to-get-started-heat-maps-included
Harrison - https://www.youtube.com/watch?v=3ERPpzrDkVg&list=PLQVvvaa0QuDfKTOs3Keq_kaG2P55YRn5v&index=39

Loading notebook preview...
Notebook previews are currently unavailable.

Ah that's too bad. Still a really great notebook that others can improve on their own.

hi,wondering why I get a terrible result if this algorithm works? any suggestions? Thank you.

Clone Algorithm
17
Loading...
Backtest from to with initial capital
Total Returns
--
Alpha
--
Beta
--
Sharpe
--
Sortino
--
Max Drawdown
--
Benchmark Returns
--
Volatility
--
Returns 1 Month 3 Month 6 Month 12 Month
Alpha 1 Month 3 Month 6 Month 12 Month
Beta 1 Month 3 Month 6 Month 12 Month
Sharpe 1 Month 3 Month 6 Month 12 Month
Sortino 1 Month 3 Month 6 Month 12 Month
Volatility 1 Month 3 Month 6 Month 12 Month
Max Drawdown 1 Month 3 Month 6 Month 12 Month
import pandas as pd
import numpy as np
from quantopian.algorithm import attach_pipeline, pipeline_output
from quantopian.pipeline import Pipeline
from quantopian.pipeline import CustomFactor
from quantopian.pipeline.data.builtin import USEquityPricing
from quantopian.pipeline.data import morningstar
from datetime import datetime
from quantopian.pipeline.filters import Q1500US

import statsmodels.api as sm
from statsmodels import regression


# time frame on which we want to compute Fama-French
normal_days = 31
# approximate the number of trading days in that period
# this is the number of trading days we'll look back on,
# on every trading day.
business_days = int(0.69 * normal_days)


# global em_df

ff_result=[]
date_index=[]

stock_held=15

# em_df=pd.DataFrame()



class Returns(CustomFactor):
    """
    this factor outputs the returns over the period defined by 
    business_days, ending on the previous trading day, for every security.
    """
    window_length = business_days
    inputs = [USEquityPricing.close]
    def compute(self,today,assets,out,price):
        out[:] = (price[-1] - price[0]) / price[0] * 100

class MarketEquity(CustomFactor):
    """
    this factor outputs the market cap of every security on the day.
    """
    window_length = business_days
    inputs = [morningstar.valuation.market_cap]
    def compute(self,today,assets,out,mcap):
        out[:] = mcap[0]

class BookEquity(CustomFactor):
    """
    this factor outputs the book value of every security on the day.
    """
    window_length = business_days
    inputs = [morningstar.balance_sheet.tangible_book_value]
    def compute(self,today,assets,out,book):
        out[:] = book[0]
                                        
class CommonStock(CustomFactor):
    """
    this factor outputs 1.0 for all securities that are either common stock or SPY,
    and outputs 0.0 for all other securities. This is to filter out ETFs and other
    types of share that we do not wish to consider.
    """
    window_length = business_days
    inputs = [morningstar.share_class_reference.is_primary_share]
    def compute(self,today,assets,out, share_class):
        out[:] = ((share_class[-1].astype(bool)) | (assets == 8554)).astype(float)                                     
        
def initialize(context):
    """
    use our factors to add our pipes and screens.
    """
    context.longs=[]


    pipe = Pipeline()
    common_stock = CommonStock()
    # filter down to securities that are either common stock or SPY
    pipe.set_screen(common_stock.eq(1))
    
    # pipe = Pipeline().query(common_stock.eq(1))
    attach_pipeline(pipe, 'ff_example')
    mkt_cap = MarketEquity()
    pipe.add(mkt_cap,'market_cap')
    
    book_equity = BookEquity()
    # book equity over market equity
    be_me = book_equity/mkt_cap
    pipe.add(be_me,'be_me')

    returns = Returns()
    pipe.add(returns,'returns')
    
    
    
        
    pipe2= attach_pipeline(Pipeline(), 'my_pipeline')

    my_factor = Returns(mask=Q1500US())
    pipe2.set_screen(Q1500US())
    pipe2.add(my_factor, 'my_pipeline')
    
    
    
    schedule_function(func=print_fama_french, date_rule=date_rules.every_day())
    schedule_function(my_rebalance, date_rules.week_start())
    schedule_function(func=open_positions,date_rule=date_rules.every_day(),time_rule=time_rules.market_open(hours=0,minutes=1))
    
    
def open_positions(context,data):
    long_list=context.longs
    log.info(context.longs)
    if len(long_list)==0:
        return
    port = context.portfolio.positions

    for sec in port:
        if port not in long_list and data.can_trade(sec):
            order_target_percent(sec,0)
    for sec in long_list:
        if data.can_trade(sec):
           order_target_percent(sec,1.0/len(long_list))

def print_fama_french(context, data):
    # print the Fama-French factors for the period defined by business_days
    # ending on the previous trading day.
    date_index.append(get_datetime())
    ff_result.append([context.rm_rf, context.smb, context.hml])
    
def before_trading_start(context,data):
    """
    every trading day, we use our pipes to construct the Fama-French
    portfolios, and then calculate the Fama-French factors appropriately.
    """
    spy = sid(8554)
    
    factors = pipeline_output('ff_example')
    context.stocks=factors.index
    # get the data we're going to use
    returns = factors['returns']
    mkt_cap = factors.sort(['market_cap'], ascending=True)
    be_me = factors.sort(['be_me'], ascending=True)
    
    # to compose the six portfolios, split our universe into portions
    half = int(len(mkt_cap)*0.5)
    small_caps = mkt_cap[:half]
    big_caps = mkt_cap[half:]
    
    thirty = int(len(be_me)*0.3)
    seventy = int(len(be_me)*0.7)
    growth = be_me[:thirty]
    neutral = be_me[thirty:seventy]
    value = be_me[seventy:]
    
    # now use the portions to construct the portfolios.
    # note: these portfolios are just lists (indices) of equities
    small_value = small_caps.index.intersection(value.index)
    small_neutral = small_caps.index.intersection(neutral.index)
    small_growth = small_caps.index.intersection(growth.index)
    
    big_value = big_caps.index.intersection(value.index)
    big_neutral = big_caps.index.intersection(neutral.index)
    big_growth = big_caps.index.intersection(growth.index)
    
    # take the mean to get the portfolio return, assuming uniform
    # allocation to its constituent equities.
    sv = returns[small_value].mean()
    sn = returns[small_neutral].mean()
    sg = returns[small_growth].mean()
    
    bv = returns[big_value].mean()
    bn = returns[big_neutral].mean()
    bg = returns[big_growth].mean()
    
    # computing Rm-Rf (Market Returns - Risk-Free Returns). we take the 
    # rate of risk-free returns to be zero, so this is simply SPY's returns.
    # have to set an initial dummy value
    context.rm_rf = float('nan')
    if spy in returns.index:
        context.rm_rf = returns.loc[spy]
    
    # computing SMB
    context.smb = (sv + sn + sg)/3 - (bv + bn + bg)/3
    
    # computing HML
    context.hml = (sv + bv)/2 - (sg + bg)/2
    
    # print([context.rm_rf,context.smb,context.hml])
    # ff_result.append([context.rm_rf,context.smb,context.hml])
    
    
    
def my_rebalance(context,data):
    stocks_longs=[]
    ff_df=pd.DataFrame(ff_result,index=date_index,columns=['rf','smb','hml'])

    my_pipe=pipeline_output('my_pipeline')
    his_data=data.history(my_pipe.index,'price',len(date_index),'1d')
    df4=np.diff(np.log(his_data),axis=0)+0*his_data[1:]

    # print(ff_df[1:].shape)
    # print(df4.shape)
    
    stock_alpha={}
    for sec in df4.columns:
        t_r=linreg(ff_df[1:],df4[sec])
        if(np.isnan(t_r[0])):
            return
        stock_alpha[sec]=[t_r[0]]
    for key, value in sorted(stock_alpha.iteritems(), key=lambda (k,v): (v,k)):
        # print ("%s: %s" % (key, value))
        stocks_longs.append(key)
    # scores=pd.DataFrame(stock_alpha)
    # print(scores.head())
    # print(stocks_longs[0:15])
    context.longs=stocks_longs[0:15]
# #consider kdj indicator    
#     tradable_position = []
#     for sec in stocks_longs:
#         price_history = data.history(
#             sec,
#             fields=['close','high','low'],
#             bar_count=120,
#             frequency='1d'
#         )
#         if price_history.iloc[0][0] is None:
#             continue
#         rsv = np.arange(0,12,0.1)
#         for j in range(119,10,-1):
#             lowest = 1000.0
#             highest = -1.0
#             for i in range(j,j-9,-1) :
#                 if price_history.iloc[i][2] < lowest:
#                     lowest = price_history.iloc[i][2]
#                 if price_history.iloc[i][1] > highest:
#                     highest = price_history.iloc[i][1]
#             rsv[j] = 100*(price_history.iloc[j][0] - lowest) / (highest - lowest)
#         kline = np.arange(0,12,0.1)
#         for i in range(11,120,1):
#             if i == 11:
#                 kline[i] = (rsv[i]+rsv[i-1]+rsv[i-2])/3
#             else :
#                 kline[i] = rsv[i]/3 + 2*kline[i-1]/3
    
#         dline = np.arange(0,12,0.1)
#         for i in range(11,120,1):
#             if i == 11:
#                 dline[i] = (kline[i]+kline[i-1]+kline[i-2])/3
#             else :
#                 dline[i] = kline[i]/3 + 2*dline[i-1]/3

#         if data.can_trade(sec) and kline[119] < dline[119]+10 and kline[119] > dline[119]-10 and kline[119] >= kline[117] and kline[117] < dline[117] and dline[119] <= 40:
#             tradable_position.append(sec)
#     if len(tradable_position) != 0:
#         weights = 1.0/len(tradable_position)
#         for sec in tradable_position:
#                 order_target_percent(sec, weights)
#                 log.info(str(sec)+' '+str(weights))
#     log.info(len(tradable_position))
#     log.info('stock value: '+str(context.portfolio.positions_value))
#     log.info('cash: '+str(context.portfolio.cash))
#     log.info(len(my_positions))
#     context.longs=tradable_position[0:15]
    
    
    
    
    
    
    
    
    
    
    
    # print(context.longs)
    
    # print(his_data.head())
    
    
    # print(ff_df.head())
    
def linreg(X,Y,columns=3):
    X=sm.add_constant(np.array(X))
    Y=np.array(Y)
    if len(Y)>business_days:
        results = regression.linear_model.OLS(Y, X).fit()
        return results.params
    else:
        return [float("nan")]*(columns+1)    
There was a runtime error.

@lifan, I believe this algorithm is just to demonstrate how to generate Fama French factors. Which are a common tool in risk management, but aren't used to forecast returns and produce alpha. See these lectures for more info:

Defining factor risk
https://www.quantopian.com/lectures/factor-risk-exposure
How to use factor risk data to reduce risk
https://www.quantopian.com/lectures/risk-constrained-portfolio-optimization
How to determine if you have alpha
https://www.quantopian.com/lectures/factor-analysis

Can somebody link a good lecture to understanding Fama French Factors? I've found lectures linked on Quantopian to be easy to understand. Thanks.

Hi, I do not find the same results as mentioned, that are 1.53, -4.67, -4.33 for instance for july 2015, what would be a possible explanation?

Especially the HmL factor, which is completely different to data from French website

Many thanks

@Clément Felley, experiencing the same problem.

I checked through the back-tested results of both the blog code, and the cloned version which I ran, the benchmark return values are off by a couple of % even though the time frames are the same. Can that be a possible reason? The underlying data for the companies has been updated or something? (because the migrated code section mentions the prices are now adjusted) Or is there something we are doing wrong here? (Moreover, fama french values for other time frames don't really match up either). Please do inform

Thanks in advance.

Clone Algorithm
6
Loading...
Backtest from to with initial capital
Total Returns
--
Alpha
--
Beta
--
Sharpe
--
Sortino
--
Max Drawdown
--
Benchmark Returns
--
Volatility
--
Returns 1 Month 3 Month 6 Month 12 Month
Alpha 1 Month 3 Month 6 Month 12 Month
Beta 1 Month 3 Month 6 Month 12 Month
Sharpe 1 Month 3 Month 6 Month 12 Month
Sortino 1 Month 3 Month 6 Month 12 Month
Volatility 1 Month 3 Month 6 Month 12 Month
Max Drawdown 1 Month 3 Month 6 Month 12 Month
# 2015-06-01 to 2015-10-01

import pandas as pd
import numpy as np
from quantopian.algorithm import attach_pipeline, pipeline_output
from quantopian.pipeline import Pipeline
from quantopian.pipeline import CustomFactor
from quantopian.pipeline.data.builtin import USEquityPricing
from quantopian.pipeline.data import morningstar

# time frame on which we want to compute Fama-French
normal_days =31
# approximate the number of trading days in that period
# this is the number of trading days we'll look back on,
# on every trading day.
business_days = int(0.69 * normal_days)

class Returns(CustomFactor):
    """
    this factor outputs the returns over the period defined by 
    business_days, ending on the previous trading day, for every security.
    """
    window_length = business_days
    inputs = [USEquityPricing.close]
    def compute(self,today,assets,out,price):
        out[:] = (price[-1] - price[0]) / price[0] * 100

class MarketEquity(CustomFactor):
    """
    this factor outputs the market cap of every security on the day.
    """
    window_length = business_days
    inputs = [morningstar.valuation.market_cap]
    def compute(self,today,assets,out,mcap):
        out[:] = mcap[0]

class BookEquity(CustomFactor):
    """
    this factor outputs the book value of every security on the day.
    """
    window_length = business_days
    inputs = [morningstar.balance_sheet.tangible_book_value]
    def compute(self,today,assets,out,book):
        out[:] = book[0]
                                        
class CommonStock(CustomFactor):
    """
    this factor outputs 1.0 for all securities that are either common stock or SPY,
    and outputs 0.0 for all other securities. This is to filter out ETFs and other
    types of share that we do not wish to consider.
    """
    window_length = business_days
    inputs = [morningstar.share_class_reference.is_primary_share]
    def compute(self,today,assets,out, share_class):
        out[:] = ((share_class[-1].astype(bool)) | (assets == 8554)).astype(float)                                     
        
def initialize(context):
    """
    use our factors to add our pipes and screens.
    """
    pipe = Pipeline()
    attach_pipeline(pipe, 'ff_example')
    
    common_stock = CommonStock()
    # filter down to securities that are either common stock or SPY
    pipe.set_screen(common_stock.eq(1))
    mkt_cap = MarketEquity()
    pipe.add(mkt_cap,'market_cap')
    
    book_equity = BookEquity()
    # book equity over market equity
    be_me = book_equity/mkt_cap
    pipe.add(be_me,'be_me')

    returns = Returns()
    pipe.add(returns,'returns')
    schedule_function(func=print_fama_french, date_rule=date_rules.every_day())

def print_fama_french(context, data):
    # print the Fama-French factors for the period defined by business_days
    # ending on the previous trading day.
    print(context.rm_rf, context.smb, context.hml)
    
def before_trading_start(context,data):
    """
    every trading day, we use our pipes to construct the Fama-French
    portfolios, and then calculate the Fama-French factors appropriately.
    """
    spy = sid(8554)
    
    factors = pipeline_output('ff_example')
    
    # get the data we're going to use
    returns = factors['returns']
    mkt_cap = factors.sort(['market_cap'], ascending=True)
    be_me = factors.sort(['be_me'], ascending=True)
    
    # to compose the six portfolios, split our universe into portions
    half = int(len(mkt_cap)*0.5)
    small_caps = mkt_cap[:half]
    big_caps = mkt_cap[half:]
    
    thirty = int(len(be_me)*0.3)
    seventy = int(len(be_me)*0.7)
    growth = be_me[:thirty]
    neutral = be_me[thirty:seventy]
    value = be_me[seventy:]
    
    # now use the portions to construct the portfolios.
    # note: these portfolios are just lists (indices) of equities
    small_value = small_caps.index.intersection(value.index)
    small_neutral = small_caps.index.intersection(neutral.index)
    small_growth = small_caps.index.intersection(growth.index)
    
    big_value = big_caps.index.intersection(value.index)
    big_neutral = big_caps.index.intersection(neutral.index)
    big_growth = big_caps.index.intersection(growth.index)
    
    # take the mean to get the portfolio return, assuming uniform
    # allocation to its constituent equities.
    sv = returns[small_value].mean()
    sn = returns[small_neutral].mean()
    sg = returns[small_growth].mean()
    
    bv = returns[big_value].mean()
    bn = returns[big_neutral].mean()
    bg = returns[big_growth].mean()
    
    # computing Rm-Rf (Market Returns - Risk-Free Returns). we take the 
    # rate of risk-free returns to be zero, so this is simply SPY's returns.
    # have to set an initial dummy value
    context.rm_rf = float('nan')
    if spy in returns.index:
        context.rm_rf = returns.loc[spy]
    
    # computing SMB
    context.smb = (sv + sn + sg)/3 - (bv + bn + bg)/3
    
    # computing HML
    context.hml = (sv + bv)/2 - (sg + bg)/2
There was a runtime error.

This is quite interesting, still wondering though, is there a way to constraint Fama-French values for specific equity exchanges?