Quantopian Risk Model In Algorithms

A few weeks ago, we introduced the Quantopian Risk Model to make it easier to build strategies that are suitable for capital allocations. At the time of launch, the Quantopian risk model output was available in research for performance attribution. Performance attribution is a great tool for understanding your risk exposure after an algorithm has completed, but because it's done after the fact, it doesn't assist an algorithm with dynamically managing its risk exposure as it runs. Today, we're introducing new tools to help manage your risk in algorithms.

#### How Do I Access Risk Model Data?

The easiest way to access risk model data in an algorithm is to use risk_loading_pipeline, a new function we've added to the quantopian.pipeline.experimental module. risk_loading_pipeline constructs a pre-configured Pipeline that produces a column of output for each factor in the Quantopian Risk Model.

The pipeline created by risk_loading_pipeline works just like any other pipeline. Here's a simple example of how you can use risk_loading_pipeline in an algorithm:

from quantopian.pipeline.experimental import risk_loading_pipeline

def initialize(context):
attach_pipeline(risk_loading_pipeline(), 'risk_loading_pipeline')

def before_trading_start(context, data):
context.risk_loading_pipeline = pipeline_output('risk_loading_pipeline')
# Multiple pipelines are supported in an algorithm.
context.my_other_pipeline = pipeline_output('my_other_pipeline')

def my_method(context, data):
# Access context.risk_loading_pipeline in your methods.


The value of context.risk_loading_pipeline in the example is a DataFrame containing a row for every asset and a column for every output of the pipeline. Since risk_loading_pipeline has an output for each column of the Quantopian Risk Model, its result will look like this:

The risk_loading_pipeline is a convenient way to access all the common risk factors in one go. You can also access individual risk factors, like in this example which accesses only the Size factor:

from quantopian.pipeline import Pipeline
from quantopian.pipeline.experimental import Size

def initialize(context):
pipe = Pipeline()
pipe.add(Size(), 'size')
attach_pipeline(pipe, 'size_factor_pipeline')

def before_trading_start(context, data):
context.size_factor_pipeline = pipeline_output('size_factor_pipeline')

def my_method(context, data):
# Access context.size_factor_pipeline in your methods.


Since you can access multiple pipelines in an algorithm, you can use the risk pipelines side-by-side with any pipelines that your algorithm has already defined. The risk model pipeline is also accessible in the research environment via run_pipeline:

from quantopian.pipeline.experimental import risk_loading_pipeline
from quantopian.research import run_pipeline
run_pipeline(risk_loading_pipeline(), '2014-01-01', '2014-01-02')


#### How Can I Use This Data?

One of the main use cases is to keep your common risk exposure within certain bounds, perhaps in order to meet the contest entry criteria. You can do this by passing a constraint to order_optimal_portfolio, to help place orders such that your common risk exposure thresholds are not exceeded. The snippet shows an example of how you'd constrain your portfolio exposure to common risk:

from quantopian.pipeline.experimental import risk_loading_pipeline
import quantopian.optimize as opt

def initialize(context):
attach_pipeline(risk_loading_pipeline(), 'risk_loading_pipeline')

def before_trading_start(context, data):
context.risk_loading_pipeline = pipeline_output('risk_loading_pipeline')

def place_orders(context, data):
# Constrain our risk exposures. We're using the latest version of the default bounds,
# except for momentum, where we've overridden the default and set a bound of 10% on both
# longs and shorts.
constrain_sector_style_risk = opt.experimental.RiskModelExposure(
risk_model_loadings=context.risk_loading_pipeline,
version=opt.Newest,
min_momentum=-0.1,
max_momentum=0.1,
)

order_optimal_portfolio(
objective=some_objective,
constraints=[constrain_sector_style_risk],
)


By default, the RiskModelExposure constraint will constrain the sector exposures to 18%, and the style exposures to 36%. These are both slightly under the contest entry criteria of 20% and 40% respectively. These default values may change in the future (for example, in response to a change in the contest entry criteria). If you'd always like to use in the latest version of the defaults, pass in quantopian.optimize.Newest as the version, and your algorithm's behavior will automatically update to match the new default behavior. On the other hand, if you'd like to ensure that your algorithm's behavior doesn't change when the defaults change, you can pass version=0, and your algorithm will always use the version zero defaults. Right now, version 0 is the only version of the defaults available, but we may add more.

You can also override some (or all!) of the defaults if you'd like to change your exposure limits to various factors. The snippet above shows an example where my momentum exposure is constrained to 10%, instead of the default of 36%. The documentation contains the full list of parameters that you can pass in to RiskModelExposure.

#### What Else Can I Do With This?

We're looking forward to working that out together! While performance attribution was a tool we had been using internally for some time, the risk data in algorithms is a new feature to both of us. Optimization constraints are a good first step (particularly to help meet the contest entry criteria), but we'd like to work together on:

• Exploring use of the risk data in alpha generation and combination. While the risk model by itself is not an alpha model, there may be interesting approaches in 1) combining the risk factors with your existing alpha factors, 2) finding a combination of the risk factors that is an alpha factor, or 3) using the risk factors to design a factor-neutral pure alpha strategy.
• Refining best practices for using the risk data in optimization constraints.

In the weeks to come we'll be posting more examples of ways to use the risk data in algorithms, along with more lectures and educational material about how to use the risk model across the platform.

#### Putting It All Together

The attached backtest is an example of a full-featured algorithm using the risk model. Thanks to the risk constraints, it now meets the new contest entry criteria.

619
Loading...
Total Returns
--
Alpha
--
Beta
--
Sharpe
--
Sortino
--
Max Drawdown
--
Benchmark Returns
--
Volatility
--
 Returns 1 Month 3 Month 6 Month 12 Month
 Alpha 1 Month 3 Month 6 Month 12 Month
 Beta 1 Month 3 Month 6 Month 12 Month
 Sharpe 1 Month 3 Month 6 Month 12 Month
 Sortino 1 Month 3 Month 6 Month 12 Month
 Volatility 1 Month 3 Month 6 Month 12 Month
 Max Drawdown 1 Month 3 Month 6 Month 12 Month
import pandas as pd

import quantopian.algorithm as algo
import quantopian.optimize as opt

from quantopian.pipeline import Pipeline
from quantopian.pipeline.data import builtin, Fundamentals, psychsignal
from quantopian.pipeline.factors import AverageDollarVolume, RollingLinearRegressionOfReturns
from quantopian.pipeline.factors.fundamentals import MarketCap
from quantopian.pipeline.classifiers.fundamentals import Sector
from quantopian.pipeline.experimental import QTradableStocksUS, risk_loading_pipeline

# Algorithm Parameters
# --------------------
UNIVERSE_SIZE = 1000
LIQUIDITY_LOOKBACK_LENGTH = 100

MINUTES_AFTER_OPEN_TO_TRADE = 5

MAX_GROSS_LEVERAGE = 1.0
MAX_SHORT_POSITION_SIZE = 0.01  # 1%
MAX_LONG_POSITION_SIZE = 0.01   # 1%

def initialize(context):
# Universe Selection
# ------------------
base_universe = QTradableStocksUS()

# From what remains, each month, take the top UNIVERSE_SIZE stocks by average dollar
# volume traded.
monthly_top_volume = (
AverageDollarVolume(window_length=LIQUIDITY_LOOKBACK_LENGTH)
.top(UNIVERSE_SIZE, mask=base_universe)
.downsample('week_start')
)
# The final universe is the monthly top volume &-ed with the original base universe.
# &-ing these is necessary because the top volume universe is calculated at the start
# of each month, and an asset might fall out of the base universe during that month.
universe = monthly_top_volume & base_universe

# Alpha Generation
# ----------------
# Compute Z-scores of free cash flow yield and earnings yield.
# Both of these are fundamental value measures.
fcf_zscore = Fundamentals.fcf_yield.latest.zscore(mask=universe)
yield_zscore = Fundamentals.earning_yield.latest.zscore(mask=universe)
sentiment_zscore = psychsignal.stocktwits.bull_minus_bear.latest.zscore(mask=universe)

# Alpha Combination
# -----------------
# Assign every asset a combined rank and center the values at 0.
# For UNIVERSE_SIZE=500, the range of values should be roughly -250 to 250.
combined_alpha = (fcf_zscore + yield_zscore + sentiment_zscore).rank().demean()

beta = 0.66*RollingLinearRegressionOfReturns(
target=sid(8554),
returns_length=5,
regression_length=260,
mask=combined_alpha.notnull() & Sector().notnull()
).beta + 0.33*1.0

# Schedule Tasks
# --------------
# Create and register a pipeline computing our combined alpha and a sector
# code for every stock in our universe. We'll use these values in our
# optimization below.
pipe = Pipeline(
columns={
'alpha': combined_alpha,
'sector': Sector(),
'sentiment': sentiment_zscore,
'beta': beta,
},
# combined_alpha will be NaN for all stocks not in our universe,
# but we also want to make sure that we have a sector code for everything
# we trade.
screen=combined_alpha.notnull() & Sector().notnull() & beta.notnull(),
)

# Multiple pipelines can be used in a single algorithm.
algo.attach_pipeline(pipe, 'pipe')
algo.attach_pipeline(risk_loading_pipeline(), 'risk_loading_pipeline')

# Schedule a function, 'do_portfolio_construction', to run twice a week
# ten minutes after market open.
algo.schedule_function(
do_portfolio_construction,
date_rule=algo.date_rules.week_start(),
time_rule=algo.time_rules.market_open(minutes=MINUTES_AFTER_OPEN_TO_TRADE),
half_days=False,
)

def before_trading_start(context, data):
# Call pipeline_output in before_trading_start so that pipeline
# computations happen in the 5 minute timeout of BTS instead of the 1
# minute timeout of handle_data/scheduled functions.
context.pipeline_data = algo.pipeline_output('pipe')
context.risk_loading_pipeline = algo.pipeline_output('risk_loading_pipeline')

# Portfolio Construction
# ----------------------
def do_portfolio_construction(context, data):
pipeline_data = context.pipeline_data

# Objective
# ---------
# For our objective, we simply use our naive ranks as an alpha coefficient
# and try to maximize that alpha.
#
# This is a **very** naive model. Since our alphas are so widely spread out,
# we should expect to always allocate the maximum amount of long/short
# capital to assets with high/low ranks.
#
# A more sophisticated model would apply some re-scaling here to try to generate
# more meaningful predictions of future returns.
objective = opt.MaximizeAlpha(pipeline_data.alpha)

# Constraints
# -----------
# Constrain our gross leverage to 1.0 or less. This means that the absolute
# value of our long and short positions should not exceed the value of our
# portfolio.
constrain_gross_leverage = opt.MaxGrossExposure(MAX_GROSS_LEVERAGE)

# Constrain individual position size to no more than a fixed percentage
# of our portfolio. Because our alphas are so widely distributed, we
# should expect to end up hitting this max for every stock in our universe.
constrain_pos_size = opt.PositionConcentration.with_equal_bounds(
-MAX_SHORT_POSITION_SIZE,
MAX_LONG_POSITION_SIZE,
)

# Constrain ourselves to allocate the same amount of capital to
# long and short positions.
market_neutral = opt.DollarNeutral()

# Constrain beta-to-SPY to remain under the contest criteria.
beta_neutral = opt.FactorExposure(
pipeline_data[['beta']],
min_exposures={'beta': -0.05},
max_exposures={'beta': 0.05},
)

# Constrain exposure to common sector and style risk
# factors, using the latest default values. At the time
# of writing, those are +-0.18 for sector and +-0.36 for
# style.
constrain_sector_style_risk = opt.experimental.RiskModelExposure(
context.risk_loading_pipeline,
version=opt.Newest,
)

# Run the optimization. This will calculate new portfolio weights and
# manage moving our portfolio toward the target.
algo.order_optimal_portfolio(
objective=objective,
constraints=[
constrain_gross_leverage,
constrain_pos_size,
market_neutral,
constrain_sector_style_risk,
beta_neutral,
],
)
There was a runtime error.
Disclaimer

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.

29 responses

Wow. Simply attach this risk factor pipeline and then add the output as a constraint to the order method. The resulting algorithm should magically meet all the exposure requirements of the contest. Nice.

This is cool. But why isn't the beta to market a built in factor for the risk model?

@ Burrito Dan: the Q risk model does not involve the market factor is due to its factor model structure. To select factors, we need to choose the common factors which are uncorrelated with each other as much as possible while the combination of selected factors can represent the returns of most assets in the market as much as possible. We think the current setting of common factors was our best choice to achieve that goal without making the process of calculation for the risk model too complicated. The model is never done, and we have several changes and improvements in mind to be tested and made in the future.

Disclaimer

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.

@Rene Isn't the market factor orthogonal to the common factors, by construction?

@Burrito Dan: The market factor is highly correlated with most of the sector factors.

Is this re-balancing twice a week or everyday?

This is definitely a step-up for quantopian. Great job, Q team!

@Abhijeet: Can you expand on this point, "2) finding a combination of the risk factors that is an alpha factor" and perhaps gives examples of all 3 bullets?

The market factor is highly correlated with most of the sector factors.

@ Rene -

If the market factor is highly correlated with the sector factors, then what is the point of having the sector factors in the first place? This makes it sound like the sectors aren't all that distinct from the overall market, and as such, aren't really strong risk factors (i.e. diversification across sectors doesn't really achieve anything). I guess you'll flesh this out in the Quantopian Risk Model white paper you are working on; if so, no need to reply here.

@Abhi @Rene: I have some perplexing results from constraining 2 factors: momentum and short_term_reversal.

How I implemented the method to use the risk model data from Pipeline to Optimize API:

# In myPipeline(), the risk factors are added:

myPipeline.add(Momentum(), 'momentum')
myPipeline.add(ShortTermReversal(), 'short_term_reversal')

# In before_trading_start(), the risk factor values are assigned to context.riskData:

context.riskData = pipeline_output('myPipeline')

# In Optimize API, the 2 factor exposure limits are set to override the opt.Newest defaults:

momentumExposure = 0.4  # Testing range 0.3~0.6
reversalExposure = 0.4  # Testing range 0.2~0.5

constrainStyleRiskFactors = opt.experimental.RiskModelExposure(
risk_model_loadings = context.riskData,
version = opt.Newest,
min_momentum = -momentumExposure,
max_momentum = momentumExposure,
min_short_term_reversal = -reversalExposure,
max_short_term_reversal = reversalExposure
)

constraints.append(constrainStyleRiskFactors)


The results show that momentum risk is reasonably well controlled to set limits - I have also tested a range from 0.3~0.6 to be sure.

Not so for short_term_reversal, whereas the Exposure Summary table reports 0.53, the Daily Style Factor Exposures chart shows roughly +0.25 risks above 0.53 not constrained, plus a few spikes into the 0.9+ zone.

I wonder what could be amiss.. why the high +0.25 and spikes that could not be constrained by the  ShortTermReversal()  factor.

Possible that the  ShortTermReversal()  method was detecting that the strategy returns somehow matched the "short_term_reversal" signals, or the strategy simply could not be effectively constrained by the method used otherwise for common short_term_reversal trades?

I'm ok to send a tear sheet to show the results if you are able to help. Thanks

@ Karl : The short-term reversal factor is a faster moving factor (14-day window length) comparing to other risk factors. If you find a strategy has large exposure in the short-term reversal on the performance attribution tearsheet, I suggest you to consider changing it from the design of this algo. If you want to use the optimization tool to bound short-term reversal factor exposure, you probably need to rebalance algo’s positions frequently. Note: Using optimization may change an algo's behaviors in an expected (proper) way, but it is also possible to change an algo's behaviors in an unexpected (improper) way. Observing the performance attribution notebooks and using the optimization tools on the research platform can help to determine whether the algo is running as designed.

Don't know if I can post the following here. So, taking a chance. If anyone objects, I will take it off. If Q objects, they can delete it too since again it does reference the strategy at the top of this thread.

Took the above Risk Model as starting point. Wanted to see if I could extract more from it.

None of the programming logic was changed, only fixed variables that in a way dictate how the strategy will behave.

Increased selectable candidates, changed the time of day of the schedule_function to a more subdued time period (less variance), increased the bet size (less trades), put in a small leverage fraction (1.07). Eliminated the stocktwits thingy. I do not think it has any value whatsoever. This way, remaining consistent with my views. Increased the impact of the fcf_zscore (gave it dominance) for a very simple fundamental reason: a company having more free cash will tend to invest more in its own future. Reduced the beta impact, giving it more of a flat line (damped variance swings). Increased the allowed beta spread, reason is simple there too: increasing it allowed more price variance to enable more fringe trades. Each step increasing overall performance a notch.

In all, just numbers that control the ever moving swarm of price variances. See the attached tear sheets.

The strategy is now in a position where it might even win a contest with its 16.9% CAGR, while keeping its beta, volatility and drawdowns relatively low.

But, in reality, going forward, the strategy would fail in other regards. It has very little real alpha. The strategy is linear and therefore would also degrade going forward, not because it is a bad strategy, but because it does not compensate for growth degradation. It is definitely subject to the law of diminishing returns, and this, by design. To do better, it will have to take care of that limitation. There are solutions to that inherent problem. Note that any linear strategy is subject to this return degradation.

Just removing the stocktwits influence increase performance, and in itself does answer the question of its value. It also reduced the number of trades by more than half. More profits, less expenses, how could I refuse. It also confirms my evaluation of it.

Even if the tear sheet says there is alpha, looking at the cumulative return on the logarithmic scale show there is almost none. You do get a return that is above but close to the market average, however, calling it alpha might be an exaggeration.

This does not mean that the strategy is worthless. Because of its low beta and low volatility, it could be combined with more volatile strategies to reduce overall variance and volatility. It might even hold its own after compensating for the return degradation.

The profits seems to come mainly from having extended the holding period by a couple of weeks thereby benefiting from the underlying market trend. There are no surprises here. But still, no real sustainable alpha either. It is by design that this strategy will produce less and less return-wise going forward.

The strategy is scalable to some extent which is a big thing (tested up to 4x, could go a bit more). It could give some control over a set of more volatile strategies since volatility gets averaged out. It is not so good when leveraging since it would barely cover its borrowing costs.

For the moment, that is about all I could extract without changing the program's logic which would in essence change its very nature.

My note is to show how much impact just changing variables (outside of the program's logic) can improve overall performance. As if slightly reshaping this swarm of variance around its center of mass was sufficient to almost triple its performance as it rolled down history.

32
Loading notebook preview...

@evgeny We'd like this thread to be about the risk model usage in general. Can you please delete your reply, and repost it as a new stand-alone thread? Thank you!

Disclaimer

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.

@Guy, Great analysis and modifications. Can you please post exact values of the variable modifications so I can reproduce it? I tried based on your narrative but coud not reproduce it. I would like to try your suggestion:

Because of its low beta and low volatility, it could be combined with more volatile strategies to reduce overall variance and volatility. It might even hold its own after compensating for the return degradation.

This study also confirms my previous analysis that the beta constraints has no effect to the final portfolio beta of uncorrelated returns. So in effect, it is not needed to have beta constraints just the requirement that the portfolio beta be between -0.3 and 0.3.

Hi everyone,

This might be a rather noobish question, but I'm new to Quantopian so please bear with me.

So I was playing around with this risk model as shown in Abhijeet's article above. I had an algorithm that was designed to allocate 100% of the portfolio into a long position based on certain criteria. When I added the risk_loading_pipeline and the constrain_sector_style_risk to the constraints in the order_optimal_portfolio command, I notice the algorithm began shorting. I realize it's probably doing so in an effort to hedge and hence maintain the Sector and Style exposures under constraints, but how is it deciding which short positions to take? There are no short positions in my Target Weights that I'm telling it to order.

Some insight into how it's shorting and how this risk model makes decisions in general would be very helpful!

@Nishanth

If you are using the TargetWeights objective then the algorithm will try to find weights that minimize the 'distance' from the specified weights, while at the same time satisfying all the constraints. Conceptually, the optimize function loops through all possible permutations of weights for all securities. Possible weights can be either positive or negative corresponding to long or short the security. The distance is simply the square root of the sum of the squares of the differences from the specified target weights (ie Euclidean distance). Optimize chooses the set of weights that result in the smallest distance while also meeting all the provided constraints.

As a simple example showing why optimize would ever short a security, consider a universe of only AAPL and IBM. These are both in the tech sector. If the target weight for each is .5 (ie both long), the default risk factor constraint will try to limit tech sector exposure to +-18%. Ordering all long of each will result in a 100% sector exposure. Therefore, the automagic optimize method will modify the weights to get the exposure within the constraints. So, it shorts one of them to achieve this. Probably a toss up which will be shorted if this were the only constraint. However, the other risk constraints will typically kick in and decide.

Hope that helps.

@Nishanth -

You should be able to get a long-only portfolio with:

constraints.append(
opt.PositionConcentration.with_equal_bounds(
min=0,
max=MAX_POSITION_SIZE
))


Set MAX_POSITION _SIZE > 0.

In respect of John Loeber's Fame-French post and presentation slides, is it possible to use the Quantopian Risk Model to define and constrain Fame-French Factors risks?

Thanks

@Karl good question. Fundamentally, the Q risk model is not same with the Fama-French based model. However, I think you could try to just constrain sector factors, value factor, size factor with Q risk model to achieve similar results.

Hi Karl,

If you want to constrain exposure to some of the risk factors, but leave others unconstrained, you could do it a couple of ways. Like you mentioned, FactorExposure is one possible approach - if you want to constrain just a few of the risk factor exposures, and leave most of them unconstrained, it's a good bet.

However, if you want to constrain most of the risk factor exposures, and leave just a few unconstrained, you could save a few lines of code with a combination of opt.NotConstrained and RiskModelExposure. For example, if you wanted to constrain all of the sector factors, along with size and value, that works out to 'constrain everything except momentum, short_term_reversal, and volatility'. One way to write that is:

opt.experimental.RiskModelExposure(
risk_model_loadings=context.risk_loading_pipeline,
version=opt.Newest,
min_momentum=opt.NotConstrained,
max_momentum=opt.NotConstrained,
min_short_term_reversal=opt.NotConstrained,
max_short_term_reversal=opt.NotConstrained,
min_volatility=opt.NotConstrained,
max_volatility=opt.NotConstrained,
)


@Karl, interesting findings and the "Less is More" is befitting the moniker! Consistent with my findings at the Optimize API constraints about the redundancy of beta constraint with dollar neutral constraint also has the "Less is More" theme.

Forgot to tell you, it's also "vintage", LOL!

The algo above might be the best starting point right now in my opinion. If recreation is one's goal here, you can skip this message. For those who want to succeed and make money, sorry to sound dramatic but you'd be wise to pay careful attention to this and dig into it, spend some time experimenting with this code presented, to understand rather than brushing this aside as we humans tend to do as we coast along. People don't like bad news so just imagine what amazingly great news it is going to be, once you resolve this discomfort standing in our way ...

The algorithm above looks healthy in the chart for those ~4 years and yet if started on its posted date to today is down -2.27%.
This could be why. The fundamentals might not be as reliable as we would like to think ...

At Upcoming Changes to Morningstar Fundamental Data, Doug Baldwin did some detailed work investigating the update frequency on fundamentals. He found that uncertainties are pretty much beyond the pale. The notebook contains more information. We would have to write quite a bit more code to work around the troubles noted, these things are outside of Quantopian's control, it's based on the data feed.

At Fundamentals updating daily vs monthly or quarterly my backtest for profiling fundamental updates shows for fcf_yield and earning_yield used by the backtest above, the number of value changes are distributed widely (unlike some others quite a bit more consistent). And what does that mean. Most companies report Fundamentals.cash once per year. Is fcf (Free Cash Flow) yield based on share price in relation to Fundamentals.cash at last report? If that's the case I would expect them all to change every day. Are the inconsistencies in data updating or company reporting, both or neither? The answers will have to come from bigger brains than mine.

Hi Blue Seahawk,

My understanding, and also confirmed by the fundamental data source page, is that:

fcf_yield = fcf per share / price
earning_yield = EPS / price [the inverse of P/E)

Since prices update daily, these values will always update daily as well. The numerator only update quarterly however, when companies release their 10Qs (quarterly and non-audited I believe) and 10Ks (annual and audited).

FCF comes from the cashflow statement, and is the cash generated from operations, after all capital expenditures have been paid for. Cash is a balance-sheet entry on the asset side :), and is not directly related to FCF unless the company retains all FCF on the books as cash. Increase/growth in both are a good and healthy sign in my book.

Just my 2 SEK worth, hope this helps.

Joakim

Appreciated. I wonder how often an increase in 10Q fcf vs previous quarter translates to price increases (from value investors) or from my type looking to anticipate what they will be doing. :) I added some more data on fundamentals here.

Looks good with the original date range, nearly 3x. If anyone can find a way to avoid the Mar 2016 downturn, would be great.

129
Loading...
Total Returns
--
Alpha
--
Beta
--
Sharpe
--
Sortino
--
Max Drawdown
--
Benchmark Returns
--
Volatility
--
 Returns 1 Month 3 Month 6 Month 12 Month
 Alpha 1 Month 3 Month 6 Month 12 Month
 Beta 1 Month 3 Month 6 Month 12 Month
 Sharpe 1 Month 3 Month 6 Month 12 Month
 Sortino 1 Month 3 Month 6 Month 12 Month
 Volatility 1 Month 3 Month 6 Month 12 Month
 Max Drawdown 1 Month 3 Month 6 Month 12 Month
'''
https://www.quantopian.com/posts/quantopian-risk-model-in-algorithms
'''

import numpy as np
import quantopian.optimize  as opt
import quantopian.algorithm as algo
from quantopian.pipeline import Pipeline
from quantopian.pipeline.data    import builtin, Fundamentals
from quantopian.pipeline.factors import AverageDollarVolume, RollingLinearRegressionOfReturns
from quantopian.pipeline.factors import CustomFactor
from quantopian.pipeline.filters import QTradableStocksUS
from quantopian.pipeline.classifiers.fundamentals import Sector
from quantopian.pipeline.experimental import risk_loading_pipeline

def initialize(context):
log.info('{}    {} to {}'.format(int(context.portfolio.starting_cash),
get_environment('start').date(), get_environment('end').date()))

#schedule_function(trade, date_rules.week_start(), time_rules.market_open(minutes=5))
#schedule_function(trade, date_rules.every_day(), time_rules.market_open(minutes=5))

m  = QTradableStocksUS()  # adding to mask on next line with &=
m &= ( AverageDollarVolume(window_length=100).top(500, mask=m).downsample('week_start') )

a      = Fundamentals.fcf_yield    .latest.rank(mask=m)
b      = Fundamentals.earning_yield.latest.rank(mask=m)
# select was intended to be Min or Max rank (an experiment) but Mean worked better. Whatever.
select = Mean( inputs = [ a, b ], window_length = 1, mask=m )
alpha  = select.rank(mask=m).demean()

beta = 0.66 * RollingLinearRegressionOfReturns(target=sid(8554), returns_length=5,
regression_length=126, mask=(m & alpha.notnull() & Sector().notnull()) ).beta + 0.33

pipe = Pipeline(
columns = {
'alpha'    : alpha,
'beta'     : beta,
'fcf'      : a,
'earning'  : b,
'select'   : select,
'sector'   : Sector(),
},
screen = m & alpha.notnull() & Sector().notnull() & beta.notnull(),
)
'''  Btw, NaN here, why?
alpha      beta  earning    fcf  sector  select
Equity(239 [AIG])     36.5  1.492571    500.0    NaN     103   269.5
'''

algo.attach_pipeline(pipe, 'pipe')
algo.attach_pipeline(risk_loading_pipeline(), 'risk_pipe')

def before_trading_start(context, data):
c = context
record(lv  = c.account.leverage)
record(pos = len(c.portfolio.positions))
if get_datetime().weekday() != 1: return  # 1 is Tuesday

context.pdata     = algo.pipeline_output('pipe')
context.risk_pipe = algo.pipeline_output('risk_pipe')

if 'log_pipe_done' not in context:       # show pipe info once
log_pipe(context, data, context.pdata, 4) #, details=['fcf', 'earn', 'select'])

# Optimize can be called from here.
#   No result difference, just a bit more efficient using weekday above.
trade(context, data)

def trade(context, data):
pdata = context.pdata

algo.order_optimal_portfolio(
objective   = opt.MaximizeAlpha( pdata.alpha ),
constraints = [
opt.MaxGrossExposure(1.0),
opt.PositionConcentration.with_equal_bounds(-.035, .035),
opt.DollarNeutral(),
opt.experimental.RiskModelExposure(context.risk_pipe,version=opt.Newest),
opt.FactorExposure( pdata[['beta']],
min_exposures={'beta': -0.05}, max_exposures={'beta':  0.05}),
]
)

class Mean(CustomFactor):
def compute(self, today, assets, out, one, two):
one = nanfill(one); two = nanfill(two)
out[:] = np.mean([one, two], axis=0)

class Min(CustomFactor):
def compute(self, today, assets, out, one, two):
one = nanfill(one); two = nanfill(two)
out[:] = np.min([one, two], axis=0)

class Max(CustomFactor):
def compute(self, today, assets, out, one, two):
one = nanfill(one); two = nanfill(two)
out[:] = np.max([one, two], axis=0)

def nanfill(_in):   # Forward Fill NaNs
'''
From https://stackoverflow.com/questions/41190852/most-efficient-way-to-forward-fill-nan-values-in-numpy-array
https://www.quantopian.com/posts/forward-filling-nans-in-pipeline
'''

#return _in            # uncomment to not run the code below

'''
nan_num = np.count_nonzero(np.isnan(_in))   # count nans
if nan_num:
log.info(nan_num)
#log.info(str(_in))
'''
mask = np.isnan(_in)
idx = np.where(~mask,np.arange(mask.shape[1]),0)
np.maximum.accumulate(idx,axis=1, out=idx)
_in[mask] = _in[np.nonzero(mask)[0], idx[mask]]
return _in

def log_pipe(context, data, df, num, details=None):
''' Log info about pipeline output or any DataFrame or Series (df)
See https://www.quantopian.com/posts/overview-of-pipeline-content-easy-to-add-to-your-backtest
'''

# Options
log_nan_only = 0
show_sectors = 0
show_sorted_details = 1

if not len(df):
log.info('Empty')
return

# Series ......
context.log_pipe_done = 1 ; padmax = 6 ; content = ''
if 'Series' in str(type(df)):    # is Series, not DataFrame
nan_count = len(df[df != df])
nan_count = 'NaNs {}/{}'.format(nan_count, len(df)) if nan_count else ''
if (log_nan_only and nan_count) or not log_nan_only:
pad = max(6, len(str(df.max())))
log.info('{}{}{}   Series {}  len {}'.format('min' .rjust(pad+5),
'mean'.rjust(pad+5), 'max' .rjust(pad+5),  df.name, len(df)))
log.info('{}{}{} {}'.format(str(df.min()) .rjust(pad+5),
str(df.mean()).rjust(pad+5), str(df.max()) .rjust(pad+5), nan_count
))
return

# DataFrame ......
content_min_max = [ ['','min','mean','max',''] ]
for col in df.columns:
if col == 'sector' and not show_sectors: continue
nan_count = len(df[col][df[col] != df[col]])
nan_count = 'NaNs {}/{}'.format(nan_count, len(df)) if nan_count else ''
padmax    = max( padmax, max(6, len(str(df[col].max()))) )
content_min_max.append([col, str(df[col] .min()), str(df[col].mean()), str(df[col] .max()), nan_count])
if log_nan_only and nan_count or not log_nan_only:
content = 'Rows: {}  Columns: {}'.format(df.shape[0], df.shape[1])
if len(df.columns) == 1: content = 'Rows: {}'.format(df.shape[0])

paddings = [6 for i in range(4)]
for lst in content_min_max:    # set max lengths
i = 0
for val in lst[:4]:    # value in each sub-list
paddings[i] = max(paddings[i], len(str(val)))
i += 1
headr = content_min_max[0]
content += ('\n{}{}{}{}{}'.format(
headr[0] .rjust(paddings[0]),
(headr[1]).rjust(paddings[1]+5),
(headr[2]).rjust(paddings[2]+5),
(headr[3]).rjust(paddings[3]+5),
''
))
for lst in content_min_max[1:]:    # populate content using max lengths
content += ('\n{}{}{}{}     {}'.format(
lst[0].rjust(paddings[0]),
lst[1].rjust(paddings[1]+5),
lst[2].rjust(paddings[2]+5),
lst[3].rjust(paddings[3]+5),
lst[4],
))
log.info(content)

if not show_sorted_details: return
if len(df.columns) == 1:    return    # skip detail if only 1 column
if details == None:
details = df.columns
for detail in details:
if detail == 'sector': continue
hi = df[details].sort_values(by=detail, ascending=False).head(num)
lo = df[details].sort_values(by=detail, ascending=False).tail(num)
content  = ''
content += ('_ _ _   {}   _ _ _'  .format(detail))
content += ('\n\t... {} highs\n{}'.format(detail, str(hi)))
content += ('\n\t... {} lows \n{}'.format(detail, str(lo)))
if log_nan_only and not len(lo[lo[detail] != lo[detail]]):
continue  # skip if no nans
log.info(content)
There was a runtime error.

For the purpose of utilizing risk_loading_pipeline in an algo, is it necessary to include the references to "algo"?
Such as,
import quantopian.algorithm as algo
algo.attach_pipeline(risk_loading_pipeline(), 'risk_loading_pipeline')
algo.schedule_function
algo.order_optimal_portfolio

None of my existing algos use "algo" before attach_pipeline, schedule_function or order_optimal_portfolio.
Should they? Is there something about risk_loading_pipeline that needs "algo" to work properly?

Thanks

You don't technically need to use algo. before any of those functions. The algorithm module is 'magically' already imported in Quantopian algorithms. However, it is considered good practice to explicitly import all modules to make it more clear where they came from. We are making an effort to take this approach when writing new tutorials, documentation, and examples.

I hope this helps.

Disclaimer

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.

I am trying to backtest between 2006 and 2008 and on few of the days for some stocks, I get exceptions similar to the one below that I got for May 3, 2006:

ValueError:NaN or Inf values provided to FactorExposure for argument 'loadings'. Rows/Columns with NaNs: row=Equity(28054 [DBC]) col='momentum' row=Equity(28054 [DBC]) col='short_term_reversal' row=Equity(28054 [DBC]) col='size' row=Equity(28054 [DBC]) col='value' row=Equity(28054 [DBC]) col='volatility'

Could you please add the missing data on those days? You can get the list of days and stocks for which data is missing by using the QTradableStocksUS Universe and running a backtest. After 2008 onwards, the risk factor data runs fine.