Back to Community
News Sentiment Pipeline Factors with Sentdex

For those interested in using news sentiment data feeds, this thread is dedicated to showing pipeline factor examples created with Sentdex's News Sentiment data feed. These factors are meant for you to build, iterate on, and use in other Pipeline based algorithms you may have.

There will be a number of different factors on this thread and new ones will be updated periodically. To view a complete list of template pipeline factors for all data feeds, please visit this factor library.

So here's what you're going to get:

  • Research: You'll get the economic hypothesis behind the factor and the iterations I went through before arriving at the final factor. You'll also be able to clone and visualize this factor yourselves by adjusting the date ranges and data imports available (for those who don't have the premium datafeed). This process is something you can replicate for yourselves easily by using the Factor Tearsheet.
  • Pyfolio in-sample and out-of-sample factor performance: You'll get the in-sample (3 Jan 2014 - 21 Jan 2016) and out-of-sample (3 Jan 2014 - 18 Apr 2016) results of a template algorithm using this factor.
  • Template algorithm: You'll get the template algorithm used to evaluate this factor. This algorithm is not geared for trading as commissions and slippage are set to zero; however, it provides a good, liquid universe filter for you to build and add other pipeline factors. Commissions and slippage are set to zero in order to analyze the alpha signal of the factor independent of other variables. I recommend using these factors as building blocks for your overall strategy.

Data Description

  • sentiment_signal - A standalone sentiment score from -3 to 6 for stocks
  • Coverage extends to almost all the S&P 500 securities as well as a few other key mentions

All Factors

Each factor is linked to the full research and backtest. You can find them all listed here:

Factor 1: Average monthly article sentiment

class AverageMonthlyArticleSentiment(CustomFactor):  
    # Economic hypothesis: Article sentiment can reflect the  
    # public's mood about a given security. In this case, use the past  
    # 30 day's article sentiment to make decisions for the next 20 trading  
    # days  
    inputs = [sentdex.sentiment_signal]  
    window_length = 30

    def compute(self, today, assets, out, sentiment_signal):  
        out[:] = np.nanmean(sentiment_signal, axis=0)  

This algorithm is for education - the algorithm is not intended to provide investment advice.

Disclaimer

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.

3 responses

Factor 1: Average monthly article sentiment

Updated for Q2:

class AverageMonthlyArticleSentiment(CustomFactor):  
    # Economic hypothesis: Article sentiment can reflect the  
    # public's mood about a given security. In this case, use the past  
    # 30 day's article sentiment to make decisions for the next 20 trading  
    # days  
    inputs = [sentdex.sentiment_signal]  
    window_length = 30

    def compute(self, today, assets, out, sentiment_signal):  
        out[:] = np.nanmean(sentiment_signal, axis=0)  
Loading notebook preview...
Notebook previews are currently unavailable.

Factor 1: Average monthly article sentiment

Updated for Q2: In-sample template backtest

class AverageMonthlyArticleSentiment(CustomFactor):  
    # Economic hypothesis: Article sentiment can reflect the  
    # public's mood about a given security. In this case, use the past  
    # 30 day's article sentiment to make decisions for the next 20 trading  
    # days  
    inputs = [sentdex.sentiment_signal]  
    window_length = 30

    def compute(self, today, assets, out, sentiment_signal):  
        out[:] = np.nanmean(sentiment_signal, axis=0)  
Clone Algorithm
Loading...
Backtest from to with initial capital
Total Returns
--
Alpha
--
Beta
--
Sharpe
--
Sortino
--
Max Drawdown
--
Benchmark Returns
--
Volatility
--
Returns 1 Month 3 Month 6 Month 12 Month
Alpha 1 Month 3 Month 6 Month 12 Month
Beta 1 Month 3 Month 6 Month 12 Month
Sharpe 1 Month 3 Month 6 Month 12 Month
Sortino 1 Month 3 Month 6 Month 12 Month
Volatility 1 Month 3 Month 6 Month 12 Month
Max Drawdown 1 Month 3 Month 6 Month 12 Month
import pandas as pd
import numpy as np

from quantopian.algorithm import attach_pipeline, pipeline_output
from quantopian.pipeline import Pipeline
from quantopian.pipeline.factors import CustomFactor, AverageDollarVolume
from quantopian.pipeline.classifiers.morningstar import Sector

# For use in your algorithms
# Using the full paid dataset in your pipeline algo
# from quantopian.pipeline.data.sentdex import sentiment as sentdex

# Using the free sample in your pipeline algo
from quantopian.pipeline.data.sentdex import sentiment_free as sentdex

class AverageMonthlyArticleSentiment(CustomFactor):
    # Economic hypothesis: Article sentiment can reflect the
    # public's mood about a given security. In this case, use the past
    # 30 day's article sentiment to make decisions for the next 20 trading
    # days
    inputs = [sentdex.sentiment_signal]
    window_length = 30
    
    def compute(self, today, assets, out, sentiment_signal):
        out[:] = np.nanmean(sentiment_signal, axis=0)
    
def make_pipeline():
    # Create our pipeline
    pipe = Pipeline()
    
    # Screen out penny stocks and low liquidity securities.
    dollar_volume = AverageDollarVolume(window_length=20)
    is_liquid = dollar_volume.rank(ascending=False) < 1000
    
    # Create the mask that we will use for our percentile methods.
    base_universe = (is_liquid)

    # Filter down to stocks in the top/bottom 10% by sentiment rank
    factor = AverageMonthlyArticleSentiment()
    longs = factor.percentile_between(80, 100, mask=base_universe)
    shorts = factor.percentile_between(0, 20, mask=base_universe)

    # Add Accern to the Pipeline
    pipe.add(longs, "longs")
    pipe.add(shorts, "shorts")

    # Set our pipeline screens
    pipe.set_screen((longs | shorts))
    return pipe

# Put any initialization logic here. The context object will be passed to
# the other methods in your algorithm.
def initialize(context):
    attach_pipeline(make_pipeline(), name='factors')
    
    # Create our scheduled functions
    schedule_function(rebalance, date_rules.month_start())
    schedule_function(cancel_open_orders, date_rules.every_day(),
                      time_rules.market_close())

    set_commission(commission.PerShare(cost=0, min_trade_cost=0))
    set_slippage(slippage.FixedSlippage(spread=0))

def before_trading_start(context, data):
    # Assign long and short baskets
    results = pipeline_output('factors')
    assets_in_universe = results.index
    context.longs = assets_in_universe[results.longs]
    context.shorts = assets_in_universe[results.shorts]
    
# Will be called on every trade event for the securities you specify. 
def handle_data(context, data):
    pass
    
def cancel_open_orders(context, data):
    # Record our leverage, exposure, positions, and number of open
    # orders
    record(lever=context.account.leverage,
           exposure=context.account.net_leverage,
           num_pos=len(context.portfolio.positions),
           oo=len(get_open_orders()))

    # Cancel any open orders at the end of each day 
    for asset, orders in get_open_orders().items():
        cancel_order(order)
    
def rebalance(context, data):
    short_weight = -1.0/len(context.shorts)
    long_weight = 1.0/len(context.longs)
    assets_in_universe = (context.longs  | context.shorts)

    # Order our shorts
    for security in context.shorts:
        if data.can_trade(security):
            order_target_percent(security, short_weight)
            
    # Order our longs
    for security in context.longs:
        if data.can_trade(security):
            order_target_percent(security, long_weight)
            
    # Order securities not in the portfolio
    for security in context.portfolio.positions:
        if data.can_trade(security):
            if security not in assets_in_universe:
                order_target_percent(security, 0)
There was a runtime error.

Factor 1: Average monthly article sentiment

Updated for Q2: Out-of-sample template backtest

class AverageMonthlyArticleSentiment(CustomFactor):  
    # Economic hypothesis: Article sentiment can reflect the  
    # public's mood about a given security. In this case, use the past  
    # 30 day's article sentiment to make decisions for the next 20 trading  
    # days  
    inputs = [sentdex.sentiment_signal]  
    window_length = 30

    def compute(self, today, assets, out, sentiment_signal):  
        out[:] = np.nanmean(sentiment_signal, axis=0)  
Clone Algorithm
Loading...
Backtest from to with initial capital
Total Returns
--
Alpha
--
Beta
--
Sharpe
--
Sortino
--
Max Drawdown
--
Benchmark Returns
--
Volatility
--
Returns 1 Month 3 Month 6 Month 12 Month
Alpha 1 Month 3 Month 6 Month 12 Month
Beta 1 Month 3 Month 6 Month 12 Month
Sharpe 1 Month 3 Month 6 Month 12 Month
Sortino 1 Month 3 Month 6 Month 12 Month
Volatility 1 Month 3 Month 6 Month 12 Month
Max Drawdown 1 Month 3 Month 6 Month 12 Month
import pandas as pd
import numpy as np

from quantopian.algorithm import attach_pipeline, pipeline_output
from quantopian.pipeline import Pipeline
from quantopian.pipeline.factors import CustomFactor, AverageDollarVolume
from quantopian.pipeline.classifiers.morningstar import Sector

# For use in your algorithms
# Using the full paid dataset in your pipeline algo
from quantopian.pipeline.data.sentdex import sentiment as sentdex

# Using the free sample in your pipeline algo
# from quantopian.pipeline.data.sentdex import sentiment_free as sentdex

class AverageMonthlyArticleSentiment(CustomFactor):
    # Economic hypothesis: Article sentiment can reflect the
    # public's mood about a given security. In this case, use the past
    # 30 day's article sentiment to make decisions for the next 20 trading
    # days
    inputs = [sentdex.sentiment_signal]
    window_length = 30
    
    def compute(self, today, assets, out, sentiment_signal):
        out[:] = np.nanmean(sentiment_signal, axis=0)
    
def make_pipeline():
    # Create our pipeline
    pipe = Pipeline()
    
    # Screen out penny stocks and low liquidity securities.
    dollar_volume = AverageDollarVolume(window_length=20)
    is_liquid = dollar_volume.rank(ascending=False) < 1000
    
    # Create the mask that we will use for our percentile methods.
    base_universe = (is_liquid)

    # Filter down to stocks in the top/bottom 10% by sentiment rank
    factor = AverageMonthlyArticleSentiment()
    longs = factor.percentile_between(80, 100, mask=base_universe)
    shorts = factor.percentile_between(0, 20, mask=base_universe)

    # Add Accern to the Pipeline
    pipe.add(longs, "longs")
    pipe.add(shorts, "shorts")

    # Set our pipeline screens
    pipe.set_screen((longs | shorts))
    return pipe

# Put any initialization logic here. The context object will be passed to
# the other methods in your algorithm.
def initialize(context):
    attach_pipeline(make_pipeline(), name='factors')
    
    # Create our scheduled functions
    schedule_function(rebalance, date_rules.month_start())
    schedule_function(cancel_open_orders, date_rules.every_day(),
                      time_rules.market_close())

    set_commission(commission.PerShare(cost=0, min_trade_cost=0))
    set_slippage(slippage.FixedSlippage(spread=0))

def before_trading_start(context, data):
    # Assign long and short baskets
    results = pipeline_output('factors')
    assets_in_universe = results.index
    context.longs = assets_in_universe[results.longs]
    context.shorts = assets_in_universe[results.shorts]
    
# Will be called on every trade event for the securities you specify. 
def handle_data(context, data):
    pass
    
def cancel_open_orders(context, data):
    # Record our leverage, exposure, positions, and number of open
    # orders
    record(lever=context.account.leverage,
           exposure=context.account.net_leverage,
           num_pos=len(context.portfolio.positions),
           oo=len(get_open_orders()))

    # Cancel any open orders at the end of each day 
    for asset, orders in get_open_orders().items():
        cancel_order(order)
    
def rebalance(context, data):
    short_weight = -1.0/len(context.shorts)
    long_weight = 1.0/len(context.longs)
    assets_in_universe = (context.longs  | context.shorts)

    # Order our shorts
    for security in context.shorts:
        if data.can_trade(security):
            order_target_percent(security, short_weight)
            
    # Order our longs
    for security in context.longs:
        if data.can_trade(security):
            order_target_percent(security, long_weight)
            
    # Order securities not in the portfolio
    for security in context.portfolio.positions:
        if data.can_trade(security):
            if security not in assets_in_universe:
                order_target_percent(security, 0)
There was a runtime error.