Back to Community
How to add current price to USEquityPricing.close ?

I want to calculate a custom factor with closing price along with current intraday prices.

Is there a way to append current day prices to USEquityPricing.close before or after calling the custom factor ?

def compute(self, today, assets, out, close ...  

or does this have to be done using data.history() ? The problem I see with data.history is that you have to pass the sid of all 8000+ US Equities .. how would I do that?

Thank you

3 responses

Pipeline only has access to yesterdays pricing and data (as of 'before_trading_start'), so no, you cannot incorporate the current price directly into a CustomFactor.

However, you can easily get the current prices of all securities returned by your pipe by using the 'data.current' method. It accepts a list as well as a single security. The index of the pipeline output is just such a list of securities. No need to get 'all 8000+' securities. I like to then store those current prices in the same dataframe as the other pipeline outputs so everything is in one neat place. Something like this...

def before_trading_start(context, data):  
    # Get a dataframe of our pipe data.  
    context.output = pipeline_output('my_pipe')  


def enter_buy_sell_orders(context, data):  
    # Get the current prices for all stocks returned by our pipe (ie context.output.index)  
    # Add those prices as a new column named 'current_price' in 'context.data'  
    context.output['current_price'] = data.current(context.output.index, 'price')

    # Do any logic you wish on any of the columns in the pipe dataframe output  
     stocks_to_buy = context.output.query('current_price < yesterday_close').head(TARGET_STOCKS).index  

See attached algorithm. Good luck.

Clone Algorithm
48
Loading...
Total Returns
--
Alpha
--
Beta
--
Sharpe
--
Sortino
--
Max Drawdown
--
Benchmark Returns
--
Volatility
--
Returns 1 Month 3 Month 6 Month 12 Month
Alpha 1 Month 3 Month 6 Month 12 Month
Beta 1 Month 3 Month 6 Month 12 Month
Sharpe 1 Month 3 Month 6 Month 12 Month
Sortino 1 Month 3 Month 6 Month 12 Month
Volatility 1 Month 3 Month 6 Month 12 Month
Max Drawdown 1 Month 3 Month 6 Month 12 Month
'''
Basic template for stock selection and trading.
Pipline factors are used to 'pick' the stocks 
'''

# The following imports need to included when using Pipeline
from quantopian.algorithm import attach_pipeline, pipeline_output
from quantopian.pipeline import Pipeline

# Import all the built in Quantopian filters and factors (just in case)
import quantopian.pipeline.filters as Filters
import quantopian.pipeline.factors as Factors

# Import Pandas and Numpy (just in case we want to use their functionality)
import pandas as pd
import numpy as np

# Import any specialiazed packages here (eg scipy.optimize or scipy.stats)
pass

# Import any needed datasets
from quantopian.pipeline.data.builtin import USEquityPricing


# Set any 'constants' you will be using
TARGET_STOCKS = 20
TARGET_LEVERAGE = 1.0


def initialize(context):
    """
    Called once at the start of the algorithm.
    """   
    
    # Set commission model or omit and the default Q models will be used
    set_commission(commission.PerShare(cost=0.0, min_trade_cost=0.0))
    set_slippage(slippage.FixedSlippage(spread=0))
    

    # Attach the pipeline defined in my_pipe so we have data to use
    attach_pipeline(pipe_definition(context), name='my_pipe')

  
    # Schedule when to trade
    schedule_function(enter_buy_sell_orders, date_rules.week_start(), time_rules.market_open())

    # Schedule when to record any tracking data
    schedule_function(my_record_vars, date_rules.every_day(), time_rules.market_close())
     

         
def pipe_definition(context):
    '''
    Here is where the pipline definition is set.
    Specifically it defines which collumns appear in the resulting dataframe.
    Typically don't apply a screen (or filter) to the entire result or at least be
    careful if you do. If any close rules depend upon data for current positions then ensure that
    those positions are always included in the dataframe (and not filtered out).
    '''
    
    # Create a universe filter which defines our baseline set of securities
    # If no filter is used then ALL assets in the Q database will potentially be returned
    # This is not what one typically wants because 
    #    1) it includes a mix of ETFs and stocks
    #    2) it includes very low liquid and 'penny' stocks
    #
    # This filter can also be used as a mask in factors to potentially speed up some calcs
    # Built in filters (eg Q500US) can be used or filters can be made directly from datasets as below
    universe = Filters.Q1500US()
    
    # Create any basic data factors that your logic will use.
    # This is done by simply using the 'latest' method on a datacolumn object.
    # Just ensure the dataset is imported first.
    yesterday_close = USEquityPricing.close.latest    

    # Create any built in factors you want to use (in this case Returns). 
    # Just ensure they are imported first.
    gain = Factors.Returns(inputs=[USEquityPricing.close], window_length=2, mask = universe)
    
    # Create any built in filters you want to use.
    pass

    # Create any filters based upon factors defined above.
    # These are easily made with the built in methods such as '.top' etc applied to a factor
    top_gainers = gain.percentile_between(75, 100, mask = universe)
    
    # Define the columns and any screen which we want our pipeline to return
    return Pipeline(
            columns = {
            'yesterday_close' : yesterday_close,
            'gain' : gain,
            },
            screen = top_gainers
            )
    
 
def before_trading_start(context, data):
    '''
    Run pipeline_output to get the latest data for each security.
    The data is returned in a 2D pandas dataframe. Rows are the security objects.
    Columns are what was defined in the pipeline definition.
    '''
    
    # Get a dataframe of our pipe data. Placed in the context object so it's available
    # to other functions and methods (quasi global)
    context.output = pipeline_output('my_pipe')

   
def enter_buy_sell_orders(context, data):
    '''
    Let's buy the 20 lowest priced stocks returned from the pipeline
    Order an equal amount of each stock
    Try to buy at yesterdays low by using a limit order
    '''
    # Add the current prices as a column in context.data for each stock returned by the pipeline
    context.output['current_price'] = data.current(context.output.index, 'price')
    
    # Use the .query method to find stocks to buy. Only take the top TARGET_STOCKS qty. 
    # The index will be the equity objects.
    stocks_to_buy = context.output.query('current_price < yesterday_close').head(TARGET_STOCKS).index
    
    try:
        weight = TARGET_LEVERAGE/stocks_to_buy.size
    except ZeroDivisionError:
        weight = 0.0
        log.info('no stocks to buy today')
        
    for stock in stocks_to_buy:
        if data.can_trade(stock):
            limit_price = context.output.get_value(stock, 'current_price')
            order_target_percent(stock, weight, style=LimitOrder(limit_price))

    # Sell everything that we don't want to buy
    for stock in context.portfolio.positions:
        if stock not in stocks_to_buy and data.can_trade(stock):
            order_target_percent(stock, 0)


def my_record_vars(context, data):
    """
    Plot variables at the end of each day.
    """
    
    # Record the number of positions held each day
    qty_of_positions = len(context.portfolio.positions)
    record(positions=qty_of_positions)
 
There was a runtime error.

Hi Dan,

Thanks for your response. That's too bad we cannot incorporate current price directly in CustomFactor. This sort of handicaps the pipeline/filters in live trading scenarios when needing to calculate real-time technical indicators?
What about doing lookups of current pricing and appending it to USEquityPricing.close before passing it as input to a CustomFactor ?

Pipelines are a great tool, but yes, they can't do everything. The single biggest reason for using pipeline is that it's optimized to actually 'pre-fetch' the data asynchronous to the algorithm execution. This GREATLY enhances speed. It's actually grabbing data for multiple days into the future each time it's called (to reduce the database calls). Therefore, you can't 'append' any 'realtime' data to a custom factor. It would also be nice to change a filter based upon current holdings, but no can do. The pipeline has already pre-fetched and pre-calculated the data. Maybe look at this post for some more insight https://www.quantopian.com/posts/custom-factor-calculation-over-iterating-help .

You can always use the data.current and data.history functions along with any of the TaLib functions for intraday calcs. Then simply add them to the pipeline output as in the above algorithm. Above I simply added a column for 'current_price'. This could just as easily been a column for '10_bar_rsi' gotten from the output of a TaLib function.