Back to Community
ASDASD

ASDASDASD

5 responses

Derek, this is a great script and thanks for sharing! This addresses a lot of common requests and questions about algorithm management.

These are very clear and elegant functions!

Disclaimer

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.

Hi,

I have taken the liberty to clean up and factor this code a little but not completely (as Alisa offered it as a good example for something she thought I needed to do this morning). I applied techniques from the 'functional programming' style. The resulting code is lighter on syntax/noise/moving parts, more robust and less brittle/more flexible (i.e, higher-order). Please apply the same techniques throughout the rest of the code and it will be higher-quality/more robust, clearer, more readable, more reusable and more maintainable.

If you cannot see easily which changes were made or have difficulty understanding the changes or why they were made, please feel free to ask for clarification.

Thank you,
Jonathan

Clone Algorithm
60
Loading...
Backtest from to with initial capital
Total Returns
--
Alpha
--
Beta
--
Sharpe
--
Sortino
--
Max Drawdown
--
Benchmark Returns
--
Volatility
--
Returns 1 Month 3 Month 6 Month 12 Month
Alpha 1 Month 3 Month 6 Month 12 Month
Beta 1 Month 3 Month 6 Month 12 Month
Sharpe 1 Month 3 Month 6 Month 12 Month
Sortino 1 Month 3 Month 6 Month 12 Month
Volatility 1 Month 3 Month 6 Month 12 Month
Max Drawdown 1 Month 3 Month 6 Month 12 Month
"""
AUTHOR:  DEREK M Tishler
EMAIL:   [email protected]
DATE:    10/26/2014
SUMMARY: Assorted helper function for new users to Quantopian trading.

Factored by:
    Jonathan Leonard  ([email protected])
"""
# Module Imports
import pandas as pd
from datetime import timedelta
import operator
from functools import partial

# HELPER FUNCTIONS

fst = lambda tup: tup[0]
snd = lambda tup: tup[1]

def get_all_open_orders():
    return reduce(operator.add, map(snd, get_open_orders().iteritems()), [])
    
def get_orders_for_stock(stock):
    orders = get_all_open_orders()
    return filter(lambda o: o.sid == stock and o.amount != 0, orders)

def check_if_no_conflicting_orders(stock):
    # Check that we are not already trying to move this stock
    return len(get_orders_for_stock(stock)) == 0

def check_invalid_positions(context, securities):
    # Check that the portfolio does not contain any broken positions
    # or external securities
    for sid, position in context.portfolio.positions.iteritems():
        if sid not in securities and position.amount != 0:
            errmsg = \
            "Invalid position found: {sid} amount = {amt} on {date}".format(sid=position.sid,
                                                                            amt=position.amount,
                                                                            date=get_datetime())
            raise Exception(errmsg)

def do_cancel_order(context, sym, do_delete, order):
    log.info("X CANCELED {0:s} with {1:,d} / {2:,d} filled"\
        .format(sym,
                order.filled,
                order.amount))
    cancel_order(order)
    if do_delete: del context.duration[order.id]

def close_orders_after_time(context, stock, currentTime):
    # Ensure that the orders are not open past its desired duration.
    #  For live trading you could replace this with VWAPBestEffort,
    #  https://www.quantopian.com/help#api-VWAPBestEffort

    f = lambda o: o.id in context.duration and context.duration[o.id] <= currentTime
    map(partial(do_cancel_order, context, stock.symbol, True), filter(f, get_orders_for_stock(stock)))

def end_of_day(context, data):
    # cancel any order at the end of day. Do it ourselves so we can see slow moving stocks.

    log.info("")
    log.info("* EOD: Stoping Orders & Printing Held *")
    log.info("")

    #Print what positions we are holding overnight
    for stock in data:
        if context.portfolio.positions[stock.sid].amount != 0:
            log.info("{0:s} has remaining {1:,d} Positions worth ${2:,.2f}"\
                     .format(stock.symbol,
                             context.portfolio.positions[stock.sid].amount,
                             context.portfolio.positions[stock.sid].cost_basis\
                             *context.portfolio.positions[stock.sid].amount))

    # Cancel any open orders ourselves(In live trading this would be done for us, soon in backtest too)
    open_orders = get_all_open_orders()
    map(partial(do_cancel_order, context, stock.symbol, False), open_orders) # stock here just happens to be last one in list (is that truly what is desired?)

def fire_sale(context, data):
    #Sell everything in the portfolio, at market price
    log.info("# Fire Sale #")
    for stock in data:
        if context.portfolio.positions[stock.sid].amount != 0:
            order_target(stock, 0.0)
            value_of_open_orders(context, data)
            availibleCash = context.portfolio.cash-context.cashCommitedToBuy-context.cashCommitedToSell
            log.info("- SELL {0:,d} of {1:s} at ${2:,.2f} for ${3:,.2f} / ${4:,.2f}"\
                         .format(context.portfolio.positions[stock.sid].amount,
                                 stock.symbol,
                                 data[stock]['price'],
                                 data[stock]['price']*context.portfolio.positions[stock.sid].amount,
                                 availibleCash))   
    
        
    
def percent_diff(val1,val2):
    return abs(val1-val2)/((val1+val2)/2.0)

def value_of_open_orders(context, data):
    #current cash commited to open orders
    context.currentCash = context.portfolio.cash
    open_orders = get_open_orders()
    context.cashCommitedToBuy = 0.0
    context.cashCommitedToSell = 0.0
    if open_orders:
        for security, orders in open_orders.iteritems():
            for oo in orders:
                #estimate value of existing order with current price, best to use order conditons?
                #log.info(oo.amount * data[oo.sid]['price'])
                if(oo.amount>0):
                    context.cashCommitedToBuy += oo.amount * data[oo.sid]['price']
                elif(oo.amount<0):
                    context.cashCommitedToSell += oo.amount * data[oo.sid]['price']

#END HELPER FUNCTIONS

#For the sake of example, define some universe and apply the helper fucntions to illustrate their use.
def initialize(context):

    #Manually define stocks instead of downloading a universe with Fetch or using a cross section with set_universe
    context.stocks = symbols('AAPL', 'IBM', 'CSCO', 'SYN')
        
    #Dictionary of durations, optional for any order
    context.duration = {}
    
    #Inside the data loop, keep track of commited cash for the new orders.
    #  The portfolio cash is not updated on an intra cycle time scale yet.
    context.cashCommitedToBuy  = 0 
    context.cashCommitedToSell = 0
    
    #set a more realistic commission for IB
    set_commission(commission.PerShare(cost=0.014, min_trade_cost=1.4))
    
    #Prevent shorting, not needed here but it will stop 
    #  runaway code, like if you buy condition goes nuts 
    #  borrowing uncontrollably.
    set_long_only()
    
    log.info("---Prices below reflect market price or average held value at time of action and"+
             " NOT the value of the transactions. Use the Run Full Backtest"+
             " button and view the transactions tab to view real prices.---")

# Will be called on every trade event for the securities you specify. 
def handle_data(context, data):
    #Get EST Time
    exchange_time = pd.Timestamp(get_datetime()).tz_convert('US/Eastern')
    
    #Check that our portfolio does not  contain any invalid/external positions/securities
    check_invalid_positions(context, data)
    
    #Perform a fire sale the next morning, since we held a couple positions overnight
    if exchange_time.day == 3:
        if (exchange_time.hour == 9 and exchange_time.minute == 31):
            fire_sale(context,data)
        #The backtester only allows for a two day test currently, lets just trade on 7/2/2014
        return
    #
    
    #Print portfolio state at the start and enc of the day
    if (exchange_time.hour == 9 and exchange_time.minute == 31) or\
       (exchange_time.hour == 16):
        value_of_open_orders(context, data)
        log.info("")
        log.info("** Cash: ${0:,.2f} | Capital: ${1:,.2f} | BUY:{2:,.2f} | SELL: ${3:,.2f} **"
                 .format(context.portfolio.cash,
                         context.portfolio.capital_used,
                         context.cashCommitedToBuy,
                         context.cashCommitedToSell))
        log.info("")
    #
    
    #End of day
    if exchange_time.hour == 15 and exchange_time.minute == 55:
        #Close all orders, print open positions(cause our goal was to not have any at end of day)
        end_of_day(context, data)
        
        #exit handle_data here to prevent more trading
        return
    #
    
    #For this time event, act on each security in the portfolio
    for stock in data:
        
        #Ensure that the orders are not open past its desired duration.
        #  For live trading you could replace this with VWAPBestEffort, https://www.quantopian.com/help#api-VWAPBestEffort
        close_orders_after_time(context, stock, exchange_time)
        
        #If no price data for this stock this event cycle, we cant act(data[stock]['price'] will throw error)
        if not 'price' in data[stock]:
            #No price in this time step!
            log.info("X Could not fetch price for {0:s} at {1:d}:{2:d}"\
                     .format(data[stock]['price'],
                             exchange_time.hour,
                             exchange_time.minute))
            #skip to next stock in data
            continue
    
        #Lets BUY each of our stocks in the morning, 
        #  using a fixed time is not ideal but we only 
        #  want to place a single order per stock this script
        if exchange_time.hour == 9 and  exchange_time.minute == 31:

            #No exiting positions this script cause we used a fix time and no previous positons, 
            #  but as a habbit lets make sure we dont already have MOVE for this stock
            if check_if_no_conflicting_orders(stock) and context.portfolio.positions[stock.sid].amount == 0:
                #Partition the portfolio cash equally among the universe
                orderId    = order_target_percent(stock, 1.0/len(data))
                shareCount = get_order(orderId).amount
                
                #example, lets set a duration time for SYN to avoid price drift with slow fill rates
                if stock.symbol == 'SYN':
                    context.duration[orderId]   = exchange_time + timedelta(minutes=60)
                    
                value_of_open_orders(context, data)
                availibleCash = context.portfolio.cash-context.cashCommitedToBuy-context.cashCommitedToSell
                    
                #NOTE: the reported price here is the current market price, 
                #  depending on order style the actual transaction could
                #  be occuring after a price change. 
                #  Check out order types in doc: https://www.quantopian.com/help#api-order-methods
                log.info("+ BUY {0:,d} of {1:s} at ${2:,.2f} for ${3:,.2f} / ${4:,.2f}"\
                         .format(shareCount,
                                 stock.symbol,data[stock]['price'],
                                 data[stock]['price']*shareCount, 
                                 availibleCash))
                
        #Toward the end of the day, lets exitt our positions. Again the use of a fixed time is for example only
        if exchange_time.hour == 15 and exchange_time.minute == 30:

            #Ensure no conflicting position or orders before we BUY
            if check_if_no_conflicting_orders(stock) and context.portfolio.positions[stock.sid].amount != 0:
                #Take this position to zero(exit, entierly, the Buy or Short)
                order_target(stock, 0.0)

                value_of_open_orders(context, data)
                availibleCash = context.portfolio.cash-context.cashCommitedToBuy-context.cashCommitedToSell

                #NOTE: the reported price here is the current market price, 
                #  depending on order style the actual transaction could
                #  be occuring after a price change. 
                #  Check out order types in doc: https://www.quantopian.com/help#api-order-methods
                log.info("- SELL {0:,d} of {1:s} at ${2:,.2f} for ${3:,.2f} / ${4:,.2f}"\
                         .format(context.portfolio.positions[stock.sid].amount,
                                 stock.symbol,
                                 data[stock]['price'],
                                 data[stock]['price']*context.portfolio.positions[stock.sid].amount,
                                 availibleCash))   
        #Just for illustration in the logs, lets "accidently" hold a position overnight as well as an open order.
        if exchange_time.hour == 15 and exchange_time.minute == 45:
            #So after we have sold, lets just take a position in order 
            #  to see what happens if we hold position at end of 
            #  day, same for open order of SYN
            if stock.symbol == 'AAPL' or stock.symbol == 'SYN':
                #Ensure no conflicting position or orders before we BUY
                if check_if_no_conflicting_orders(stock) and context.portfolio.positions[stock.sid].amount == 0:
                    orderId    = order_target_percent(stock, 0.1)
                    shareCount = get_order(orderId).amount
                    value_of_open_orders(context, data)
                    availibleCash = context.portfolio.cash-context.cashCommitedToBuy-context.cashCommitedToSell
                    log.info("+ BUY {0:,d} of {1:s} at ${2:,.2f} for ${3:,.2f} / ${4:,.2f}"\
                         .format(shareCount,
                                 stock.symbol,data[stock]['price'],
                                 data[stock]['price']*shareCount, 
                                 availibleCash))
There was a runtime error.

Glad to see that people apreciate functional programming. You might be want to take a look at the itertools package. While python is strict, we can use generators to achieve lazily evaluated sequences. For example, in your function:

def get_all_open_orders():  
    return reduce(operator.add, map(snd, get_open_orders().iteritems()), [])  

you might be suprised to know that this function will traverse the length of the open orders twice, once for the map, and once for the fold (iteritems returns an iterator so this will be lazy). This is because python is eagerly evaluating the list returned by map so that it can pass it to reduce. If you are coming from python 3, you should know that map in python 2 does not return a generator, but instead returns a list. you can replace this call to map with its counterpart from the itertools package like:

def get_all_open_orders():  
    return reduce(operator.add, imap(snd, get_open_orders().iteritems()), [])  

There are counterparts for other functions too, for example: ifilter, izip, dropwhile, takewhile and other goodies you might want from your functional programming toolkit.

Just a note, there is already a name for foldl (+) in the standard libraries, you just need to coerce it a little ^_^

def get_all_open_orders():  
    return sum(imap(snd, get_open_orders().iteritems()), [])  

keep up the λ's

Disclaimer

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.

Hi Joe,

Yea, I'm aware of itertools and such. Maybe I'm weird but I always go for the core functions before importing additional modules unless performance is critical (e.g., range v. xrange). I didn't figure performance was critical in this particular case so didn't need to import anything extra.

Thanks for the tips though!

Thank you. This has been very helpful to read through. One comment brought up a question for me...

The comment: "Cancle any open orders ourselves(In live trading this would be done for us, soon in backtest too)"

Does this mean that open orders do not persist in Live trading, and are only good for the day?