Back to Community
order_target_percent question

Let me first thank Quatopian for a great tool, I love it! And the community discussions have been invaluable.

Now to my issue:

I've run into something I don't quite understand. To illustrate, I've set up this simple long daily algo that buys when there are no positions and sells everything when there are. I don't short, so should never have a negative positions value. I noticed that sometimes (I haven't found a way to reliably reproduce this, except in this test algo) a portfolio position has a duplicate and the algo sells it twice. To catch this, I've set up a random selection of assets to buy and the condition to only sell if my positions value >0. You'll notice from the end of transaction details that FSLR sold 44 shares twice even though only 44 shares were purchased the day before. Any explanation?

Thanks

Clone Algorithm
35
Loading...
Backtest from to with initial capital
Total Returns
--
Alpha
--
Beta
--
Sharpe
--
Sortino
--
Max Drawdown
--
Benchmark Returns
--
Volatility
--
Returns 1 Month 3 Month 6 Month 12 Month
Alpha 1 Month 3 Month 6 Month 12 Month
Beta 1 Month 3 Month 6 Month 12 Month
Sharpe 1 Month 3 Month 6 Month 12 Month
Sortino 1 Month 3 Month 6 Month 12 Month
Volatility 1 Month 3 Month 6 Month 12 Month
Max Drawdown 1 Month 3 Month 6 Month 12 Month
import random

# Put any initialization logic here.  The context object will be passed to
# the other methods in your algorithm.
def initialize(context):
    context.stocks = [symbol('AET'), symbol('AFL'), symbol('AIG'), symbol('ALTR'),
                      symbol('AMGN'), symbol('AMT'), symbol('ARG'), symbol('BLK'),
                      symbol('CB'), symbol('CMS'), symbol('CSCO'), 
                      symbol('CTL'), symbol('CVX'), symbol('EXC'), symbol('FDX'),
                      symbol('FSLR'), symbol('GAS', as_of_date='12-10-2011'),
                      symbol('GCI'), symbol('HOG'), symbol('HPQ'),  
                      symbol('HSY'), symbol('INTC'), symbol('IRM'), symbol('JWN'), 
                      symbol('KEY'), symbol('LEN'), symbol('LNC'), symbol('MAS'), 
                      symbol('MSFT'), symbol('NOV'), symbol('ORCL'),
                      symbol('PFG'), symbol('PNC'), symbol('PPG'), symbol('QCOM'), 
                      symbol('SE', as_of_date='12-15-2006'), symbol('SEE'), symbol('SYY'), 
                      symbol('TSS'), symbol('URBN'), symbol('V', as_of_date='03-19-2008'), 
                      symbol('VZ'), symbol('WIN', as_of_date='01-01-2006')]
    
    context.portSize = 2
    context.alloc = 1.0/context.portSize
    set_slippage(slippage.FixedSlippage(spread=0.00))
    set_commission(commission.PerTrade(cost=0.00))

# Will be called on every trade event for the securities you specify. 
def handle_data(context, data):
    
    # Implement your algorithm logic here.

    # data[sid(X)] holds the trade event data for that security.
    # context.portfolio holds the current portfolio state.

    # Place orders with the order(SID, amount) method.

    # TODO: implement your own logic here.
    any_open_orders = get_open_orders() #check for any open orders
    
    if not any_open_orders:
        if not context.portfolio.positions_value:
            portStocks = random.sample(context.stocks, context.portSize)
            for stock in portStocks:
                order_target_percent(stock, context.alloc)

        else:
            portStocks = context.portfolio.positions
            if context.portfolio.positions_value > 0:
                for stock in portStocks:
                    order_target_percent(stock, 0)


                
There was a runtime error.
9 responses

And just to clear it up a little, it's not dependent on a particular asset. e.g. next time you run this, it could happen with a different sid. I don't think it's a symbol vs. sid bug

Interesting scenario! You presented a very elegant algo to show the problem.

This looks related to a recently discovered bug and we're working on a fix. It's related to placing an order inside a loop that is iterating over positions. I'll let you know once it's patched and thanks again for the well written algo.

Disclaimer

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.

That was a great example, thank you. Definitely made my head pop when I saw it - that shouldn't happen!

The bug was fixed and shipped last night, so you shouldn't be able to reproduce it now. The underlying problem was pretty impressive. When a stock was held and then closed, when it was closed the positions object was being left with a vestigial object. When your code later iterated over the position, it could end up re-ordering the positions object, even mid-loop. That meant that a single stock could be processed first in the loop, then the object get re-ordered, and then the stock would be last and the re-processed. Very confusing behavior for sure.

I'm sorry for the frustration I'm sure this caused.

Disclaimer

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.

Dan, Alisa, thank you both for your quick response. I retested the algo and while the problem appears resolved for daily, it still seems to show up in minutely. I attach the even more simplified algo backtest.

Clone Algorithm
35
Loading...
Backtest from to with initial capital
Total Returns
--
Alpha
--
Beta
--
Sharpe
--
Sortino
--
Max Drawdown
--
Benchmark Returns
--
Volatility
--
Returns 1 Month 3 Month 6 Month 12 Month
Alpha 1 Month 3 Month 6 Month 12 Month
Beta 1 Month 3 Month 6 Month 12 Month
Sharpe 1 Month 3 Month 6 Month 12 Month
Sortino 1 Month 3 Month 6 Month 12 Month
Volatility 1 Month 3 Month 6 Month 12 Month
Max Drawdown 1 Month 3 Month 6 Month 12 Month
import random

# Put any initialization logic here.  The context object will be passed to
# the other methods in your algorithm.
def initialize(context):
    set_universe(universe.DollarVolumeUniverse(floor_percentile=99.0,
                                               ceiling_percentile=100.0))
    context.portSize = 2
    context.alloc = 1.0/context.portSize

# Will be called on every trade event for the securities you specify. 
def handle_data(context, data):
    
    if context.portfolio.positions_value==0:
        for stock in random.sample(data, context.portSize):
            order_target_percent(stock, context.alloc)

    elif context.portfolio.positions_value > 0:
        for stock in context.portfolio.positions:
            order_target_percent(stock, 0)


                
There was a runtime error.

This is the backtest behaving as designed, though it might not be intuitive. Let me explain what's happening.

The algorithm is filling orders across multiple bars according to the default slippage model, which will trade up to 25% of the stock's volume for that bar. If you're trying to place an order that accounts for more than 25% of the stock's transacted volume, the remaining shares will get filled in the following bar. To fill your entire order in each bar, you can force the slippage to 0.

When you account for slippage, here is your algo logic:

  • Bar 1: Portfolio value is 0, submit an order
  • Bar 2: Portfolio value is 0, fill the order
  • Bar 3: Portfolio value > 0, sell the stock
  • Bar 4: Portfolio value > 0, sell the stock again (note: the order wasn't completely sold in the previous bar so now you're shorting the stock)

When the sell orders from bars 3 and 4 are filled, your portfolio value is < 0 and the algorithm stops placing additional orders.

To fix this logic, try placing new orders only if there are no open orders. Take a look at the attached algo, I think this is working as expected.

Clone Algorithm
2
Loading...
Backtest from to with initial capital
Total Returns
--
Alpha
--
Beta
--
Sharpe
--
Sortino
--
Max Drawdown
--
Benchmark Returns
--
Volatility
--
Returns 1 Month 3 Month 6 Month 12 Month
Alpha 1 Month 3 Month 6 Month 12 Month
Beta 1 Month 3 Month 6 Month 12 Month
Sharpe 1 Month 3 Month 6 Month 12 Month
Sortino 1 Month 3 Month 6 Month 12 Month
Volatility 1 Month 3 Month 6 Month 12 Month
Max Drawdown 1 Month 3 Month 6 Month 12 Month
import random

# Put any initialization logic here.  The context object will be passed to
# the other methods in your algorithm.
def initialize(context):
    set_universe(universe.DollarVolumeUniverse(floor_percentile=99,
                                               ceiling_percentile=100.0))
    context.portSize = 2
    context.alloc = 1.0/context.portSize
    
    #force the fill in a single bar
    set_slippage(slippage.FixedSlippage(spread=0))

# Will be called on every trade event for the securities you specify. 
def handle_data(context, data):

       # Check if there are any existing open orders. has_orders() defined below
       has_orders = has_open_orders(data,context)
       
       
       # If there are open orders, wait til next minute
       if has_orders == True:
            log.info('Has open orders')
            return
        
       # If there are no open orders we can trade
       elif has_orders == False:
    
            if context.portfolio.positions_value==0:
                for stock in random.sample(data, context.portSize):
                    order_target_percent(stock, context.alloc)
                    #log.info("Bought stock " + str(stock.symbol))


            elif context.portfolio.positions_value > 0:
                for stock in context.portfolio.positions:
                    order_target_percent(stock, 0)
                    #log.info("Sell stock " + str(stock.symbol))


                
## RETURNS TRUE IF THERE ARE PENDING OPEN ORDERS, OTHERWISE RETURNS FALSE
def has_open_orders(data,context):               
# Only rebalance when we have zero pending orders.
    has_orders = False
    for stk in data:
        orders = get_open_orders(stk)
        if orders:
            for oo in orders:                  
                message = 'Open order for {amount} shares in {stock}'  
                message = message.format(amount=oo.amount, stock=stk)  
            has_orders = True
    return has_orders           
    if has_orders:
        return  
There was a runtime error.

My bad. Thanks!

EDIT: Don't mind this post, get_open_orders() is a dictionary of sid's

@Alisa

I am curious, in the above, why did you not just use:

    for oo in get_open_orders():  
        message = 'Open order for {amount} shares in {stock}'  
        message = message.format(amount=oo.amount, stock=oo.sid)  
        has_orders = True  

instead of

    for stk in data:  
        orders = get_open_orders(stk)  
        if orders:  
            for oo in orders:  
                message = 'Open order for {amount} shares in {stock}'  
                message = message.format(amount=oo.amount, stock=stk)  
            has_orders = True  

I'm rather new, and trying to acquire best practices

I am having the same issue that Sekuri originally had. My algo sells twice what it previously purchased. However, I am not having this issue with order_target_percent(); I am having the issue with order_target(). Is it possible that the 'purchasing multiple times when within loop' issue still exists with order_target() because I've tried everything. I've controlled orders to only execute if no current orders are open and that hasn't worked. I've also limited execution of the buy/sell orders to occur every half hour and made each order action place a log entry. I've simply found that log entries indicate a single sale order of context.portfolio.positions[stock].amount and Transaction Details in the backtest indicates an order value double that of context.portfolio.positions[stock].amount.

I just modified my algo to work with order_target_percent() and I am no longer having issues. So the problem must be exclusive to order method order_target()! Please help Quantopian Staff