Back to Community
Relative orders in backtest bug

According to the documentation, relative orders, when run in the backtest get treated as market orders. In the attached backtest, however, the relative order (which is submitted at the first bar) only trades at 9:56 AM. If I replace it by an equivalent limit order, it trades at 9:32AM. So that suggests that the relative order is not being treated as a market (or even a limit) order since it takes longer to fill. AAPL is a very, very liquid stock so it should certainly not take that long regardless.

Am I wrong in my expectations of how the rel order should behave in the backtest?

Also, is there an easy way to detect if an algo is running in a backtest? I may want to use limit orders in backtest but relative in production.

Clone Algorithm
3
Loading...
Backtest from to with initial capital
Total Returns
--
Alpha
--
Beta
--
Sharpe
--
Sortino
--
Max Drawdown
--
Benchmark Returns
--
Volatility
--
Returns 1 Month 3 Month 6 Month 12 Month
Alpha 1 Month 3 Month 6 Month 12 Month
Beta 1 Month 3 Month 6 Month 12 Month
Sharpe 1 Month 3 Month 6 Month 12 Month
Sortino 1 Month 3 Month 6 Month 12 Month
Volatility 1 Month 3 Month 6 Month 12 Month
Max Drawdown 1 Month 3 Month 6 Month 12 Month
from brokers.ib import RelativeOrder

# Put any initialization logic here.  The context object will be passed to
# the other methods in your algorithm.
def initialize(context):
    pass


def has_orders(context):
    for sec in [sid(24)]:
        if get_open_orders(sec):
            return True
    return False
        

# Will be called on every trade event for the securities you specify. 
def handle_data(context, data):
    useRel = True
    
    if context.portfolio.positions[sid(24)].amount == 0 and not has_orders(context):
        limit = data[sid(24)].close_price + 1.00
        if useRel:
            order(sid(24), 50, style = RelativeOrder(limit_price=limit, offset=0.01))            
        else:
            order(sid(24), 50, limit_price = limit)  
There was a runtime error.
8 responses

You realize you're working with limit orders here right?

A limit order placed ABOVE market price should (theoretically) be converted to a marketable limit (a market order at the going price (the ASK here as you're buying long)) .

That's why your second statement, limit_price = limit, get's hit right away (it's a market order).

The behavior of Relative Order looks to be exactly this: Price must move through the specified limit price, before the true limit order is placed. After that, price must fall (for a buy long order) before the limit order is executed. See the log here. At 9:56 price has penetrated the 125.26 threshold, and returned below to execute at 125.25.

Normally you would place your limit order BELOW market. STOP orders go above (in a buy long execution).

Alex, you discussed RelativeOrders back here https://www.quantopian.com/posts/relative-orders I'm surprised that this issue didn't come up then.

13:31  Close: 124.26  
13:32  Close: 124.56  
13:33  Close: 124.38  
13:34  Close: 124.46  
13:35  Close: 124.43  
13:36  Close: 124.63  
13:37  Close: 124.81  
13:38  Close: 124.81  
13:39  Close: 124.88  
13:40  Close: 125.09  
13:41  Close: 125.01  
13:42  Close: 124.96  
13:43  Close: 124.91  
13:44  Close: 124.94  
13:45  Close: 124.94  
13:46  Close: 125.25  
13:47  Close: 125.08  
13:48  Close: 125.37  
13:49  Close: 125.28  
13:50  Close: 125.37  
13:51  Close: 125.44  
13:52  Close: 125.32  
13:53  Close: 125.27  
13:54  Close: 125.32  
13:55  Close: 125.28  
13:56  Close: 125.25

According to the docs, 'When backtesting this order type, or forward-testing a Quantopian-backed paper trading algorithm, the order is modeled as a simple market order. It is only executed as a true RelativeOrder when run with an IB-backed paper or real-money algorithm'

If you've noticed, the relative order I placed had a limit $1 above the market and an offset of +0.01. That means that in production, it would constantly quote 1 cent better than NBBO, up to the limit price, and would get filled very quickly. If the behavior in backtest is such that market price must move above the limit price for the order to become a market order, that's very bizarre because it means that the limit price is being treated as a stop instead.

BTW, it's not true that normally you'd place your limit order below the market for a buy order. A very common use is to use limit orders when you intend the order to be instantly marketable (execute instantly) but want some protection against a mistake / flash crash scenario and don't want to send a market order so you don't end up buying for 9999.99 if something weird happens with the market.

PS: I believe that in the past, relative orders worked more correctly in the backtest and the current behavior is the result of some change. I've used them in production for over half a year now and they work just fine w/ IB.

@Alex, it's always hard to tell the expertise level of forum members... When you used a limit above price I assumed you were unclear on the concept. Yes, conversion to marketable limit orders are in both of our answers...

If you say RelativeOrders behaved like limit orders in the past when place above market, then their behavior has changed. It's as if they're acting like stoplimit orders; first the stop is touched then the limit with offset is enacted. Which might be a useful behavior but not the one published.

Unfortunately, the pseudo-stop is very non-useful behavior because it means that the backtest behaves drastically different from production and backtest results aren't related to actual trading results.

Can someone from Quantopian comment on this and either verify that it's a bug (and hopefully suggest an ETA for a resolution) or inform me that this is the expected behavior? And for the short term, does someone know how to determine if an algo is running in backtest vs live so I can put an if-statement and submit relative orders only for live trading?

Alex, this indeed looks like a bug in the backtester how RelativeOrder is handling the limit_price parameter. Digging into it, RelativeOrder works with the "offset" and "pct_offset" parameters, but is handling the limit_price parameter in a strange way. We'll take a look at it and get it fixed.

To answer your other question,

is there an easy way to detect if an algo is running in a backtest?

You can use the get_environment function: https://www.quantopian.com/help#api-get-environment

from zipline.api import get_environment

def handle_data(context, data):  
  # execute this code when algorithm is running with an  
  # Interactive Brokers account  
  if get_environment('arena') == 'IB':  
    pass

  # execute this code when algorithm is running in a backtest  
  if get_environment('arena') == 'backtest':  
    pass

Cheers,
Alisa

Disclaimer

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.

Can you explain to me how relative orders are currently implemented in the backtest? The API documentation of 'When backtesting this order type, or forward-testing a Quantopian-backed paper trading algorithm, the order is modeled as a simple market order.' appears to be inaccurate.

The documentation is correct, in backtesting RelativeOrders should be modeled as market orders in the simulation. However, there appears to be a bug where if a limit_price parameter is passed, this behavior breaks down. It's in our queue to patch and I'll post an update here once the limit_price behaves correctly.

Hi Alex, I wanted to follow up here and let you know this is fixed. RelativeOrders behave correctly in the backtest, and are modeled as regular market orders when a limit_price parameter is present.

Thanks for your patience and give it a whirl.

Cheers,
Alisa