Back to Community
Any way to suppress "YYYY-MM-DD WARN Your order for XX shares of XYZ failed to fill by the end of day and was canceled." ?

I am working on an algo that is a both a bottom-feeder and a top-seller. Meaning, it starts the day with a long list of symbols it would like to buy but only if the price is right. And as well, it establishes a price for everything it holds. So, all its orders are either buy or sell at a limit price and most days only a few are filled.

At the end of each day, the log file is full of "YYYY-MM-DD WARN Your order for XX shares of XYZ failed to fill by the end of day and was canceled." messages which are of no interest in this case but hide other valuable info.

Any way to suppress that log message?

Richard
http://quant-coder.prokopyshen.com/ContactMe

8 responses

I've been manually cancelling any open order at EOD as a workaround.

I would check the liquidity of the equity's or book depth at target price level, but it could be a list of different issues. You may want to place a loop to stop continued orders on said security with open orders. For example,
for sec in context.securities_list:
open_orders = get_open_orders()
if sec in open_orders:
continue

SOLVED!

I always thought of Quantopian orders as "Good for the Day", but I think more accurately they are "Good for the Day + we warn you in the log when we cancel them".

So, to more accurately emulate a "Good for the Day with no warning", I run the following code snippet just before market close to anticipate what Quantopian will be doing:


    # Cancel any open orders  
    for S in get_open_orders():  
       for O in get_open_orders(S):  
          cancel_order(O)

I wonder how did you run your code before market close ?
@Richard Prokopyshen

Use the schedule function to run specific code at specific times. Place any schedule functions in the initialize portion of one's algo. Check out the docs https://www.quantopian.com/docs/api-reference/algorithm-api-reference#quantopian.algorithm.schedule_function.

So, in this case, to run a function to cancel all orders at market close do something like this

def initialize(context):  
    """  
    Initialize constants, create pipeline, and schedule functions  
    """

    # Cancel any open orders  
    # Do this at market close to suppress partial fill warnings  
    schedule_function(  
        func=cancel_all_open_orders,  
        date_rule=date_rules.every_day(),  
        time_rule=time_rules.market_close()  
    ) 

def cancel_all_open_orders(context, data):  
    """  
    Cancels all open orders  
    """  
    for security, open_orders in get_open_orders().items():  
        for order_object in open_orders:  
            cancel_order(order_object)

One comment...

While the 'failed to fill by the end of day' logging may be annoying, it is VERY informative. One should really modify their algo logic to mitigate this warning rather than ignore it. If an order fails to fill it implies your algo isn't in control. Ideally, one's algo should specify exactly how much of each stock to buy or sell. The backtester will then simulate what price the order fills at. If orders are being canceled, it means the backtester is essentially taking over the order quantity logic. Moreover, in live trading, there is the real potential that these orders will fill. The backtest results could therefore differ dramatically from actual results if there are a lot of these 'unfilled orders'. I would strongly recommend addressing any underlying issues in one's algorithm and ensure orders do all fill by the end of the day.

Disclaimer

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.

@Dan Whitnable
Excellent comment Dan; I completely concur. Just to give an example, my algo was attempting to establish positions in thinly traded small-caps that lacked liquidity. And, as you suggested, I had to rethink my logic at a different level once I thought this through. In the end, I did cap order quantity based on past volume, and then resubmit the next day if needed. If it took a week to enter a position then that is what it took.

Best,
Richard

@Dan can you clarify how we can get around this message and how the backtest takes over order quantity logic.

Is this because there isn't enough volume traded in the given equity? Should we get around it by not trading low volume equities?

@Robin Gane-McCalla You can 'get around' this message by closing all open orders at the end of the day. That is what the code above will do. The cancel_all_open_orders function will cancel any unfilled, or partially filled, orders. Therefore, when the backtest engine checks for open orders at the end of the day, it will never find any, and therefore, won't generate those warning messages.

The short answers to your questions are 'yes' and 'yes'.

Yes, the reason why orders go unfilled, or partially filled, is there isn't enough volume traded. The order filling process is controlled by the slippage model. The default model fills an order with a maximum of 10% of the minute volume. Consider the case where one orders 100 shares of XYZ and the trading volume was 870 and 900 shares for the following two minutes. The order would fill 87 shares in minute 1 and the remaining 13 shares in minute 2. One can over-ride the slippage model or write a completely custom one. However, the default has been shown to be pretty close to 'real life'.

Note the situation where there may be enough daily volume but one trades in the last few minutes of the day. The same thing happens. Not only consider the total daily volume but the 'net volume left in the day'.

Yes, generally do not trade low volume stocks. The backtest engine will most certainly be wrong or misleading especially if placing market orders. It's not just a matter of eliminating the warnings, but ensuring the backtest results resemble live trading results. The QTradableStocks universe will screen these out and is why it's good practice to start there.