Back to Community
(Rookie Mistake) Slippage Setting Does Work!

I was told that Setting the slippage to 0.0 would cure the morbid stupidity coded into the platform that I've seen where orders simply refuse to fill. Well, it doesn't. Second, I'm not at all happy that I have to make public what I'm doing as result of bugs in the platform, which I'm starting to suspect are there intentionally, because this only happens when you write algorithms against penny stocks, which obviously have the highest yielding potential. There is absolutely no excuse that anyone can give me that will explain what is happening here, and since I have to make pubic what I'm doing, clone the f... away and call yourself a genius! The Algorithm is designed to capture the top stocks of the day. If you do any trading, you will see that the biggest moves emanate from the $1 - $10 range on a regular basis. However, many times, stocks will gap up and fall off. So, the idea was the find a threshold, above the opening price, that I could get in and get out quickly, ensuring that I would catch the biggest movers on a regular basis. Well, I figured out how to do that. What I haven't figured out is why, ZSAN, and EYEG, fail to fill, dragging down the algorithm. In fact, the only times the algorithm loses money is when the platform decides to randomly insert this "order failed to fill" bull s.....! It's not an issue of volume, because I'm printing that out. There is more than enough liquidity. It's not even a limit order. It's a MARKET order closeout. How can I feel confident to put my money into something that inserts random garbage decisions for no reason? And how can I turn this nonsense off once and for all? Seriously, this is truly annoying.

Clone Algorithm
6
Loading...
Backtest from to with initial capital
Total Returns
--
Alpha
--
Beta
--
Sharpe
--
Sortino
--
Max Drawdown
--
Benchmark Returns
--
Volatility
--
Returns 1 Month 3 Month 6 Month 12 Month
Alpha 1 Month 3 Month 6 Month 12 Month
Beta 1 Month 3 Month 6 Month 12 Month
Sharpe 1 Month 3 Month 6 Month 12 Month
Sortino 1 Month 3 Month 6 Month 12 Month
Volatility 1 Month 3 Month 6 Month 12 Month
Max Drawdown 1 Month 3 Month 6 Month 12 Month
# Backtest ID: 591e75801d799065ff9130ed
There was a runtime error.
10 responses

I changed it to use VolumeShareSlippage. 100% of the current bar:

set_slippage(slippage.VolumeShareSlippage(volume_limit=1,price_impact=0.1))

This does not appear to work either. according to the documentation:

In the VolumeShareSlippage model, the price you get is a function of your order size relative to the security's actual traded volume. Setting the volume_limit to 1.00 will permit the backtester to use up to 100% of the bar towards filling your order. Using the same example, this will fill 60 shares in the next minute bar.

2017-01-06 12:05 PRINT ETRM - vo: 310052, avo: 3684247.2042
2017-01-06 12:06 PRINT ETRM (exit) - vo: 68740
2017-01-09 09:38 PRINT ETRM - vo: 213823, avo: 5680940.0771
2017-01-09 09:40 PRINT ETRM (exit) - vo: 224115
2017-01-11 09:37 PRINT BIOC - vo: 1538752, avo: 3373198.5
2017-01-11 09:38 PRINT BIOC (exit) - vo: 1350808
2017-01-24 09:34 PRINT CLRB - vo: 817292, avo: 746156.3
2017-01-24 09:35 PRINT CLRB (exit) - vo: 729603
2017-02-07 10:48 PRINT ALT - vo: 356157, avo: 7275918.3
2017-02-07 10:49 PRINT ALT (exit) - vo: 639133
2017-02-10 09:40 PRINT IMMU - vo: 647842, avo: 1961880.3
2017-02-10 09:41 PRINT IMMU (exit) - vo: 425664
2017-02-10 09:57 PRINT BIOC - vo: 393821, avo: 3206122.9
2017-02-10 09:58 PRINT BIOC (exit) - vo: 187992
2017-02-13 10:15 PRINT PLX - vo: 246857, avo: 4226413.6
2017-02-13 10:16 PRINT PLX (exit) - vo: 220318
2017-02-14 09:44 PRINT ZSAN - vo: 248359, avo: 4834072.6
2017-02-14 09:46 PRINT ZSAN (exit) - vo: 337074
2017-03-01 09:53 PRINT DVAX - vo: 353699, avo: 1407931.3
2017-03-01 09:54 PRINT DVAX (exit) - vo: 310365
2017-03-02 09:32 PRINT MYOS - vo: 291618, avo: 456584.3
2017-03-02 09:33 PRINT MYOS (exit) - vo: 264806
2017-03-09 09:56 PRINT OCRX - vo: 838086, avo: 4281677.5
2017-03-09 09:57 PRINT OCRX (exit) - vo: 442538
2017-03-13 09:36 PRINT CERU - vo: 393478, avo: 919062.5
2017-03-13 09:37 PRINT CERU (exit) - vo: 274092
2017-03-15 10:10 PRINT CPRX - vo: 277016, avo: 971880.7
2017-03-15 10:11 PRINT CPRX (exit) - vo: 239156
2017-03-24 09:36 PRINT HTGM - vo: 675801, avo: 4670065.0
2017-03-24 16:00 WARN Your order for 173 shares of HTGM failed to fill by the end of day and was canceled.
2017-04-04 10:24 PRINT CYCC - vo: 272880, avo: 1647135.7
2017-04-04 10:25 PRINT CYCC (exit) - vo: 26903
2017-04-10 09:50 PRINT GERN - vo: 560080, avo: 1705183.4
2017-04-10 09:51 PRINT GERN (exit) - vo: 238723
2017-04-10 10:30 PRINT open: 2.48, hi: 2.73, lo: 2.45

2017-04-10 10:30 PRINT GERN (final) - vo: 16461
2017-04-10 16:00 WARN Your order for -499 shares of GERN failed to fill by the end of day and was canceled.

That's a mistake.

There are existing limit orders where the limit was not reached, the limit orders were not filled by eod, the warn happens when stop or limit are not reached too. Meanwhile you had closed the positions, with the limit orders lingering. Timeline:
Limit to buy. Succeeds
Limit to sell. Pending
Market to sell. Succeeds
Notice of unfilled pending Limit order to sell

You might want to edit the original message and title, adding something like: Oops, rookie mistake. The title is False so you should really change it in some way.

In this function, consider canceling all including limit and stop orders first like the two added lines here:

def closeout_2(context, data):  
     for s in context.portfolio.positions:  
        for o in get_open_orders(s):  
            cancel_order(o.id)  
        print s.symbol + " - vol: " + str( data.current(s,'volume'))  
        amt = context.portfolio.positions[s].amount * -1  
        order_target(s,amt)  

Next, order_target for negative amt means if long, go short and visa-versa so that should be order_target(s, 0) or order(s, amt). The only stocks that lost money were the ones affected by that, they currently have both long and short positions.

This stop order line is targeting 0 however no position is held at that point so the stop orders never wind up coming into existence. Have to wait until a position exists (next minute if market order [unless unfilled] or until your limit to buy kicks in) to enter a stop order targeting a close of the position. You'll agree that this is tricky stuff, even more when partial fills are also considered.

order_target(s,0,style=StopOrder(context.sl[s]))  

@ L. Williams

Blue is correct. The issue is not with the slippage method. Rather its your implementation of limit orders and ordering.

Look guys. I appreciate the input, but the last order of the day is a Market Order. It is there as a (day trading) safeguard to ensure stocks are closed out by EOD, if the limit order condition is not met. I don't care what you do, the closeout will not occur. I've even rewritten the current code to use nothing but Market Orders and it still does not close out. If you look at the graph, you will see that the orders are failing to fill and carrying over to the next day. This is clearly indicated by the variables being printed and the blue line below displaying the dips in cash where sell orders, inexplicably, fail to fill. If you can show me a working example, I will gladly change the title and heap praise galore, but as it stands I'm leaning closer and closer to Quantconnect, and I don't want to do this because I don't want to pay for trades. So any help you guys can provide would be awesome. I can repost the code if needed with market orders, and you will see that setting slippage has no effect on the closeouts.

Lastly, the code below issues a Market Order for any open positions. Having set Slippage spread to 0.0, the market order is supposed to fill immediately on the next bar. That is not happening. If you can, please explain away?

def my_record_vars(context, data):
record(stocks=len(context.portfolio.positions))
record(cash=context.portfolio.cash)
record(leverage=context.account.leverage)
record(vix=context.vix)

 for s in context.portfolio.positions:  
        **order_target(s,0)**    *# This is issued 120 minutes into the day to ensure we have no carry over*  
        op = data.history(s,'open',1,'1d')[0]  
        hi = data.history(s,'high',1,'1d')[0]  
        lo = data.history(s,'low',1,'1d')[0]  
        print s.symbol  
        print "open: " + str(op) + ", hi: " + str(hi) + ", lo: " + str(lo)  

@ L. Williams
A couple of thoughts...

The code you had above (copied below) will simply place an order to close any existing positions. However, it will NOT close any orders that are open but not filled. To really close out, one should also cancel all open orders.

for s in context.portfolio.positions:  
        order_target(s,0)    *# This is issued 120 minutes into the day to ensure we have no carry over*  

I changed your original posted algorithm to use 'VolumeShareSlippage'

    set_slippage(slippage.VolumeShareSlippage(volume_limit=1.0, price_impact=0))

I've also added some debug code to your algorithm above. You can now see when orders are entered and when they are filled. That might help you see better what's going on. Note the 'order id' in the printout so you can see see how each order progresses. You can see from the 'limit price' and 'last price' outputs that the reason some orders do not fill is that the limit price is not reached.

With the 'VolumeShareSlippage' set to 1.0 the orders will consume 100% of the volume ordered on any given bar. So, if you have an order for 1000 shares and the volume was only 750 shares, then your order would fill 750 shares and still have outstanding 250 shares. You can see this happen in your trading from the logs. If what you want to do is fill 100% of your order REGARDLESS of actual volume then use

    set_slippage(slippage.FixedSlippage(spread=0))

That will completely fill your order independent of actual bar volume as long as the bar volume is greater than zero. Your order won't trade in a zero volume bar.

Because in no case will an order fill if there is zero volume in a bar, you can't be 100% certain that positions will all close at the end of the day. (this is very true in live trading also). The following should cover you pretty well though.

for security, order_list in get_open_orders().items():  
        for open_order in order_list:  
            cancel_order(open_order)

for security in context.portfolio.positions:  
    order_target(security, 0)  
Clone Algorithm
2
Loading...
Backtest from to with initial capital
Total Returns
--
Alpha
--
Beta
--
Sharpe
--
Sortino
--
Max Drawdown
--
Benchmark Returns
--
Volatility
--
Returns 1 Month 3 Month 6 Month 12 Month
Alpha 1 Month 3 Month 6 Month 12 Month
Beta 1 Month 3 Month 6 Month 12 Month
Sharpe 1 Month 3 Month 6 Month 12 Month
Sortino 1 Month 3 Month 6 Month 12 Month
Volatility 1 Month 3 Month 6 Month 12 Month
Max Drawdown 1 Month 3 Month 6 Month 12 Month
# Backtest ID: 591f36e0ceb5c949f18af7e1
There was a runtime error.

Detail on GERN you cited. 7aff is last four of an order id. It is that limit order that got canceled. The limit order to buy and market order to sell were both filled.

2017-04-10 06:50  PRINT GERN - vo: 560080, avo: 1705183.4  
                             Min  Action     Chng Sym    Prc   Now  Style Prc    PnL     ID  
2017-04-10 06:50 _t:361 INFO  20  Buy        5255 GERN   2.73     0 limit 2.74          04ba  
2017-04-10 06:51 _t:361 INFO  21    Bot      5255 GERN   2.70  5255 limit 2.74          04ba  
2017-04-10 06:51 _t:361 INFO  21  Sell      -5255 GERN   2.70  5255 limit 2.72          7aff  
2017-04-10 08:30  PRINT GERN  
2017-04-10 08:30  PRINT open: 2.48, hi: 2.73, lo: 2.45  
2017-04-10 08:30 _t:361 INFO 120  Sell      -5255 GERN   2.60  5255                     97a5  
2017-04-10 08:31 _t:361 INFO 121    Sold    -5255 GERN   2.61     0             [-518]  97a5  
2017-04-10 13:00 WARN Your order for -5255 shares of GERN failed to fill by the end of day and was canceled.  
2017-04-11 06:31 _t:361 INFO   1     Cancel -5255 GERN   2.53     0 limit 2.72          7aff  

When I add my tools I turn off any record_vars if it exists to make room for my charting, and was surprised to find later that you're doing ordering inside that, explains the shorting I mentioned, pardon my mistake.

Ok, I'm moving the following to the reblance function. If there isn't carry over from the previous day, leverage should be zero and cash balance should be full. If this is NOT the case, then none of what is stated here is accurate!

record(cash=context.portfolio.cash)
record(leverage=context.account.leverage)

Ok, I moved it to verify that what you are all saying is indeed TRUE. I was getting confused, and thinking that the position was being held overnight. It is indeed closing out the position. I also made several other mods and this looks really good now. You guys f.... rock!!! Thanks a lot.

Clone Algorithm
36
Loading...
Backtest from to with initial capital
Total Returns
--
Alpha
--
Beta
--
Sharpe
--
Sortino
--
Max Drawdown
--
Benchmark Returns
--
Volatility
--
Returns 1 Month 3 Month 6 Month 12 Month
Alpha 1 Month 3 Month 6 Month 12 Month
Beta 1 Month 3 Month 6 Month 12 Month
Sharpe 1 Month 3 Month 6 Month 12 Month
Sortino 1 Month 3 Month 6 Month 12 Month
Volatility 1 Month 3 Month 6 Month 12 Month
Max Drawdown 1 Month 3 Month 6 Month 12 Month
# Backtest ID: 591f573081e4a8665a118fbd
There was a runtime error.

@ L. Williams

Always glad to help. Good Luck!