Back to Community
Limitations of backtest model for short-selling

Several of my algorithm ideas that have survived out-of-sample test have been pairs strategies focusing on exchange-traded products. I might tell you a bunch of stuff you already know here, so bear with me. Many ETPs are convoluted, are "not meant to be held more than a day", or are apparently marketed only to retail investors. Examples are:

1) Inverse funds such as SH. If you short both SPY and SH on Quantopian, you will see a square-wave pattern with a quarterly period -- this corresponds to the dividend cycle of SPY and SH, one paying out, one paying in, when both are short-sold. In addition to the square-wave pattern, shorting both gives a mean return of about 1% per year. This is due to the fee difference between the funds.

2) Futures-backed commodity ETNs like WEAT. I think WEAT is one of the worst funds on the exchange, with a whopping expense ratio of 3.34%. WEAT can be hedged by going long on consumer staples with a fund like XLP.

3) Volatility ETNs like VXX. Volatility ETNs would be great to short because they often come with explicit inverses. VXX can be hedged by simultaneously going short on VXX and XIV.

4) Leveraged funds. Obviously leveraged funds are not permitted in the competition (I guess I should have taken this as a clue).

5) Other funds with high fees. One thing common to 1-4 above is that they charge abnormally high fees. This points out a fifth strategy, namely just a long-short strategy based on ETF management expense ratios.

Unfortunately I now suspect that most if not all of the funds listed above would not be desirable to short, because either huge collateral would be needed, a high fee would be charged, or the broker may not actually be able to locate the shares to borrow. I found this out from a few threads on the forums, where people were talking about shorting leveraged ETNs in a similar fashion:
https://www.quantopian.com/posts/first-try-algo-star-star-why-isnt-this-allowed-on-contest-star-star
https://www.quantopian.com/posts/i-really-wish-they-allowed-the-use-of-leveraged-etfs-7-dot-18-sharpe-13-dot-65-sortino-0-dot-02-beta-0-dot-01-volatility

I don't know whether this totally sinks all of (1-5) above, or if it just reduces their profitability. What do you think?

9 responses

Hey Doug,

In general ETF based strategies fall prey to a variety of real world issues, some of which you described. There is indeed structural inefficiency present in many instruments, but similarly to micro-cap equities, it is often near impossible to take advantage of this inefficiency due to trading difficulties and costs.

The other issue is position concentration risk. In general any strategy which places all its eggs in one basket is not gonna be desirable.
https://www.quantopian.com/lectures#Position-Concentration-Risk

In general the issue is that even if you combined many models trading of different ETFs, you're still trading on so few underlying instruments that you're in trouble. Also, Sharpe Ratio can be mathematically shown to be IC (ability to forecast future prices) * Breadth (number of independent bets). ETF strategies can satisfy the IC component, but rarely the breadth. Quants generally want to find small and consistent edges across a wide breadth of different instruments. This is why factor based development is so popular. You find small edges that are consistent amongst hundreds of stocks.
https://www.quantopian.com/lectures#Long-Short-Equity
https://www.quantopian.com/lectures#Factor-Analysis

Here's an interesting factor flow strategy
https://www.quantopian.com/posts/trading-expected-factor-flows

In general if you want ideas my recommendation is to investigate factor interactions. For instance, whereas a momentum factor is well known and will have little value these days, a sophisticated combination of momentum with a sentiment or corporate events data set may be interesting. For an idea, you could try checking whether momentum is more or less present in stocks that are popular in sentiment that month. Or whether positive sentiment before earnings reports results in more momentum or mean reversion behavior.

Disclaimer

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.

Hey Delaney,

The ETF universe is definitely pretty limited, especially when restricted to esoteric funds. Maybe that would eliminate options 1-4. That being said, a long-short strategy on the ETF universe seems pretty reasonable. For example, long the 100 funds with the lowest management expense ratio (MER) and short the 100 funds with the highest MER, limited to funds with at least $1M/day dollar volume. Firstly, money is directly being siphoned out in the fees -- this might be only one or two percent, but every bit helps. Secondly, I would claim that "normal" funds like broad equity and bond funds tend to have the lowest fees, while the weirdest (I would say worst) funds often charge high fees for their strange strategies. Unfortunately, I can't really test this in Quantopian, because shorting the funds I want to short may not be possible (or profitable) due to things outside the backtest model. Obviously you can't model everything, but the attractiveness of short-selling weird securities seems to lead people astray (at least myself and apparently a few others on the forums). Maybe it's just a lesson hard-learned, like the difficulty of trading micro-cap equity you mentioned. Ideally, it'd be possible to reject these types of strategies using a different slippage or commission model.

Thanks a lot for your suggestions regarding factor interactions. I'm quite interested in that, it's a totally different discussion, of course.

Absolutely, one of the current limitations that we have in our backtester is that while we model liquidity based on volume, we don't have short availability data or other metrics folks might use to determine these things. It is a problem because often the least tradeable instruments build up the most inefficiencies due to their inaccessibility. Therefore when people are looking for trading opportunities, the least tradeable stuff can shoot to the top of the list.

For now I would try to stick to the Q1500, into which we put a lot of work. It's the universe required for algorithms that receive an allocation from -- and are traded by -- Quantopian.
https://www.quantopian.com/lectures#Universe-Selection

Thanks a lot Delaney -- that makes a lot of sense.

I've been experimenting with shorting leveraged ETFs for a while - some in real trading and more in an IB paper trading account. The aim was to take advantage of the leverage beta decay and high fee rates. Even including commissions, all pairs are showing a profit, with the best being tickers such as JNUG/JDST, or DGAZ/UGAZ. The more volatile the better...

Unfortunately, the short borrow fees eat up just about all (or more than all) of the profits, making the trade not workable. The one and only trade I've found that makes quite good money - far more than the fees -- is shorting various ratios of XIV/TVIX. Both have relatively low borrow rates and when VIX is in contango, TVIX mostly drops like a stone.

What would be useful on the Quantopian platform would be a way of plugging in the short borrow fee so we can get a better idea of what might work. Is there any plan to include short borrow fees into the platform?

Thanks for the info John. One question, have you also tried XIV with VXX instead of TVIX?

Hey John,

If you run a backtest, you can import the results into research by using the get_backtest function. This gives you a backtest object with all the data accessible. You can use whatever assumptions about borrow fees you'd like and recompute returns based on all the transactions. That's likely the best approach for now. It would be great to have more details incorporated into the backtester, although some are specific to each person and so are trickier to model.

@Douglas -- VXX doesn't work as well as TVIX since with TVIX, not only do you get the contango decay, you get the leveraged beta decay. I just assume XIV*2 = 1*TVIX to be flat. Only upside to VXX is it seems to be readily available to short. I've only had a few instances where TVIX was unavailable. Hasn't been a problem since I started doing this back in November.

@Delaney -- I know how to use the get_backtest function, but not sure how I would apply the borrow fees. Thanks for the suggestion - I'll play around with it. Although my interest has waned now I see how much the borrow fees are in practice. Broker was pretty happy with me though.

I think you'd have to write a function that looped through your held positions and added up a PnL from borrow fees, then subtracted that from your overall PnL. Sorry to hear about your fee experience; and yeah it would be great to have that data integrated into the backtest somehow, but again your borrow fee depends on your relationship with the broker as far as I know.