Weeeee, selling unlimited shares! 4 million percent growth!

Quantopian Devs,

You should probably have something hardcoded into your backtester to prevent people from buying shares with money they don't have or selling shares that they haven't bought....

• Bayan
23
Total Returns
--
Alpha
--
Beta
--
Sharpe
--
Sortino
--
Max Drawdown
--
Benchmark Returns
--
Volatility
--
 Returns 1 Month 3 Month 6 Month 12 Month
 Alpha 1 Month 3 Month 6 Month 12 Month
 Beta 1 Month 3 Month 6 Month 12 Month
 Sharpe 1 Month 3 Month 6 Month 12 Month
 Sortino 1 Month 3 Month 6 Month 12 Month
 Volatility 1 Month 3 Month 6 Month 12 Month
 Max Drawdown 1 Month 3 Month 6 Month 12 Month
  # For this example, we're going to write a simple momentum script.  When the
# stock goes up quickly, we're going to buy; when it goes down quickly, we're
# going to sell.  Hopefully we'll ride the waves.

# To run an algorithm in Quantopian, you need two functions: initialize and
# handle_data.
import math

def initialize(context):
# This initialize function sets any data or variables that you'll use in
# your algorithm.  For instance, you'll want to define the security (or
# securities) you want to backtest.  You'll also want to define any
# parameters or values you're going to use.

# In our example, we're looking at Apple.  If you re-type this line
# yourself, you'll see the auto-complete that is available for the
# security ID.
context.aapl = sid(24)

# In these two lines, we set the maximum and minimum we want our algorithm
# to go long or short our security.  You don't have to set limits like this
# when you write an algorithm, but it's good practice.
context.max_notional = 1000000.1
context.min_notional = -1000000.0

def handle_data(context, data):
# This handle_data function is where the real work is done.  Our data is
# minute-level tick data, and each minute is called a frame.  This function
# runs on each frame of the data.

# We've built a handful of useful data transforms for you to use.  In this
# line, we're computing the volume-weighted-average-price of the security
# defined above, in the context.aapl variable.  For this example, we're
# specifying a three-day average.
vwap = data[context.aapl].vwap(3)
# We need a variable for the current price of the security to compare to
# the average.
price = data[context.aapl].price

cash = context.portfolio.cash

# Another powerful built-in feature of the Quantopian backtester is the
# portfolio object.  The portfolio object tracks your positions, cash,
# cost basis of specific holdings, and more.  In this line, we calculate
# how long or short our position is at this minute.
notional = context.portfolio.positions[context.aapl].amount * price

# This is the meat of the algorithm, placed in this if statement.  If the
# price of the security is .5% less than the 3-day volume weighted average
# price AND we haven't reached our maximum short, then we call the order
# command and sell 100 shares.  Similarly, if the stock is .5% higher than
# the 3-day average AND we haven't reached our maximum long, then we call
# the order command and buy 100 shares.
#if price < vwap * 0.995 and notional > context.min_notional:
order(context.aapl,-10000000)
#elif price > vwap * 1.005 and notional < context.max_notional:
#  order(context.aapl,+math.floor(cash/price))

This backtest was created using an older version of the backtester. Please re-run this backtest to see results using the latest backtester. Learn more about the recent changes.
There was a runtime error.
12 responses

Aren't you allowed to sell shares you haven't bought? Isn't that the whole point of short selling? The buying without money thing I don't know. Maybe assume unlimited credit? No idea (I'm really new at this, so could be wrong)

In theory, yes, but no broker with any sanity would allow it.

Every broker that I've seen has a margin requirement in order to prevent the case that you owe money from leveraging.

Also, your total aggregate realized loss should never exceed the sum of the total aggregate deposits and total aggregate realized profit.

• Bayan

Bayan, so far we've depended on the coder to put in the appropriate limits. In the sample algorithm, for instance, we put in some rudimentary limits - look at the references to "notional" in that code.

I'm interested in adding other limits in the future that more closely mimic the broker's limits. That will be very dependent on the account and the broker, though. For instance, you might have Reg T margin or you might have Portfolio Margin and there would be a huge difference what you can do. Since I don't have a one-size-fits-all, or even a one-size-fits-many solution yet, we're leaving it to the individual to figure out the right limits.

More feedback is welcome!

Disclaimer

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.

@ Dan and Daniel,

The first thing I do whenever I test a website is to try and break it. I am currently using mql4 to program my own expert advisers and indicators. I'm familiar with implementing the restrictions in code.

What I'm getting at is that without this restriction, it will be harder to compare backtest results with others. I'd have to go through their code to make sure that the result isn't because they forgot to set their trading restrictions properly. In addition, newbies to the website might not be familiar with how trading works and say to themselves "wow I just changed a few lines in the sample algorithm and now I get 4 million percent growth in a year!"

It would also be a shame to see people transition to live-trading then find out that their code wasn't working like they thought.

• Bayan

@Bayan - Welcome to Quantopian. Let me let you know, I am a professional money manager and I've been trying to break Quantopian for the past few months as well (I've also been trying to break my own backtester coded in extended basic - this is called "due diligence"). I'm not familiar with mql4.

If you are familiar with implementing restrictions in code then the comments raised by me and by Dan should not be outside your wheelhouse. It is far from Quantopian's responsibility to hard code a restriction on leverage. Leverage can be used in a very strategic way and should not be hard coded as a restriction. So I disagree with your concern 100%. Newbies should be equally aware of these "false positives".

As far as other issues I've encountered as concerns with the backtester, please see my other posts. I intend to post the links below by 8am tomorrow, but at this point I have way exceed my 16 hour day limit and I must call it a night. Will post links here tomorrow.

EDIT:
See the post below of the issues I've been focused on within Quantopian.

@Daniel,

Such an honor and privilege to have such a veteran like yourself on this website. I'm sure I'll be benefiting as much as the others from your experience.

My qualm is, for a newbie, they may not be aware of such false positives for a long time. They may be thinking for "Wow I made a great program, this is easy" for a very long time.

How about this idea, a flag or warning to let users know at what point the cash levels in their account were below the smallest lot size. This way, experienced users will have the freedom to ignore it and newbies will become aware of it.

• Bayan

Bayan, your point is well taken. It would be a bad thing if we handed newbies a set of pointy tools without also sharing the risks of the pointy parts. If I thought someone was taking one of these naive algorithms and trading with it today I'd be shocked, and I'd try to educate them, quickly, about the risks they were taking.

The live trading is where the pointy parts come in to play. We're working to make sure that people who do live trading with an algorithm are educated about the realities of their algorithm, including their broker limits. I believe that anyone who cloned the example above would be enlightened before they actually put money into it.

I'd like to make the backtesting experience useful and educational. I want to make it so our members can profit from their ideas, and are educated about the risks they're taking. Any suggestions or improvements are most welcome.

Perhaps a set_leverage(leverage_handler) function could be added, akin to set_universe, etc.? It could handle a few common cases (and default to one), while providing a null handler that didn't restrict anything, just like the current system. This manager would have to sit between the current order(...) and the component that actually handles orders and fills. It would merely check the current context.portfolio object and see if the order is valid within its rules, and pass it on if so, otherwise block it. In either case, its decision would need to be returned by order(...) to let the algorithm know whether the order could be entered. Whatever object would handle this information could also return the order information in case the order can be filled.

Likely, this would be useful upstream in zipline, and I'll look into seeing if I can contribute this, although I haven't really contributed to OSS before.

@Ryan - this was my approach as well. Check out this algo that sets leverage and let me know if it is similar to what you are thinking - https://quantopian.com/posts/template-limiting-leverage-by-brandon-ogle-and-dan-sandberg - essentially orders that would exceed the set leverage for the portfolio are rejected. One must also check the portfolio leverage at each new day to see if a margin call would be generated. This is the most complicated part. In other words, a $5000 short sale could be placed in an account with$10,000 resulting in $15,000 in cash and a -$5000 position in (let's say) ticker X. If the price of ticker X doubles, you would have a -$10,000 position in ticker X and still$15,000 in cash. If your margin requirement is larger than 1.5 then you would get a margin call.

Now appropriately handling the margin call can be very complicated. The simplest way would be to close out part of the short position until the margin requirement is met. But what if you have several short positions? Do you look pro rate based on position size or pro rat on the difference from cost basis? Do you close a long position instead to increase cash? There are literally infinite possibilities. That is why functions to limit leverage must not be "black boxed" into the back-end of the program. They must be flexible and edit-able within the code so each trader can address his/her strategy as desired.

I said I'd post some of the links to other issues I've raised about the backtester by this morning so here they are:

1)
The data feed that Quantopian uses is compiled from all trades within all exchanges. For example, XLY trades on AMEX in New York, Chicago, etc. The exchange reports an "official" closing price but this may not necessarily be the very last price of the day. As a result, there are discrepancies between Quantopian's datafeed and, for example, IQ Feed (a popular subscriber datafeed for algo trading) or Yahoo Finance. The caveats here are 1) small differences in cost basis between backtesters and 2) small differences in momentum values for given securities - https://quantopian.com/posts/problems-with-data-feeds-prices

2)
The benchmark is purportedly an investment in SPY. However, if you generate an algo to actually buy and hold SPY it will always outperform. The reason is that the benchmark is the price return of SPY and does not take into account dividend distributions. The caveat here is that you will frequently outperform the benchmark but in reality will underperform SPY. - https://quantopian.com/posts/question-regarding-the-benchmark

3)
I had an issue with forward filling in fetcher that caused some false buy signals. The caveat here is to make sure that you have data for each ticker and for each day of the backtest. Otherwise the most recent signal is forward filled. That sort of forward filling may be desirable depending on the algo. - https://quantopian.com/posts/help-with-fetcher-and-filling-nan-values

4)
I occasionally get runtime errors that I can not explain. These frequently occur with the internal functions available. Because the compiler catches the error and produces and error message, there is no false data generated but this does present an obstacle to generating certain algos. - https://quantopian.com/posts/runtime-error-with-vwap

5)
I have derived and implemented a basic sector rotation strategy within my firm that basically rotates between the sector SPDRs every quarter. It is not open source at the moment but we may open it up at some point. I have posted the "rankings" for the SPDRs from 2001 to 2013 in a csv and read them using Quantopian. The transactions in Quantopian appear to be consist witht he transactions I observe in my backtester (the one I coded in extended basic) but the final balance is different. The difference was initially substantial but I am finding small differences between assumptions made in each backtester and, as I resolve those differences, the answers appear to converge. I am still in the process of working on it. - https://quantopian.com/posts/sector-rotation-strategy-dot-dot-dot

Overall, there aren't necessarily errors in either Quantopian or in my in-house backtester (I'd provide my backtester but it is NOT user friendly at all). However, I have found that certain assumptions made within these backtesters have an impact on the final result, which can become compounded over a long backtest. I'm still trying to figure out all the nuance behind both backtesters, understand the caveats of the results generated by each, and work on providing the best asset management given all the tools I have.

I agree with everything you've said, essentially. There are several assumptions that would have to go into any handling of the margin. Still, I hope reasonable defaults can be made, while still allowing for more advanced users to implement the system that they would like. I'll have to review your code later, as I'm trying to figure out right now how to fit margin calls into zipline, before I go about trying to actually implement specific margin requirements.

With regards to #4 on your list of problems, I've experienced that too. In recent memory, it arose for me when I was working with minute data and only called a transform I had written once every hour (since it was very slow in runtime and in how it changed). I haven't checked under the hood, but I guess the pandas Panel that acts as a ring buffer to feed the transform assumes that it will receive data every timestep. If the transform isn't called, no data is fed to it, and then something breaks. Calling the transform for every timestep fixed the problem, even though I modified it to do nothing unless it was a new hour. However, this can't be your issue, since you do call vwap every timestep and I think those transforms are handled somewhat differently anyway.

I wish I had a better bug report to provide to hopefully narrow this down. If it is necessary to call a transform every timestep, it would be quite nice to know!