Back to Community
3000% returns over 4 years - Where am I going wrong?

The algorithm is using a Random Forest classifier and trading daily on some randomly selected stocks from the SPY in 2011. The machine learning model is recreated once a month and it uses different technical indicators as input parameters. For speed, the model only trains on the SPY. The prediction of each stock during day trading is only taken if the prediction probability is greater than 60%. My problem is this: the returns after 4 years are abnormally large and I'm very discerned with the reality of the result. Where have I gone wrong?

Clone Algorithm
23
Loading...
Backtest from to with initial capital
Total Returns
--
Alpha
--
Beta
--
Sharpe
--
Sortino
--
Max Drawdown
--
Benchmark Returns
--
Volatility
--
Returns 1 Month 3 Month 6 Month 12 Month
Alpha 1 Month 3 Month 6 Month 12 Month
Beta 1 Month 3 Month 6 Month 12 Month
Sharpe 1 Month 3 Month 6 Month 12 Month
Sortino 1 Month 3 Month 6 Month 12 Month
Volatility 1 Month 3 Month 6 Month 12 Month
Max Drawdown 1 Month 3 Month 6 Month 12 Month
# Backtest ID: 5635135e907b8657b494e3bc
There was a runtime error.
4 responses

Almost always this type of problem is caused by not managing your leverage. The backtester will allow your algo to borrow unlimited amounts of money.

You can monitor context.account.leverage in your algo to track your leverage. Lots of examples here in the forums on how to do so.

Disclaimer

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.

Josh is right.

And your PvR hits a high of only around 40% and ends at 7.77%. PvR stands for Profit vs. Risk, it is a measure of returns based on amount put into play, and so it is looking more solely at code merit while ignoring starting capital. You can worry about starting capital later when you have an investor in the picture or want to enter the contest.

Usually a problem like this can be resolved to some degree more in line realistically by inserting a line like this:

    for one_stock in data:  
        if get_open_orders(one_stock): continue  

...however that didn't do the trick here.

So you might also give track_orders() a try, and I'm interested in knowing whether that helps if you would please, if you try it. (My hunch is some unfilled orders).

It is in Daily mode so far. Your schedule lines are specifying minutes 1 and 10. They may not be running when you expected them to. Add this line in both functions and see what you can make of that. It seems to me they both run at minute zero.

    print get_datetime().time().minute  

I tend to almost always track cash these days, besides the obvious it offers a sense of when big buys/sells are happening.

    record(cash = context.portfolio.cash)  

Alright, hope those help. Also here's some code to track PvR:

    schedule_function(pvr, date_rules.every_day(), time_rules.market_close())

def pvr(context, data):  
    c = context  
    if 'risk_hi' in c:  
        shorts = 0  
        for p in c.portfolio.positions:  
            shrs = c.portfolio.positions[p].amount  
            if shrs < 0:  
                shorts += int(abs(shrs * data[p].price))

        cash_dip  = int(max(0, c.portfolio.starting_cash - c.portfolio.cash))  
        risk      = int(max(cash_dip, shorts))  # Amount in play, maximum of shorts or cash used  
        c.risk_hi = max(risk, c.risk_hi)

        if c.risk_hi != 0:     # Avoid zero-divide  
            record(PvR_Ret = (100 * c.portfolio.pnl / c.risk_hi))        # Profit_vs_Risk returns

    else:    # Init this instead in initialize() for better efficiency  
        c.risk_hi = 0  

Thank you for your input, Josh and garyha. I've modified the code to take into account the 3.0 leverage limit. However I'm still not sure if the results are realistic or not, taking into consideration the backtest ends with around -$14 million in cash.

Clone Algorithm
61
Loading...
Backtest from to with initial capital
Total Returns
--
Alpha
--
Beta
--
Sharpe
--
Sortino
--
Max Drawdown
--
Benchmark Returns
--
Volatility
--
Returns 1 Month 3 Month 6 Month 12 Month
Alpha 1 Month 3 Month 6 Month 12 Month
Beta 1 Month 3 Month 6 Month 12 Month
Sharpe 1 Month 3 Month 6 Month 12 Month
Sortino 1 Month 3 Month 6 Month 12 Month
Volatility 1 Month 3 Month 6 Month 12 Month
Max Drawdown 1 Month 3 Month 6 Month 12 Month
# Backtest ID: 5637887a6d25bd110ba85083
There was a runtime error.

Insert this on line 141 to let those sells settle (I missed this before):

    if get_open_orders(): return  

The result will look awesome (due to extreme negative cash). PvR will be about half the benchmark. Then if you set maximum_leverage to 1.0, the code will beat SPY handily. The line above will result in the loss of shorting, maybe store those calcs and use them in the next frame.