Back to Community
Question about backtesting: measuring returns, total returns and specific returns

I have written a simple algorithm with macd crossovers for testing. I only buy stocks when they cross upwards and sell them the other way around. No short positions.

I tested the same algorithm in 2 different ways:

  • first I just run the algorithm with a long_weight = 0.01 and no futher limitations, the backtest returns a specific return of about 7% on top of the common returns of about 50%

  • However if I run the same algorithm and limit the number of stocks I want to buy to 100 I get a lower total return of about 25% which is understandable because the leverage used is much lower (approx. <1 dependend on results along the way) but also the specific return turns negative -2%

My question now is: how is this specific return calculated?
Is it the result relative to the money invested or is it relative to if you had all your cash invested all the time in the benchmark ?

If it is the latter this would declare why the specific result is so much worse if you invest in maximally 100 stocks instead of without limitation. You could have made more money investing all your money. But the overall algorithm (in the sense of investment choices) would still be sound.

I'm rather new to this so I would appreciate some help.

5 responses

The common returns, which are calculated for a backtest, are simply the correlation of the net portfolio returns to five common factors (momentum, size, value, short term reversal, and volatility). It's measuring how much of a portfolio's returns can be explained by the 'well known' or common factors. The specific returns are simply the excess returns beyond this common exposure. On a given day, the total returns = common returns + specific returns.

The questions was "Is [specific returns] relative to the money invested or is it relative to if you had all your cash invested all the time in the benchmark?". The specific returns are calculated from the actual net return of the entire portfolio. If the portfolio value increased 1% then that is what is used to calculate specific returns. It is independent of leverage and actual cash value and doesn't look at how those returns were realize. It simply uses the net daily end of day returns.

Hope that makes sense.

Disclaimer

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.

Thanks for your response.
But I'm pretty shure it is not entirely correct. Here is my result with (an exagerated amount of leverage)-I go allin all the time.
The result over 1 year (2019) is : 122. 159% (over 100000%), most of which are specific returns . I'm pretty shure this cannot be reached in the real world and this not the result of net return according the definition you gave above.
Leverage plays a (huge) role. Therefor once more the question: how exactly are the returns determined?

Clone Algorithm
5
Loading...
Total Returns
--
Alpha
--
Beta
--
Sharpe
--
Sortino
--
Max Drawdown
--
Benchmark Returns
--
Volatility
--
Returns 1 Month 3 Month 6 Month 12 Month
Alpha 1 Month 3 Month 6 Month 12 Month
Beta 1 Month 3 Month 6 Month 12 Month
Sharpe 1 Month 3 Month 6 Month 12 Month
Sortino 1 Month 3 Month 6 Month 12 Month
Volatility 1 Month 3 Month 6 Month 12 Month
Max Drawdown 1 Month 3 Month 6 Month 12 Month
# Import Algorithm API
from quantopian.algorithm import attach_pipeline, pipeline_output

# Import Pipeline class and datasets
from quantopian.pipeline import Pipeline
from quantopian.pipeline.data.builtin import USEquityPricing
from quantopian.pipeline.domain import US_EQUITIES
from quantopian.pipeline.data.sentdex import sentiment

# Import built-in trading universe
from quantopian.pipeline.filters import QTradableStocksUS
from quantopian.pipeline.filters.morningstar import Q1500US, Q500US
from quantopian.pipeline.filters import StaticAssets,  StaticSids 


#Rest
from quantopian.pipeline.factors import (BollingerBands,SimpleMovingAverage,EWMA, 
    MovingAverageConvergenceDivergenceSignal, RSI, FastStochasticOscillator)
from quantopian.pipeline import CustomFactor
import numpy

class MovingAverageConvergenceDivergenceSignal2(MovingAverageConvergenceDivergenceSignal):
    window_safe = True 

class SimpleMovingAverage2(SimpleMovingAverage):
    window_safe = True
    

    
class YesterdayEverything(CustomFactor):
    def compute(self, today, asset_ids, out, values):
        window_length=2
        # Calculates yesterdays ...
        out[:] = values[0]


def initialize(context):
    # Attach pipeline to algorithm
    attach_pipeline(
        make_pipeline(),
        'my_pipeline'
    )
 
    context.initial_cash = context.portfolio.cash
#    set_benchmark(symbol('MSFT'))
    
    # Schedule rebalance function
    schedule_function(
        rebalance,
        date_rule=date_rules.every_day(),
        time_rule=time_rules.market_open()
    )


def my_compute_weights(context):
    #calculate long_weight
    long_weight = 1
    
    #calculate short_weight, geen short positions
    # short_weight = 0.5
    return long_weight
    
    
    
def before_trading_start(context, data):
    # Execute any daily actions that need to happen
    # before the start of a trading session
    context.output = pipeline_output(
        'my_pipeline'
    )
    
    #Longs
    context.macd_long = context.output[context.output['macd_long']].index.tolist()
    
    # Shorts
    context.macd_short = context.output[context.output['macd_short']].index.tolist()

    #Everyday before trading start
 #   context.long_weight = my_compute_weights(context)
    

def rebalance(context,data):

    aant = len(context.portfolio.positions)
    max = 100
    long_weight = 1
    
    #Verkoop if short and je hebt een pos. in die security
    for security in context.macd_short:
        if security in context.portfolio.positions:
            if data.can_trade(security):
                order_target_percent(security,0)
                log.info("Sell: {}".format(security))
                aant -=1
 
    # Trade if long and aantal posities < 100  
    for security in context.macd_long:
        if security not in context.portfolio.positions: 
 #           if aant < max:
                if data.can_trade(security):
                    order_target_percent(security,long_weight)
 #                   log.info("Buy: {}".format(security)) 
                    aant+=1
    log.info("aantal securities:{}".format(aant))
    log.info("leverage:: {}".format(context.account.leverage))
                   

    

def make_pipeline():
    #create base universe
    base_universe = Q500US()
    #base_universe = StaticAssets(symbols('SPY'))
#    base_universe = StaticAssets(symbols('AAPL', 'MSFT', 'AMZN', 'GOOG', 'INTC', 'FB', 'CSCO', 'NVDA', 'QCOM', 'ASML'))
   # base_universe = StaticAssets(symbols('MSFT'))
    
    macd_signal =  MovingAverageConvergenceDivergenceSignal2(inputs=[USEquityPricing.close]) 
    macd = SimpleMovingAverage2(inputs=[macd_signal],window_length=9)
    latest_close = USEquityPricing.close.latest
  
    yesterday_macd_signal = YesterdayEverything(inputs=[macd_signal], window_length=2)
    yesterday_macd = YesterdayEverything(inputs=[macd], window_length=2)  
    
    macd_long =  (macd_signal > macd) & (yesterday_macd_signal <= yesterday_macd)
    macd_short = (macd_signal < macd) & (yesterday_macd_signal >= yesterday_macd)


    return Pipeline(columns={
                            'close'          :latest_close,
                            'macd_signal'    : macd_signal,
                            'macd'           : macd,
                            'yesterday_macd_signal': yesterday_macd_signal,
                            'yesterday_macd' : yesterday_macd,
                            'macd_long'      :macd_long,
                            'macd_short'     :macd_short,
                            },
                    screen = base_universe
                   )
There was a runtime error.

The daily return of a backtest is simply the gain in portfolio value from the previous day to the current current. It's calculated after the close each day. It can be represented like this

portfolio_value = sum_of_long_positions + sum_of_short_positions + cash  
daily_return = (portfolio_value_today - portfolio_value_yesterday) / abs(portfolio_value_yesterday)

Notice this is an arithmetic return and not a log return. To get the total return, the log of the daily returns are summed.

This definition of 'daily returns' is independent from whether the strategy trades or even if it has any positions. In fact, if the portfolio didn't trade at all and was always in cash, then the daily return would be zero. One other point to notice. Dividends are automatically added to the portfolio by the backtest engine which increases cash. Likewise, commissions are subtracted from cash.

The reason you are seeing huge returns is the algorithm is using huge leverage. In theory, if you could borrow enough to support that leverage, you would indeed probably see those returns.

Maybe, the sentence above is poorly worded [specific returns are] independent of leverage and actual cash value and doesn't look at how those returns were realized . I meant it to imply the calculation for specific returns doesn't take into account leverage or cash value. It only takes into account the portfolio returns. However, both those will generally impact returns so, indirectly, specific returns can be impacted by leverage. In this case a lot.

Once again, thanks for your explanation. This made it a lot clearer for me.

I think I can see where the problem is and, correct me if I'm wrong, the calculations are not correct when using leverage.

When leverage is used it is added directly to the portfolio_value_today and not to the portfolio_value_yesterday. This is of course corrrect in an absolute sense because this was the value of the portfolio yesterday but if you add it to the portfolio_value_today then you create returns out of nothing.
E.g. if my portfoliovalue (yesterday) is 100 and I would invest 50 in a new security (with leverage) and this security and my portfolio of 100 would stay at the same price during that day I would have made a return of 50% according to the definition above. So return value is added out of nothing.

I think this would be an improved calculation for daily_return:
daily_return = (portfolio_value_today - (portfolio_value_yesterday + leverage_today)) / abs(portfolio_value_yesterday + leverage_today)

So the excess cash or leverage used to invest above your portfolio_value should be taken into account for calculating the returns. Otherwise leverage is added to the returns directly which cannot be correct.

I would like to know your view on this.

When stocks are purchased the portfolio cash is always reduced by the purchase price plus any commission. When stocks are sold the portfolio cash is increased by the sale price less any commission. This is exactly what would happen in a real brokerage account.

Therefore the comment above isn't exactly correct

if my portfolio value (yesterday) is 100 and I would invest 50 in a new security (with leverage) and this security and my portfolio of 100 would stay at the same price during that day I would have made a return of 50% according to the definition above. So return value is added out of nothing.

Let's follow the chain of events when purchasing a stock (long). For simplicity assume (for now) that the end of day closing price of our stock XYZ stays the same at $100/share. Also assume commissions are $0.


day 1 - opening balance of $100,000  
        cash: $100,000  
        stock value: $0  
        portfolio value: $100,000  (100000 + 0)  
        leverage: 0   (0 / 100000)

day 2 - purchase shares with existing cash  
        buy 1000 shares XYZ @ $100/share  
        cash: $0  
        stock value: $100,000  
        portfolio value: $100,000    (0 + 100000)  
        leverage: 1  (100000 / 100000)  
        daily returns: 0%   (100000 - 100000) / 100000

day 3 - purchase shares on margin  
        buy 1000 shares XYZ @ $100/share  
        cash:  -$100,000  
        stock value: $200,000  
        portfolio value: $100,000    (-100000 + 200000)  
        leverage: 2     (200000 / 100000)  
        daily returns: 0%    ((100000 - 100000) / 100000)

day 4 - purchase more shares on margin  
        buy 1000 shares XYZ @ $100/share  
        cash:  -$200,000  
        stock value: $300,000  
        portfolio value: $100,000    (-200000 + 300000)  
        leverage: 3    (300000 / 100000)  
        daily returns: 0%    ((100000 - 100000) / 100000)

day 5 - sell all the shares  
        sell 3000 shares XYZ @ $100/share  
        cash:  $100,000  
        stock value: $0  
        portfolio value: $100,000   (100000 + 0)  
        leverage: 0   (0 / 100000)  
        daily returns: 0%    ((100000 - 100000) / 100000)


A few things to note about the above. 1) the portfolio value always stays at $100,000 even as more shares are bought with margin and leverage increases. 2) the daily returns, and net return over the 5 days, is 0%. 3) the cash in the account goes negative when leverage is over 1. A negative cash amount implies funds which are borrowed and need to be paid back.

So, what does a similar transaction look like if the stock price goes up? Again assume no commissions but now our XYZ goes up to $101. First with a leverage of 1.


day 1 - opening balance of $100,000  
        cash: $100,000  
        stock value: $0  
        portfolio value: $100,000 (100000 + 0)  
        leverage: 0   (0 / 100000)

day 2 - purchase shares with existing cash  
        buy 1000 shares XYZ @ $100/share  
        cash: $0  
        stock value: $100,000  
        portfolio value: $100,000   (0 + 100000)  
        leverage: 1  (100000 / 100000)  
        daily returns: 0%  (100000 - 100000) / 100000

day 3 - sell all the shares at $101  
        sell 1000 shares XYZ @ $101/share  
        cash:  $101,000  
        stock value: $0  
        portfolio value: $101,000   (101000 + 0)  
        leverage: 0   (0 / 101000)  
        daily returns: 1%   ((101000 - 100000) / 100000)


Looks good. We made 1%. Now look at the same transaction but buy on margin for a leverage of 2.

day 1 - opening balance of $100,000  
        cash: $100,000  
        stock value: $0  
        portfolio value: $100,000   (100000 + 0)  
        leverage: 0   (0 / 100000)

day 2 - purchase 2000 shares using margin  
        buy 2000 shares XYZ @ $100/share  
        cash: $-100,000  
        stock value: $200,000  
        portfolio value: $100,000   (-100000 + 200000)  
        leverage: 2   (200000 / 100000)  
        daily returns: 0%   (100000 - 100000) / 100000

day 3 - sell all the shares at $101  
        sell 2000 shares XYZ @ $101/share  
        cash:  $102,000  ((2000x101) + (-100000))  
        stock value: $0  
        portfolio value: $102,000   (102000 + 0)  
        leverage: 0   (0 / 102000)  
        daily returns: 2%   ((102000 - 100000) / 100000)


Notice that by adding leverage we multiply our returns. Even though we do not directly increase our net portfolio value with leverage we do multiply any gains. This is why using margin can be a great tool. However, notice I intentionally used the word multiply and not increase. If one had daily losses (eg -1%) leverage will multiply those and be -2%.

When comparing strategies it's typically best to have an 'apples to apples' comparison and ensure one's leverage is the same. Try to keep leverage around 1. In the real world, individuals are typically limited to a leverage of 2 by one's broker (and the SEC). Additionally, borrowing money is not free. The backtester doesn't take any borrowing costs into account. So, while the returns on highly leveraged portfolios are technically correct they would probably not be realized in live trading.