Back to Community
Custom P&L measurement does not reflect backtest results

I'm doing some vetting of strategy efficiency and coded up some simple P&L metrics. When I run and tabulate the running P&L from my code I get vastly different results than what I get from the backtest engine.

I extracted the P&L measurements from the log statements and dropped them into excel and matched the recorded metric in the Quantopian chart. Now, I realize that I can't technically get the true P&L for a trade as I'm not privy to the actual closing price of the trade. But I should be able to get close. And I'm not. At least, I don't think so.

If you run this strat, you'll see the PandL line. This comes from keeping a running PnL value and incrementing (decrementing) from it on the close of every trade. Again, I won't be completely accurate - but what I'm seeing is nowhere close.

So, what's going on here?

Simple strat logic:
produce custom metric
smooth it twice.
when fast ma moves above slow ma,
- close open position
- buy
when fast ma moves below slow ma,
- close open position
- sell
if unrealized position P&L > 100
- close trade

Here's how I'm retrieving the P&L at the time of the trade:

def GetPositionPandL(context, data, securityID):  
    openQty = context.portfolio.positions[securityID].amount  
    if (openQty == 0):  
        return 0  
    entryPrice = context.portfolio.positions[securityID].cost_basis  
    currentPrice = data[securityID].close_price  
    entryNotional = entryPrice * openQty  
    currentNotional = currentPrice * openQty  
    if ( openQty > 0):  
        return currentNotional - entryNotional  
    else:  
        return entryNotional - currentNotional  
Clone Algorithm
5
Loading...
Backtest from to with initial capital
Total Returns
--
Alpha
--
Beta
--
Sharpe
--
Sortino
--
Max Drawdown
--
Benchmark Returns
--
Volatility
--
Returns 1 Month 3 Month 6 Month 12 Month
Alpha 1 Month 3 Month 6 Month 12 Month
Beta 1 Month 3 Month 6 Month 12 Month
Sharpe 1 Month 3 Month 6 Month 12 Month
Sortino 1 Month 3 Month 6 Month 12 Month
Volatility 1 Month 3 Month 6 Month 12 Month
Max Drawdown 1 Month 3 Month 6 Month 12 Month
import numpy

candleQuotients = list()
candleQuotientsMA = list()
maPeriods = 42
maPeriods2 = 11

def initialize(context):
    context.SPY = sid(8554)
    context.runningPandL = 0.0
    
def handle_data(context, data):
    
    cqMean = 0
    cqMAMean = 0
    
    cq = getSpecialMetric(data, 0, context.SPY, 0)
    if (cq != numpy.NaN and cq != None):
        candleQuotients.append(cq)
        if (len(candleQuotients) > maPeriods):
            candleQuotients.pop(0)            
            cqMean = numpy.mean(candleQuotients)
            record(CandleQuotientMean=cqMean)
            candleQuotientsMA.append(cqMean)
            if (len(candleQuotientsMA) > maPeriods2):
                candleQuotientsMA.pop(0)
                cqMAMean = numpy.mean(candleQuotientsMA)
                record(CandleQuotientMAMean=cqMAMean)

    price = data[context.SPY].price
    openQty = context.portfolio.positions[context.SPY].amount
    qty = 10000 / price;
    
    if (cqMean != 0 and cqMAMean != 0):
        if (cqMean > cqMAMean):
            if (openQty == 0):
                PlaceOrder(context, data, context.SPY, TradeSide.Long, qty)
            elif (openQty < 0):
                ClosePosition(context, data, context.SPY)
        elif (cqMean < cqMAMean):
            if (openQty == 0):
                PlaceOrder(context, data, context.SPY, TradeSide.Short, qty)
            elif (openQty > 0):
                ClosePosition(context, data, context.SPY)
    
    positionPandL = GetPositionPandL(context, data, context.SPY)
    record(UnrealizedPandL=positionPandL)
    if (positionPandL > 100):
        #log.info("Profit stop")
        ClosePosition(context, data, context.SPY)
        
def ClosePosition(context, data, stockID):
    order_target_percent(stockID, 0)
    positionPandL = GetPositionPandL(context, data, stockID)
    log.info("Position closed\t{sym}\tqty:\t{qty}\tP&L:\t{pandl}".format(sym=stockID.symbol, qty=context.portfolio.positions[stockID].amount, pandl=positionPandL))
    context.runningPandL += positionPandL
    record(PandL=context.runningPandL);

def GetPositionPandL(context, data, securityID):
    openQty = context.portfolio.positions[securityID].amount
    if (openQty == 0):
        return 0
    entryPrice = context.portfolio.positions[securityID].cost_basis
    currentPrice = data[securityID].close_price
    entryNotional = entryPrice * openQty
    currentNotional = currentPrice * openQty
    if ( openQty > 0):
        return currentNotional - entryNotional
    else:
        return entryNotional - currentNotional

def PlaceOrder(context, data, securityID, tradeSide, qty):
    if (tradeSide == TradeSide.Long):
        order(securityID, qty)
    else:
        order(securityID, -qty)
    #log.info("Order placed: {a} {b} for {c} @ {d}".format(a=securityID, b=tradeSide, c=qty, d=data[securityID].close_price))
        
@batch_transform(window_length=10)
def getSpecialMetric(dataCube, allATR, stockID, segmentCount):
    c = dataCube['close_price'][stockID][-1]
    return c

class TradeSide:
    Long = "Long"
    Short = "Short"
This backtest was created using an older version of the backtester. Please re-run this backtest to see results using the latest backtester. Learn more about the recent changes.
There was a runtime error.
6 responses

Anony, got your update tonight. I'll dig into it and get back to you.

Sorry the reply wasn't as fast as we usually are, but today was a market holiday. Most of us took some time off. Tomorrow we'll be back at full speed.

Disclaimer

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.

Thanks for being patient with us! I started digging into this tonight and I want to make sure that I understand your concern. You're worried that your running P&L doesn't match the overall algorithm return, the blue line? Is that what you were trying to replicate?

I have some initial thoughts but need to do some research and I'll circle back to you.

Disclaimer

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.

That's great to see! Happy that it worked out for you

I'm glad that there isn't an issue. If you find something you'd like us to look at, we will.

I dug into this with Alisa today. I think that the running PandL in the algorithm was trying to match the algo backtest results. The thing is that the backtester is actually pretty complex, and the simple PandL was missing some important nuances. Dividends, commissions, and slippage were things that I was going to look at next.

Also, I wanted to make sure you understood how the backtester works filling orders in daily mode. The backtester places an order in one bar, and then it is filled in the next bar. The close price of the bar is used for both calculations. That means in daily mode, if your algo places an order on a day, it is filled the next day - at the close price, not the open.

When you're trading in daily mode, yes, it would look exactly like that.

What you really should do is change your algorithm to minute mode. Then, in your chart, it would read "Minute1 Minute2. . . " Of course, you don't have to trade every minute. You can make it Buy ABC at the first minute, sell ABC at noon, by DEF at 12:01, sell DEF at 3:45, etc. It's up to you how you want that to play out.

Trading View does have some very pretty graphs. Definitely some good stuff there.

I think this function needs an abs() around the number of shares for short positions to be calculated properly.

def GetPositionPandL(context, data, securityID):  
    openQty = context.portfolio.positions[securityID].amount  
    if (openQty == 0):  
        return 0  
    entryPrice = context.portfolio.positions[securityID].cost_basis  
    currentPrice = data[securityID].close_price  
    entryNotional = entryPrice * openQty  
    currentNotional = currentPrice * openQty  
    if ( openQty > 0):  
        return currentNotional - entryNotional  
    else:  
        return entryNotional - currentNotional