Back to Community
PvR - Profit vs. Risk

As software developers, while writing trading algorithms, it is beneficial to know whether our changes increase or decrease the output in a run.

Consider an example:
1. Seeing 200% returns over two years, great.
2. Make a change.
3. Now seeing only 180% returns.
4 Throw away that change and think in another direction.
5. You actually made 5% more per trade.

So what happened? How can that be?

The problem is that your change seemed bad yet actually resulted in an increase in profit per dollar transacted for stocks (amount risked, or put into play).

To quote someone here recently, I don't want to be a party pooper. And yet, the fact of the matter is, you can't trust that returns chart.

The returns calculation employs a philosophy that all of your starting capital is at risk as soon as you open the account, and that's all that is ever at risk.
It calculates returns based on starting capital regardless of how much you actually activated (even if you spent very little of it or went negative, into margin, borrowing from the broker). To be fair, that is apparently common in the industry. Meanwhile in developing code, this will help you see straight.

In the example, the new code made different trades based on the change and those trades were overall more profitable, it's just that the amount put to work on each stock was lower. You spent less. You made more for each dollar. That's the goal, right?

PvR stands for Profit vs. Risk. It calculates returns based only on the amount put to work and as a result, is a reliable measure of code merit.

With PvR, while you are writing code, you no longer need to be concerned with trying to make sure you are spending all of the starting capital. You can worry about that later, closer to the time you're ready to go live, or whenever you decide to focus on it. The PvR metric lets you stay focused.

It also means you no longer need to be locked into a cage-match battle with margin, leverage. Doesn't matter, for now. PvR only cares about profit per dollar put to work. You can start out with $100K and spend just $10K or another time $10M, PvR will always give you a deterministic value you can rely on that measures your code merit.

By the same token, with no changes in the code, whether your starting capital field in the backtester is set at $10 or $10M, if you make the same trades, the chart will vary widely while PvR will be the same. You can always trust it.

Try it. Clone below, run the backtest, note the ending PvR, then change the starting capital value and run again. The main chart returns will be very different, however PvR will be the same.

There are three sets of PvR code in this post:
1. Minimal amount of code for backtests
2. Full code for backtests
3. Research/Notebook code, experimental

Below is a minimal amount of code for recording PvR. For best speed, replace c. with context. and remove c = context, then move the initialization to def initialize.

def handle_data(context, data):  
    pvr(context, data)

def pvr(context, data):  
    ''' Minimal custom chart of profit_vs_risk returns  
    '''  
    c = context  # Brevity, readability  
    if 'pvr' not in c:  
        # For real money, you can modify this to total cash input minus any withdrawals  
        manual_cash = c.portfolio.starting_cash  
        c.pvr = {  
            'chart_pvr'      : 1,  
            'chart_cash_low' : 1,  
            'chart_max_shrt' : 1,  
            'chart_max_risk' : 1,  
            'start'          : manual_cash,  
            'cash_low'       : manual_cash,  
            'max_shrt'       : 0,  
            'max_risk'       : 0,  
        }  
    c.pvr['cash_low'] = min(c.pvr['cash_low'], c.portfolio.cash)  
    c.pvr['max_shrt'] = max(c.pvr['max_shrt'], abs(sum([z.amount * z.last_sale_price for s, z in c.portfolio.positions.items() if z.amount < 0])))  
    c.pvr['max_risk'] = max(0, c.pvr['max_risk'], c.pvr['start'] - c.pvr['cash_low'], c.pvr['max_shrt'])

    # Profit_vs_Risk returns based on max amount actually invested, risked, long or short  
    if c.pvr['max_risk'] != 0: # Avoid zero-divide  
        if c.pvr['chart_pvr']:  record(PvR = 100 * (c.portfolio.portfolio_value - c.pvr['start']) / c.pvr['max_risk'])  
    if c.pvr['chart_cash_low']: record(CashLow = c.pvr['cash_low'])  
    if c.pvr['chart_max_shrt']: record(MxShrt  = c.pvr['max_shrt'])  
    if c.pvr['chart_max_risk']: record(MxRisk  = c.pvr['max_risk'])  
25 responses

This is the full PvR code. See the 'Source Code' tab. Just pay attention to the Options section at the beginning.
Once you are used to it, it's easy to add to any algo.

Edit: Better to use the full PvR code below (displayed in the large text block).

There's a section for initialize(), the main function, and the call to it, pvr(context, data). I've been making that call from handle_data() rather than using schedule_function() because it is more universal and adaptable that way. See the one pasted below, later, for a simplified and improved version.

Clone Algorithm
32
Loading...
Backtest from to with initial capital
Total Returns
--
Alpha
--
Beta
--
Sharpe
--
Sortino
--
Max Drawdown
--
Benchmark Returns
--
Volatility
--
Returns 1 Month 3 Month 6 Month 12 Month
Alpha 1 Month 3 Month 6 Month 12 Month
Beta 1 Month 3 Month 6 Month 12 Month
Sharpe 1 Month 3 Month 6 Month 12 Month
Sortino 1 Month 3 Month 6 Month 12 Month
Volatility 1 Month 3 Month 6 Month 12 Month
Max Drawdown 1 Month 3 Month 6 Month 12 Month
# Backtest ID: 568af93536818711686027d9
There was a runtime error.

Development in progress to chart PvR in a Research/Notebook.
Need help with this. Sometimes makes vast numbers.

#backtest = get_backtest('5637da7c2fe6e438be119651') # Place these two in a different cell ...  
#import pyfolio as pf                                #   above below, no need to always rerun them  
qrets      = backtest.daily_performance.returns          # Quantopian returns  
start      = backtest.daily_performance.starting_cash[1] # Starting capital like 100000, probably a better way  
cash_spent = start - min(backtest.daily_performance.starting_cash) # Most spent so far, should not see the future  
max_cash   = max(0, start - cash_spent)                  # Cash spent, of start or over, a logic puzzle for the mind  
cash_dip   = abs( backtest.positions.min().amount )      # Lowest single short as a positive value, should be all shorts  
riskhi     = max(max_cash, cash_dip)                     # Should be highest of cash spent or shorting  
pvr        = (qrets * start) / riskhi                    # (portf - start) / riskhi  
pf.plot_rolling_returns(qrets, pvr)                      # Plot Q returns and PvR  

If I'm understanding this correctly, the PvR number is like the total return per dollars used? Thanks for sharing.

Are you intentionally converting shorts to int and leaving the longs measure as a float?

@Minh: Yes. @Shawn: Remnant from the trim from full to minimal, fixed above now, thanks for catching it.

Updated Aug 16, 2017
New full PvR code. Now all self-contained (no need for the section in initialize), produces a summary every six months or so (126 days, set to whatever you wish) and always at the end of the run, has a fix for shorting, added pvr/day and CAGR. Info to logging window when some highs are hit is on by default.

def initialize(context):  
      for i in range(1, 391):  
        schedule_function(pvr, date_rules.every_day(), time_rules.market_open(minutes=i))

def pvr(context, data):  
    ''' Custom chart and/or logging of profit_vs_risk returns and related information  
    '''  
    import time  
    from datetime import datetime  
    from pytz import timezone      # Python will only do once, makes this portable.  
                                   #   Move to top of algo for better efficiency.  
    c = context  # Brevity is the soul of wit -- Shakespeare [for readability]  
    if 'pvr' not in c:

        # For real money, you can modify this to total cash input minus any withdrawals  
        manual_cash = c.portfolio.starting_cash  
        time_zone   = 'US/Pacific'   # Optionally change to your own time zone for wall clock time

        c.pvr = {  
            'options': {  
                # # # # # # # # # #  Options  # # # # # # # # # #  
                'logging'         : 0,    # Info to logging window with some new maximums  
                'log_summary'     : 126,  # Summary every x days. 252/yr

                'record_pvr'      : 1,    # Profit vs Risk returns (percentage)  
                'record_pvrp'     : 0,    # PvR (p)roportional neg cash vs portfolio value  
                'record_cash'     : 0,    # Cash available  
                'record_max_lvrg' : 1,    # Maximum leverage encountered  
                'record_max_risk' : 1,    # Highest risk overall  
                'record_shorting' : 0,    # Total value of any shorts  
                'record_max_shrt' : 1,    # Max value of shorting total  
                'record_cash_low' : 1,    # Any new lowest cash level  
                'record_q_return' : 0,    # Quantopian returns (percentage)  
                'record_pnl'      : 0,    # Profit-n-Loss  
                'record_risk'     : 0,    # Risked, max cash spent or shorts beyond longs+cash  
                'record_leverage' : 0,    # End of day leverage (context.account.leverage)  
                # All records are end-of-day or the last data sent to chart during any day.  
                # The way the chart operates, only the last value of the day will be seen.  
                # # # # # # # # #  End options  # # # # # # # # #  
            },  
            'pvr'        : 0,      # Profit vs Risk returns based on maximum spent  
            'cagr'       : 0,  
            'max_lvrg'   : 0,  
            'max_shrt'   : 0,  
            'max_risk'   : 0,  
            'days'       : 0.0,  
            'date_prv'   : '',  
            'date_end'   : get_environment('end').date(),  
            'cash_low'   : manual_cash,  
            'cash'       : manual_cash,  
            'start'      : manual_cash,  
            'tz'         : time_zone,  
            'begin'      : time.time(),  # For run time  
            'run_str'    : '{} to {}  ${}  {} {}'.format(get_environment('start').date(), get_environment('end').date(), int(manual_cash), datetime.now(timezone(time_zone)).strftime("%Y-%m-%d %H:%M"), time_zone)  
        }  
        if c.pvr['options']['record_pvrp']: c.pvr['options']['record_pvr'] = 0 # if pvrp is active, straight pvr is off  
        if get_environment('arena') not in ['backtest', 'live']: c.pvr['options']['log_summary'] = 1 # Every day when real money  
        log.info(c.pvr['run_str'])  
    p = c.pvr ; o = c.pvr['options'] ; pf = c.portfolio ; pnl = pf.portfolio_value - p['start']  
    def _pvr(c):  
        p['cagr'] = ((pf.portfolio_value / p['start']) ** (1 / (p['days'] / 252.))) - 1  
        ptype = 'PvR' if o['record_pvr'] else 'PvRp'  
        log.info('{} {} %/day   cagr {}   Portfolio value {}   PnL {}'.format(ptype, '%.4f' % (p['pvr'] / p['days']), '%.3f' % p['cagr'], '%.0f' % pf.portfolio_value, '%.0f' % pnl))  
        log.info('  Profited {} on {} activated/transacted for PvR of {}%'.format('%.0f' % pnl, '%.0f' % p['max_risk'], '%.1f' % p['pvr']))  
        log.info('  QRet {} PvR {} CshLw {} MxLv {} MxRisk {} MxShrt {}'.format('%.2f' % (100 * pf.returns), '%.2f' % p['pvr'], '%.0f' % p['cash_low'], '%.2f' % p['max_lvrg'], '%.0f' % p['max_risk'], '%.0f' % p['max_shrt']))  
    def _minut():  
        dt = get_datetime().astimezone(timezone(p['tz']))  
        return str((dt.hour * 60) + dt.minute - 570).rjust(3)  # (-570 = 9:31a)  
    date = get_datetime().date()  
    if p['date_prv'] != date:  
        p['date_prv'] = date  
        p['days'] += 1.0  
    do_summary = 0  
    if o['log_summary'] and p['days'] % o['log_summary'] == 0 and _minut() == '100':  
        do_summary = 1              # Log summary every x days  
    if do_summary or date == p['date_end']:  
        p['cash'] = pf.cash  
    elif p['cash'] == pf.cash and not o['logging']: return  # for speed

    shorts = sum([z.amount * z.last_sale_price for s, z in pf.positions.items() if z.amount < 0])  
    new_key_hi = 0                  # To trigger logging if on.  
    cash       = pf.cash  
    cash_dip   = int(max(0, p['start'] - cash))  
    risk       = int(max(cash_dip, -shorts))

    if o['record_pvrp'] and cash < 0:   # Let negative cash ding less when portfolio is up.  
        cash_dip = int(max(0, cash_dip * p['start'] / pf.portfolio_value))  
        # Imagine: Start with 10, grows to 1000, goes negative to -10, should not be 200% risk.

    if int(cash) < p['cash_low']:             # New cash low  
        new_key_hi = 1  
        p['cash_low'] = int(cash)             # Lowest cash level hit  
        if o['record_cash_low']: record(CashLow = p['cash_low'])

    if c.account.leverage > p['max_lvrg']:  
        new_key_hi = 1  
        p['max_lvrg'] = c.account.leverage    # Maximum intraday leverage  
        if o['record_max_lvrg']: record(MxLv    = p['max_lvrg'])

    if shorts < p['max_shrt']:  
        new_key_hi = 1  
        p['max_shrt'] = shorts                # Maximum shorts value  
        if o['record_max_shrt']: record(MxShrt  = p['max_shrt'])

    if risk > p['max_risk']:  
        new_key_hi = 1  
        p['max_risk'] = risk                  # Highest risk overall  
        if o['record_max_risk']:  record(MxRisk = p['max_risk'])

    # Profit_vs_Risk returns based on max amount actually invested, long or short  
    if p['max_risk'] != 0: # Avoid zero-divide  
        p['pvr'] = 100 * pnl / p['max_risk']  
        ptype = 'PvRp' if o['record_pvrp'] else 'PvR'  
        if o['record_pvr'] or o['record_pvrp']: record(**{ptype: p['pvr']})

    if o['record_shorting']: record(Shorts = shorts)             # Shorts value as a positve  
    if o['record_leverage']: record(Lv     = c.account.leverage) # Leverage  
    if o['record_cash']    : record(Cash   = cash)               # Cash  
    if o['record_risk']    : record(Risk   = risk)  # Amount in play, maximum of shorts or cash used  
    if o['record_q_return']: record(QRet   = 100 * pf.returns)  
    if o['record_pnl']     : record(PnL    = pnl)                # Profit|Loss

    if o['logging'] and new_key_hi:  
        log.info('{}{}{}{}{}{}{}{}{}{}{}{}'.format(_minut(),  
            ' Lv '     + '%.1f' % c.account.leverage,  
            ' MxLv '   + '%.2f' % p['max_lvrg'],  
            ' QRet '   + '%.1f' % (100 * pf.returns),  
            ' PvR '    + '%.1f' % p['pvr'],  
            ' PnL '    + '%.0f' % pnl,  
            ' Cash '   + '%.0f' % cash,  
            ' CshLw '  + '%.0f' % p['cash_low'],  
            ' Shrt '   + '%.0f' % shorts,  
            ' MxShrt ' + '%.0f' % p['max_shrt'],  
            ' Risk '   + '%.0f' % risk,  
            ' MxRisk ' + '%.0f' % p['max_risk']  
        ))  
    if do_summary: _pvr(c)  
    if get_datetime() == get_environment('end'):   # Summary at end of run  
        _pvr(c) ; elapsed = (time.time() - p['begin']) / 60  # minutes  
        log.info( '{}\nRuntime {} hr {} min'.format(p['run_str'], int(elapsed / 60), '%.1f' % (elapsed % 60)))  

Hi Gary,

I learned to like to your risk assessment and made my own PvR. Mine is a little different as it considers leverage spikes as extra money put at risked. You can even change the leverage limit to greater than 1.0 to simulate a margin account. I am posting this for anyone else who may find this useful.

Clone Algorithm
52
Loading...
Backtest from to with initial capital
Total Returns
--
Alpha
--
Beta
--
Sharpe
--
Sortino
--
Max Drawdown
--
Benchmark Returns
--
Volatility
--
Returns 1 Month 3 Month 6 Month 12 Month
Alpha 1 Month 3 Month 6 Month 12 Month
Beta 1 Month 3 Month 6 Month 12 Month
Sharpe 1 Month 3 Month 6 Month 12 Month
Sortino 1 Month 3 Month 6 Month 12 Month
Volatility 1 Month 3 Month 6 Month 12 Month
Max Drawdown 1 Month 3 Month 6 Month 12 Month
# Backtest ID: 56b6a73373637f12c08c846a
There was a runtime error.

@Blue great stuff...definitely going to incorporate it into my algos. Is there a particular range that is a good PvR value? Or just the higher the better?

Also, does calling it every minute from handle data give you much more accurate (truer?) data than calling it once at the end of the day?

I tested CAGR. It works well as far as I can tell. I added some code to make an option of recording it, took me a little while figure out why it wasn't working until I changed the number of days between summaries to 1 for daily logging. I would move the calculation out of the summary code and maybe put a logging option in for it like the other PvR items.

PvR seems better then QReturn..

But instead to calculate the return based on the "max/min value" why do not consider the "avg"?? I try to explane better with my poor english:

Usually in my algo I use these 3 record line:

record(nav=context.portfolio.portfolio_value) # total money in pocket
record(cash=context.portfolio.cash) # cash
record(port=context.portfolio.positions_value) # portfolio value

Then, visually, I compare the Qreturn with the money invested day by day (wich is rappresented by the area under the "context.portfolio.positions_value" line)
Comparing similar Qreturn, i prefer the one with little "port" and equivalent large "cash"; this means more money avaible for other algos/investment.

In others world, if in 1 year my 10%Return algo use 20% only of the money avaible in the pocket, this algo is 5 time better (not exactly, of course), for me, then another algo which return the same 10% but using 100% of my pocket.

fiuuu..very difficult in english :))

Then, to conclude, because I'm not a programmer, is there a way to calculate the area under pocket/cash/position records and then to have this numerically and not just visually?
Thanks for the patience...

Just noticed the comments above. Yes the call does need to be in handle_data. Leverage and risk for example often hit peaks and fall back by end of day. Anyone's welcome to click my name to send me a direct message too (I'm likely to see it right away). For now I'd just recommend to work with it, modifying to implement your ideas.

The large text block above is being updated, last one was yesterday (noted at the top of that post). I think PvR is quite a bit better now than when it started.

I was asked to provide some guidance about this tool, what values are good and things to watch out for etc.

Adding PvR to existing code Often, I clone new algorithms and add the PvR code above, first copying their backtest code into an editor, then commenting out any record() because the custom chart is limited to five items, also sometimes commenting out any log.info lines. Then find handle_data if it exists and replace just that def line with all of the code above. You get the idea. The call to pvr() can be anywhere in handle_data as long as it isn't after any return. so it might as well be the first line, doesn't hurt. Back to the backtester, ctrl-v to paste and ctrl-b to run it. Sometimes that whole trip might take just 15 seconds I think.

What it means Mainly, the first value like PvR 0.1230 %/day is a good gage of code merit, it neutralizes the differing amounts of calendar time in backtests (although I'm starting to warm up to CAGR added recently even though I don't understand it yet). I pay a lot of attention to Cash Low (CshLw) because the closer it is to zero the more accurate the UI metrics are, like Sharpe. And also Returns of course.

Some routes/suggestions/ideas for keeping track of info

  • At the end of the run, copy the last set of PvR info in from logging window along with run time (plus the line with start/stop dates, capital, and today's calendar date/time) to the top of the algo using three comment markers like this:
'''
2017-02-21 13:00 _pvr:209 INFO PvR 0.7512 %/day   cagr 0.5   Portfolio value 13315297   PnL 12315297  
2017-02-21 13:00 _pvr:210 INFO   Profited 12315297 on 1062492 activated/transacted for PvR of 1159.1%  
2017-02-21 13:00 _pvr:211 INFO   QRet 1231.53 PvR 1159.10 CshLw -62492 MxLv 1.01 RskHi 1062492 MxShrt 0  
2017-02-21 13:00 pvr:299 INFO 2011-01-04 to 2017-02-21  $1000000  2017-02-24 21:17 US/Eastern  
Runtime 0 hr 8.9 min  
'''
  • If %/day looks high, copy its value to the backtest title. Later, when looking thru numerous backtests you can find those that did best using that.
  • Copy the originating URL from the forums to the top of the code.
  • If %/day is very high, copy the algo to an editor and save it locally to a folder designated solely for keeping those. Then you can sort them high to low. Mine range from .03 to .75 for other's code that I have saved. Then some of my own are above 2.0. I use PvR so I can see what's going on to wind up with better strategies. (Frankly, some that are best were discovered via mistakes I made and I'm still scratching my head over some of them as to why they even work so well, their PvR is great so I have to accept that, I would still be lost without PvR, my Robinhood code is up over 100% in less than three months).
  • Also in the filename, you can copy metrics from the browser and run a macro on those, even including the paste and copy back to the clipboard (I use Notepad++), to wind up with a string like r1231.5 a0.42 b0.54 s1.35 d-33.7 for returns, alpha etc. So an entire filename might be:
    0.7512 r1231.5 a0.42 b0.54 s1.35 d-33.7.py
    That string can also go to the top of the backtest code and/or backtest title.
  • Above, that result of PvR 0.7512 %/day is from SPY who loved WVF.... Here are some others that were high among those I have saved and you can look these up (they are up to .34 %/day): 'worthy of Q fund?', 'etf market rotation strategy', 'How to Build a Pairs Trading Strategy on Quantopian', 'For Robinhood trading', 'minimum variance with constraint'. I've unfortunately lost track of some good code by others from not being disciplined enough to save them all locally.

Those are some ideas with a little more overview on this tool. It's not just mine, make it your own. While maybe not perfect, I'd be lost without PvR.

How about a toast. To your wealth! :) Use PvR to see clearly.

Sorry,still confused for PvR values.

Could summarize clear simple guidelines how to choose backtest strategy through PvR output, would be appreciated.!

Hi Blue,

Do we need to adjust something when the algo does short only. It looks like the pvr code is not proving the right metrics. Can you please shed some light?

Here is the result:
2017-04-06 22:00 _pvr:120 INFO PvR 0.0000 %/day cagr 0.5 Portfolio value 40797 PnL 30797
2017-04-06 22:00 _pvr:121 INFO Profited 30797 on 0 activated/transacted for PvR of 0.0%
2017-04-06 22:00 _pvr:122 INFO QRet 307.97 PvR 0.00 CshLw 10000 MxLv 1.41 RskHi 0 MxShrt -41912
2017-04-06 22:00 pvr:208 INFO 2014-01-02 to 2017-04-06 $10000 2017-04-11 06:20 US/Eastern

Thanks!

Thanks for catching that bug. Since shorts are a negative value, this line should have had the minus sign:

risk         = int(max(cash_dip, -shorts))  

The latest version had that fix, updated now in the message above with the full code (Jan 14), quite a few changes for speed.

For the starting post, you wrote "For best speed, replace c. with context. and remove c = context" in regards to the minimal pvr code.
Should I do the same for the full code from your post on 1/14/16?

Yep. Meanwhile to find out for sure how much difference in speed, could use this: https://www.quantopian.com/posts/timing-code

This could use an upgrade regarding short risk.

Overview: The main benefit of this pvr tool may be freedom for making changes during development to neutralize both cash usage and changes in shorting (the other risk), so up or down returns from a change in pipeline factors or ordering or whatever can be compared with certainty even when risk is not the same.

Rationale: Calculates a profit per dollar on the maximum risk, as both cash dip (including into margin) and max short value. An investor wants to know output vs input. The input is termed "risk" here. An assumption is made that quants will write code to use 100% of the initial capital but the reality of software development is, that's a lot more easily said than done, it's impossible to maintain always, and the process of getting to that goal can be messy. For example, add limit orders and suddenly your perfect 1.0 leverage is down to .62 max and avg .41. This pvr metric means leverage changes like that are also not such an immediate worry.

The upgrade needed: Uses short value at 100% of the shares value and could be punishing shorting too much, I understand that shorting can be done at 4x longs plus cash. A short margin call would only happen if the original short value falls below that. Both are changing of course. Shorting at 3x means wiggle-room. Higher long value allows for a greater short value dip and visa-versa. If someone who knows shorting well could be inclined to roll up their sleeves and write some code to handle shorting risk better, please do, I would send a limousine and helicopter at your disposal if I could.

If you want to tackle this, rather than online back and forth in rather tough task, ping me here, antispam, gmail and take the space out of this user, gary hawk, that's it.

I wonder if PvR is (simplifying) the Return/(abs(Max-Min) $ range invested) where Max-Min are the highest peaks (positive and negative) on the whole algo time period. Right?

Ideally from the investment standpoint, how much of my money is risked. The question: What constitutes risk? It's a discussion worth having. The Jan 14, 2016 version matters most. This line maybe could be improved.

risk       = int(max(cash_dip, -shorts))  

In the beginning of this post you note the limit of the Qreturn:
"The returns calculation employs a philosophy that all of your starting capital is at risk as soon as you open the account..It calculates returns based on starting capital regardless of how much you actually activated..." I totally agree with you.

But I have a similar doubt about PvR, but I'm not a programmer then please apologize if I'm wrong..

Just imagine to run a 10 years algo (252 x 10 days) wich invest only 10% of the starting capital for 2519 days (with great return) and 200% for 1 day only (cause to a bug, not expected behaviour, or some dirty data). The Pvr value will be orrible and you will discard the algo...doing a mistake..

I'm wrong?

Thanks for your time and for your routine wich is very preciuos to me!

Right, and I've seen those scenarios too, although instead of discarding the algo I'd say it could just make the margin problem visible in the custom chart to be able to address it. PvRp (proportional) above was for that type of thing, maybe take a look and you might find a way to modify and use it, but I discontinued using that myself. What I was focused on at the time was margin early on that could be like 200% of the portfolio and yet not much compared to the ending value with a lot of gain over time, to avoid a harsh penalty from early margin.

In the contest and fund with recent leverage limits near 1, the Quantopian returns and PvR are always closer to each other now versus the once-upon-a-time idea floated of maybe up to 6x leverage applied to fund algos, This profit per dollar risked value was a bit more of thing back then because by neutralizing margin it could for example make it clear that a 1.1x algo was actually better value even if a 6x appeared to be, providing the apples-to-apples comparison. (Then there's leverage from shorting of course, all a rather complex picture).

Toward the investor two most basic concerns: How much in, how much out.

On what constitutes risk, with margin and shorting the brokerage has certain requirements. For improvements I'd have to invite those with experience in them including maintenance margin requirements. But just thinking out loud, is risk simply abs(short_value) + long_value, the leverage equation numerator? Maybe try this ...

    shorts = sum([z.amount * z.last_sale_price for s, z in pf.positions.items() if z.amount < 0])  
    longs  = sum([z.amount * z.last_sale_price for s, z in pf.positions.items() if z.amount > 0])  
    risk   = abs(shorts) + longs       # from this current risk, max risk is then used, for profit per max risk  

If I could wave a magic wand I'd apply this on the top ~50 of each past contest, in place of returns as the basis for metrics calculations, hence scoring, and different algos would rise to the top. Then see if it is a better or worse evaluator/predictor using out-of-sample. Would the winners using this have performed better? Surely a test worth considering.

Glad you find this useful. I feel lost without it. :/

Here's a version that separates out the charting, scheduled to end of day aiming for efficiency. And uses abs(shorts) + longs. Lv is set to 1.2.

Clone Algorithm
17
Loading...
Backtest from to with initial capital
Total Returns
--
Alpha
--
Beta
--
Sharpe
--
Sortino
--
Max Drawdown
--
Benchmark Returns
--
Volatility
--
Returns 1 Month 3 Month 6 Month 12 Month
Alpha 1 Month 3 Month 6 Month 12 Month
Beta 1 Month 3 Month 6 Month 12 Month
Sharpe 1 Month 3 Month 6 Month 12 Month
Sortino 1 Month 3 Month 6 Month 12 Month
Volatility 1 Month 3 Month 6 Month 12 Month
Max Drawdown 1 Month 3 Month 6 Month 12 Month
# Backtest ID: 5a557ff1fa159c3fdb5b877c
There was a runtime error.

Blue!! Is Always very interesting to read your posts!

On my view are necessay different parameters to evaluate an algo but, some are MORE important then others:

Just imagine an algo wich invest all the starting capital just one day a month; and the Alpha extract is less then another algo wich is all days invested. All the backtest parameters in Q tell me that the second one is better the first one. But on my opinion is NOT.

again following the example of my previous post

Just imagine to sum the 10% of starting capital spended each of the 2519 days + the 200% of 1 only day (of starting capital), and to divide that sum by 2520 (days we are invested): now we have a "correct" weight of the $ amount invested (in daily frame). Now we can compare that value with the algo Return. Thats the way I usually do.

Honestly I do that calculation in a approximativy and intuitivly way, I'm not a programmer, just "lucky" hobbist not "yet" able to modify your routine..

Hi Blue,

Really appreciate all of your efforts with the sample coding. I tried to implement the short hand PVR code snipet you've provided above. Can you please provide some color as to how I can interpret my results?

BackTest run: 6/30/2018 - 7/30/2018 $10,000,000 Initial Capital

As of 7.30.2018
Lv = 1.03
MxLV = 1.10
PvRAvg = 15.66
PvR = 27.52

Thank you.

Lv -- leverage at the end of each day in the custom chart (keep in mind that record is daily resolution)
MxLv -- the maximum leverage ever seen in the backtest (keeps track by watching every minute all throughout).
PvRAvg -- Average PvR throughout the run, intended to be used for comparing algorithms to each other. Added as an afterthought and I don't use that very often.
PvR -- Profit vs Risk says: If the initial capital had been all invested without going over, the returns would be this value. So, if only half of the initial capital is invested, PvR will be two times the value seen in the chart for Returns. Inversely, if initial capital is $100 but $200 is put at risk (as margin and/or excess shorting), since it went beyond the original $100, PvR will be just half of the Returns value shown.

It's a way to keep track of actual profitability. If one were to start with $10 and buy a share of SPY every day, that would show fantastic returns but would be due to margin. PvR would show returns as if that margin had not happened (if starting capital matched the amount actually utilized). Makes cloning/evaluating algos a snap with no need to manually adjust for unused cash or margin or shorting spikes.

I like the percent per day value, see PvR 0.7512 %/day above from the end of the run. It could also be charted. No matter how long various backtests run, they can be saved locally copying that number for the first thing in file names and then sorting later to find algos run previously that did best. Can use an abbreviation for pvr percent per day like pd0.7512_and_the_rest_of_the_filename.py Some of mine from the past (except a more disciplined consistency in filenames would be recommendable):

ppd0.5587.py  
ppd1.1018 p555 2 yr ver 3 liq work dump worst.py  
ppd1.6888 p851 ver 3 liq work etc.py  
ppd2.3997 the 44 million.py  
ppd3.0550.py  

Or drop the ppd and start the filenames with just the numbers. A filename can be constructed automatically for copy/paste from the end of the run with a little work. I use some code like that in a tool more extensive.
0.0688_p41_q19_b0.18_d6_2015-01-05_to_2017-05-22_10M.py
p41 is PvR. Since it is higher than the Q returns q19, that indicates not all of this 10M context.portfolio.starting_cash was invested.
Beta b0.18 and Drawdown d6 (percent)