Back to Community
A Carry Strategy with Portfolio Optimization

Attached below is a traditional carry strategy as follows:

  1. At the end of each calendar quarter (Dec, Mar, June, Sept), sort all dividend yields on stocks with more than 2 billion market cap for liquidity constraint (T12M) (Wish I could of got forward Div Yield but the data's unavailable for some reason).
  2. Trim off the first 3 and last 3 stocks to avoid any data bias (e.g RBS was a stock with apparently reported 103% dividend yield for a while)
  3. Estimate covariance matrix using Ledoit-Wolf Shrinkage (https://repositori.upf.edu/bitstream/handle/10230/560/691.pdf?sequence=1)
  4. Optimize the portfolio such that it is long only (Max 3% in any stock to ensure diversification) and maximizes the follow equation:
    W' * Dividend Yield - (Lambda)*W' * Covariance Matrix * W
    where Lambda is an adjustable risk aversion coefficient (More risk averse, more priority on minimizing risk), W is a vector of stock weights.
  5. Re-balance each quarter

Return_spread is a plot of cum. return of the portfolio - SPY cum return.

Some Notes on this strategy:

  1. The strategy generates steady alpha other than the period during the recession and for the past 2-3 years (The cum. spread flattens).
  2. I have yet to contrast this with a more generic weighting method (e.g dogs of dow strategy where we just equal weight the stocks) or sorts or risk parity.
  3. Long-short tends to perform poorly due to excessive leverage and creating a leverage constraint freezes the optimizer (anyone want to give this a shot?) (My constraint was SUM(ABS(W))/SUM(W) < c where c is some constant e.g 3)
  4. Overall, long-short sucks due to periods of excessive leverage, long works but has beta. Alpha sometimes were generated due to luck and there are periods where premia is zero or negative.
Clone Algorithm
232
Loading...
Backtest from to with initial capital
Total Returns
--
Alpha
--
Beta
--
Sharpe
--
Sortino
--
Max Drawdown
--
Benchmark Returns
--
Volatility
--
Returns 1 Month 3 Month 6 Month 12 Month
Alpha 1 Month 3 Month 6 Month 12 Month
Beta 1 Month 3 Month 6 Month 12 Month
Sharpe 1 Month 3 Month 6 Month 12 Month
Sortino 1 Month 3 Month 6 Month 12 Month
Volatility 1 Month 3 Month 6 Month 12 Month
Max Drawdown 1 Month 3 Month 6 Month 12 Month
import pandas
import numpy as np
from scipy import optimize
from sklearn.covariance import LedoitWolf

def initialize(context):
    schedule_function(trade, 
                      date_rule=date_rules.month_end(), 
                      time_rule=time_rules.market_close(), 
                      half_days=False)
    context.n = 300 # screened stocks number
    context.lookback = 252 # 252*n = n years
    context.bmark = symbol('SPY')
    context.bmarkp = 0 # Placeholder for later

def before_trading_start(context):
    #log.info("Cash Position: "+str(context.portfolio.cash))
    if (get_datetime().month % 3) == 0:
        dvd = get_fundamentals(query(fundamentals.valuation_ratios.dividend_yield)  
                           .filter(fundamentals.valuation.market_cap >= 2000000000)
                           # Ensures that we get large market cap. thus liquid stocks
                           .order_by(fundamentals.valuation_ratios.dividend_yield.desc()).limit(context.n)
                          ).T
        # Remove the top three and bottom three to control outliers
        dvd = dvd.iloc[3:-3]
        update_universe(dvd.index)
        context.dvdy = dvd
        
def trade(context,data):
    if (get_datetime().month%3)==0:
        log.info(context.account.leverage)
        w = getCarried(context,data)
    
        # Close all current positions
        for stock in context.portfolio.positions:
            if stock not in w.index:
               order_target(stock, 0)  
        
        #Order
        for i,n in enumerate(w):
            if w.index[i] in data:
                if w.index[i] in context.portfolio.positions: #If position already exists
                    order_target_percent(w.index[i], n)
                else:
                    order_percent(w.index[i], n)
    
def getCarried(context,data):
        # Structure a portfolio such that it maximizes dividend yield while lowering volatility
        # Covar estimation
        price_history = history(bar_count=context.lookback, frequency="1d", field='price')
        if(context.bmarkp == 0):
            context.bmarkp = data[context.bmark].close_price
        price_history = price_history[context.dvdy.index] # Fix the +1 issue of extra stock
        ret = (price_history/price_history.shift(1)-1)[1:].dropna(axis=1)
        n = len(ret.columns)
        context.dvdy = context.dvdy.loc[ret.columns] # subset that removes all na columns
        #OAS
        covar = LedoitWolf().fit(ret).covariance_
        
        #Constant Correlation Model
        #corr = ccmodel(ret, context, n)
        #std = np.dot(pandas.DataFrame(ret.std()),pandas.DataFrame(ret.std()).T)
        #covar = pandas.DataFrame(corr*std) # Rho %*% (sigma%*%sigma^T)
        
        # sample estimate
        #covar = ret.cov() # Normal Cov
        
        # Set Solver Constraint
        cons = (
             {'type': 'eq', 'fun': lambda x: 1-sum(x)},
             {'type': 'ineq', 'fun': lambda x: 0.03-x}, # 3 % at mo
             {'type': 'ineq', 'fun': lambda x: x},
             #{'type': 'ineq', 'fun': lambda x: 50-np.sqrt(sum(pow(x,2)))/sum(x)} # Leverage Constraint
        ) 
        ra = 5000 # Risk Aversion Coefficient
        w = optimize.minimize(objDivTradeOff, # Minimize so -1 makes it maximize (see below)
                                    x0=np.array([1/n]*n), #Initial guess of 1/N
                                    args=(context.dvdy,covar,ra), # Pass in arguments
                                    method='SLSQP', jac=False, #Jacobian vector
                                    constraints=cons, # constraints set as above
                                    options=({'maxiter': 1e4}))   #Ensure convergence
        log.info('Expected Dividend Yield: '+str(np.dot(w.x,context.dvdy)*100)+'%')
        log.info('Expected Volatility: '+str(np.sqrt(np.dot(w.x.T,np.dot(covar,w.x)))*100)+'%')
        w = pandas.Series(np.round(10000*w.x)/10000)
        log.info('Number of Stocks: '+str(sum(abs(w)>0)))
        w.index = ret.columns
        return w  

def objDivTradeOff(x,divyield,covar,ra):
    return -1*(np.dot(x,divyield)-ra*np.dot(x.T,np.dot(covar,x))) # W'DivYield - lambda * W'CovarW

def jacDivTradeOff(x,divyield,covar,ra):
    return -1*(divyield - ra*2*np.dot(covar,x)) #First order derivative 

def handle_data(context, data):
    # Do custom record
    if (context.bmarkp != 0):
        spread = context.portfolio.returns - (data[context.bmark].close_price/context.bmarkp - 1)
        record(return_spread=spread)
    pass

def ccmodel(ret, context, n):
    # Assuming T x N Matrix
    norm = n*(n-1)/2
    m = sum(sum(np.triu(ret.corr(),k=1)))/norm
    a = pandas.DataFrame(np.array([m]*(n*n)).reshape(n,n), index=ret.columns, columns=ret.columns)
    for i in np.arange(n):
        a.iloc[i,i] = 1
    return a
There was a runtime error.
9 responses

Thanks for posting, soooo much to learn...

Good stuff, lots of time and thought that went into that work.
You're already aware that you have leverage in the sense of margin (negative cash, borrowing), max on 2008-10-10.
Keep in mind that the chart doesn't reflect that, the code is still positive, just less, ~165%.
Best of luck.

What is the source of the spike in June 2010? Looks a bit like an aberration...

@Simon, tbh, I went to the exact dates and i couldn't pin down one asset with an extreme rise in price, etc. If anyone would like to clone and let me know, that'd be appreciated.

@garyha, can you show me how it was levered in 2008? I just went over the logs and at each quarter, i logged the account leveraged and never seen anything > 2

Looking at the same thing it appears that on one day between 10/6/10 and 11/6/10 the value of the stocks changes ~ +$25k but the cash balance goes from negative $435k to positive $169k. There are no transactions logged in the period - very strange.

@garyha.. pls. print out.. the anomaly.. so that evryone will be aware if the algo is not performing properly as intended. thanks

In this backtest I added start and stop dates to track_orders() because I think the logging window would otherwise reach its hard limit, and chose to hone in on the date ranges 2007-05-07 to 2008-02-13 and 2010-04-26 to 2010-11-15, they looked sort of interesting in the custom chart. The code for adjusting them looks like this:

    # track_orders() dates to start or stop logging.  
    #   This is to target a date range and not overwhelm the logging window.  
    #   These lists can be empty like []  
    context.dates  = {  
        'active': 0,  
        'start' : ['2007-05-07', '2010-04-26'],  
        'stop'  : ['2008-02-13', '2010-11-15']  
    }  

I missed earlier calls for my attention unfortunately.
In the output there are tons of unfilled orders for awhile, maybe part of the problem if that's an unusual number of them.
The first number in that line is supposed to be the trading minute (1 - 390) and was working in another algo earlier today, 630 is out of range so I don't know what's up with that. If someone wants to fix it, be my guest please. Edit: That's only in daily mode and the problem is taken care of with the latest update at Track Orders.

2006-09-29trade:135INFO1.00215015566  
2006-09-29getCarried:196INFONumStcks, Exp. DivYd, Vy  55 6.31 0.30  
2006-12-29trade:135INFO1.03043371783  
2006-12-29getCarried:196INFONumStcks, Exp. DivYd, Vy  53 6.68 0.27  
2007-03-30trade:135INFO1.03275110091  
2007-03-30getCarried:196INFONumStcks, Exp. DivYd, Vy  49 5.73 0.28  
2007-05-07track_orders:73INFO 630   Sell -242 CRE at 44.74   cash -72739  
2007-05-07track_orders:73INFO 630   Sell -242 CRE at 44.74   cash -72739  
2007-05-07track_orders:73INFO 630   Sell -242 CRE at 44.74   cash -72739  
2007-05-07track_orders:73INFO 630   Sell -10 GTM at 42.0   cash -72739  
2007-05-07track_orders:73INFO 630   Sell -225 GTM at 42.0   cash -72739  
2007-05-07track_orders:73INFO 630   Sell -225 GTM at 42.0   cash -72739  
2007-05-07track_orders:73INFO 630   Sell -225 GTM at 42.0   cash -72739  
2007-05-07track_orders:73INFO 630   Sell -225 GTM at 42.0   cash -72739  
2007-05-07track_orders:73INFO 630   Sell -225 GTM at 42.0   cash -72739  
2007-05-07track_orders:73INFO 630   Sell -225 GTM at 42.0   cash -72739  
2007-05-07track_orders:73INFO 630   Sell -225 GTM at 42.0   cash -72739  
2007-05-07track_orders:73INFO 630   Sell -225 GTM at 42.0   cash -72739  
2007-05-07track_orders:73INFO 630   Sell -225 GTM at 42.0   cash -72739  
2007-05-07track_orders:73INFO 630   Sell -225 GTM at 42.0   cash -72739  
2007-05-07track_orders:73INFO 630   Sell -348 TLD at 35.37   cash -72739  
2007-05-07track_orders:73INFO 630   Sell -348 TLD at 35.37   cash -72739  
2007-05-07track_orders:73INFO 630   Sell -348 TLD at 35.37   cash -72739  
2007-05-07track_orders:73INFO 630   Sell -348 TLD at 35.37   cash -72739  
2007-05-07track_orders:73INFO 630   Sell -1098 GEMP at 3.38   cash -72739  
2007-05-07track_orders:73INFO 630   Sell -2022 PA at 24.99   cash -72739  
2007-05-07track_orders:73INFO 630   Sell -2022 PA at 24.99   cash -72739  
2007-05-07track_orders:73INFO 630   Sell -2022 PA at 24.99   cash -72739  
2007-05-08track_orders:49INFO 630         GTM -225 unfilled  
2007-05-08track_orders:49INFO 630         CRE -242 unfilled  
2007-05-08track_orders:49INFO 630         GTM -225 unfilled  
2007-05-08track_orders:49INFO 630         TLD -348 unfilled  
2007-05-08null:nullWARNLogging limit exceeded; some messages discarded  
2007-05-16track_orders:49INFO 630         GTM -225 unfilled  
2007-05-16track_orders:49INFO 630         CRE -242 unfilled  
2007-05-16track_orders:49INFO 630         GTM -225 unfilled  
2007-05-16track_orders:49INFO 630         TLD -348 unfilled  
2007-05-16track_orders:49INFO 630         GTM -225 unfilled  
2007-05-16track_orders:49INFO 630         GEMP -1098 unfilled  
2007-05-16track_orders:49INFO 630         GTM -225 unfilled  
2007-05-16track_orders:49INFO 630         GTM -225 unfilled  
2007-05-16track_orders:49INFO 630         GTM -225 unfilled  
2007-05-16track_orders:49INFO 630         GTM -225 unfilled  
2007-05-16track_orders:49INFO 630         TLD -348 unfilled  
2007-05-16track_orders:49INFO 630         CRE -242 unfilled  
2007-05-16track_orders:49INFO 630         PA -2022 unfilled  

My previous message in this thread had said ~165%, that's PvR in the chart. Here, it is 132%. There had been an upgrade and/or I introduced a bug.
I had mentioned a high in 2008, it's a bit different now.
Although the code is undoubtedly onto something that is useful and affects us all, it has been evolving. I'm actually doubting the specifics in the calculations and would love to have extra eyes on it from some of you and the fine folks at Quantopian. Maybe instead of RiskHi, the PvR value should be purely based on max_leverage. Then the shorts business could be dropped. The context.account.leverage value I think is definitely trustworthy (the only problem with it is when we try to use it in record() for a minute-level backtest and are missing 389 out of the 390 minutes of the trading day), so basing everything on that would simplify.

In the custom chart you can click legend items to toggle them off/on and bring the ones you're interested in, into range. In doing that, turning the large value items off and focusing on MaxLv, I see there is a jump early in 2003 so it might be handy to reset track_orders() to a date range there, or just remove the dates, make those empty lists. The logging window maybe would hit its limit however it doesn't matter if you are working just a few months in to the run because you might even have the end date set real early.

Clone Algorithm
14
Loading...
Backtest from to with initial capital
Total Returns
--
Alpha
--
Beta
--
Sharpe
--
Sortino
--
Max Drawdown
--
Benchmark Returns
--
Volatility
--
Returns 1 Month 3 Month 6 Month 12 Month
Alpha 1 Month 3 Month 6 Month 12 Month
Beta 1 Month 3 Month 6 Month 12 Month
Sharpe 1 Month 3 Month 6 Month 12 Month
Sortino 1 Month 3 Month 6 Month 12 Month
Volatility 1 Month 3 Month 6 Month 12 Month
Max Drawdown 1 Month 3 Month 6 Month 12 Month
import pandas
import numpy as np
from scipy import optimize
from sklearn.covariance import LedoitWolf
from pytz import timezone    # for track_orders()

def track_orders(context, data):  # Log orders created or filled.
    # https://www.quantopian.com/posts/track-orders
    #   modified here to log minute and cash

    #if 'orders' not in context:
    #    context.orders = {}

    '''
    To not overwhelm the logging window, start/stop dates can be entered in initialize()
    Example
    context.dates  = {
        'active': 0,
        'start' : ['2007-05-07', '2010-04-26'],
        'stop'  : ['2008-02-13', '2010-11-15']
    }
    '''
    if context.dates['start']:         # If the list has something in it
        date = str(get_datetime().date())
        if   date in context.dates['start']:    # See if there's a match to start
            context.dates['active'] = 1
        elif date in context.dates['stop']:     #   ... or to stop
            context.dates['active'] = 0
    else:
        context.dates['active'] = 1    # Set to active b/c no conditions

    if context.dates['active'] == 0:
        return    # Skip if off

    to_delete = []
    for id in context.orders:
        o   = get_order(id)
        sec = o.sid
        sym = sec.symbol
        bar_dt = get_datetime().astimezone(timezone('US/Eastern'))
        minute = (bar_dt.hour * 60) + bar_dt.minute - 570  # (-570 = 9:30a)
        if o.filled:        # Filled at least some, status 1 is Filled
            trade = 'Bot' if o.amount > 0 else 'Sold'
            log.info(' {}      {} {} {} at {}   cash {}'.format(minute,
                    trade, o.filled, sym, data[sec].price, int(context.portfolio.cash)))
            to_delete.append(o.id)
        else:
            log.info(' {}         {} {} unfilled'.format(
                    minute, o.sid.symbol, o.amount))

    for sec, oo_for_sid in get_open_orders().iteritems(): # Open orders
        sym = sec.symbol
        for o in oo_for_sid: # Orders per security
            bar_dt = get_datetime().astimezone(timezone('US/Eastern'))
            minute = (bar_dt.hour * 60) + bar_dt.minute - 570  # (-570 = 9:30a)
            if o.id in to_delete:
                continue
            if o.status == 2:                 # Cancelled
                log.info('    Cancelled {} {} order'.format(
                        trade, o.amount, sym, data[sec].price))
                to_delete.append(o.id)
            elif o.id not in context.orders:  # New
                context.orders[o.id] = 1
                trade = 'Buy' if o.amount > 0 else 'Sell'
                if o.limit:        # Limit order
                    log.info('   {} {} {} now {} limit {}'.format(
                            trade, o.amount, sym, data[sec].price, o.limit))
                elif o.stop:       # Stop order
                    log.info('   {} {} {} now {} stop {}'.format(
                            trade, o.amount, sym, data[sec].price, o.stop))
                else:              # Market order
                    log.info(' {}   {} {} {} at {}   cash {}'.format(minute,
                        trade, o.amount, sym, data[sec].price, int(context.portfolio.cash)))
    for d in to_delete:
        del context.orders[d]

def initialize(context):
    schedule_function(trade,
                      date_rule=date_rules.month_end(),
                      time_rule=time_rules.market_close(),
                      half_days=False)
    context.n = 300 # screened stocks number
    context.lookback = 252 # 252*n = n years
    context.bmark = symbol('SPY')
    context.bmarkp = 0 # Placeholder for later

    # for track_orders()
    context.orders = {}

    # track_orders() dates to start or stop logging.
    #   This is to target a date range and not overwhelm the logging window.
    #   These lists can be empty like []
    context.dates  = {
        'active': 0,
        'start' : ['2007-05-07', '2010-04-26'],
        'stop'  : ['2008-02-13', '2010-11-15']
    }

    #schedule_function(pvr, date_rules.every_day(), time_rules.market_close())

    # for pvr()
    c = context
    c.max_lvrg = 0
    c.risk_hi  = 0
    c.date_prv = ''
    c.cash_low = c.portfolio.starting_cash
    c.date_end = str(get_environment('end').date())
    log.info('{} to {}  {}  {}'.format(str(get_datetime().date()), c.date_end,
        int(c.cash_low), get_environment('data_frequency')))

def handle_data(context, data):
    track_orders(context, data)
    pvr(context, data)

    # Do custom record,    off using '0 and' to make room for pvr()
    if    0 and     (context.bmarkp != 0):
        spread = context.portfolio.returns - (data[context.bmark].close_price/context.bmarkp - 1)
        record(return_spread=spread)

def before_trading_start(context, data):
    #log.info("Cash Position: "+str(context.portfolio.cash))
    if (get_datetime().month % 3) == 0:
        dvd = get_fundamentals(query(fundamentals.valuation_ratios.dividend_yield)
                           .filter(fundamentals.valuation.market_cap >= 2000000000)
                           # Ensures that we get large market cap. thus liquid stocks
                           .order_by(fundamentals.valuation_ratios.dividend_yield.desc()).limit(context.n)
                          ).T
        # Remove the top three and bottom three to control outliers
        dvd = dvd.iloc[3:-3]
        update_universe(dvd.index)
        context.dvdy = dvd

def trade(context,data):
    if (get_datetime().month%3)==0:
        log.info(context.account.leverage)
        w = getCarried(context,data)

        # Close all current positions
        for stock in context.portfolio.positions:
            if stock not in w.index:
               order_target(stock, 0)

        #Order
        for i,n in enumerate(w):
            if w.index[i] in data:
                if w.index[i] in context.portfolio.positions: #If position already exists
                    order_target_percent(w.index[i], n)
                else:
                    order_percent(w.index[i], n)

def getCarried(context,data):
        # Structure a portfolio such that it maximizes dividend yield while lowering volatility
        # Covar estimation
        price_history = history(bar_count=context.lookback, frequency="1d", field='price')
        if(context.bmarkp == 0):
            context.bmarkp = data[context.bmark].close_price
        price_history = price_history[context.dvdy.index] # Fix the +1 issue of extra stock
        ret = (price_history/price_history.shift(1)-1)[1:].dropna(axis=1)
        n = len(ret.columns)
        context.dvdy = context.dvdy.loc[ret.columns] # subset that removes all na columns
        #OAS
        covar = LedoitWolf().fit(ret).covariance_

        #Constant Correlation Model
        #corr = ccmodel(ret, context, n)
        #std = np.dot(pandas.DataFrame(ret.std()),pandas.DataFrame(ret.std()).T)
        #covar = pandas.DataFrame(corr*std) # Rho %*% (sigma%*%sigma^T)

        # sample estimate
        #covar = ret.cov() # Normal Cov

        # Set Solver Constraint
        cons = (
             {'type': 'eq', 'fun': lambda x: 1-sum(x)},
             {'type': 'ineq', 'fun': lambda x: 0.03-x}, # 3 % at mo
             {'type': 'ineq', 'fun': lambda x: x},
             #{'type': 'ineq', 'fun': lambda x: 50-np.sqrt(sum(pow(x,2)))/sum(x)} # Leverage Constraint
        )
        ra = 5000 # Risk Aversion Coefficient
        w = optimize.minimize(objDivTradeOff, # Minimize so -1 makes it maximize (see below)
                                    x0=np.array([1/n]*n), #Initial guess of 1/N
                                    args=(context.dvdy,covar,ra), # Pass in arguments
                                    method='SLSQP', jac=False, #Jacobian vector
                                    constraints=cons, # constraints set as above
                                    options=({'maxiter': 1e4}))   #Ensure convergence
        #log.info('Expected Dividend Yield: '+str(np.dot(w.x,context.dvdy)*100)+'%')
        #log.info('Expected Volatility: '+str(np.sqrt(np.dot(w.x.T,np.dot(covar,w.x)))*100)+'%')
        # To save space in logging window, one line
        divyld = np.dot(w.x,context.dvdy)[0] * 100
        vltlty = np.sqrt(np.dot(w.x.T,np.dot(covar,w.x))) * 100

        w = pandas.Series(np.round(10000*w.x)/10000)
        #log.info('Number of Stocks: '+str(sum(abs(w)>0)))

        log.info('NumStcks, Exp. DivYd, Vy  {} {} {}'.format(
                str(sum(abs(w)>0)), '%.2f' % divyld, '%.2f' % vltlty ))

        w.index = ret.columns
        return w

def objDivTradeOff(x,divyield,covar,ra):
    return -1*(np.dot(x,divyield)-ra*np.dot(x.T,np.dot(covar,x))) # W'DivYield - lambda * W'CovarW

def jacDivTradeOff(x,divyield,covar,ra):
    return -1*(divyield - ra*2*np.dot(covar,x)) #First order derivative

def ccmodel(ret, context, n):
    # Assuming T x N Matrix
    norm = n*(n-1)/2
    m = sum(sum(np.triu(ret.corr(),k=1)))/norm
    a = pandas.DataFrame(np.array([m]*(n*n)).reshape(n,n), index=ret.columns, columns=ret.columns)
    for i in np.arange(n):
        a.iloc[i,i] = 1
    return a

def pvr(context, data):
    ''' Custom chart and/or log of profit_vs_risk returns and related information
    '''
    # # # # # # # # # #  Options  # # # # # # # # # #
    record_max_lvrg = 1          # Maximum leverage encountered
    record_leverage = 0          # Leverage (context.account.leverage)
    record_q_return = 0          # Quantopian returns (percentage)
    record_pvr      = 1          # Profit vs Risk returns (percentage)
    record_pnl      = 0          # Profit-n-Loss
    record_shorting = 1          # Total value of any shorts
    record_risk     = 0          # Risked, maximum cash spent or shorts in excess of cash at any time
    record_risk_hi  = 1          # Highest risk overall
    record_cash     = 0          # Cash available
    record_cash_low = 1          # Any new lowest cash level
    logging         = 1          # Also log to the logging window conditionally (1) or not (0)
    log_method      = 'risk_hi'  # 'daily' or 'risk_hi'

    c = context                          # For brevity
    new_cash_low = 0                     # To trigger logging in cash_low case
    date = str(get_datetime().date())    # To trigger logging in daily case
    cash = c.portfolio.cash

    if int(cash) < c.cash_low:    # New cash low
        new_cash_low = 1
        c.cash_low   = int(cash)
        if record_cash_low:
            record(CashLow = int(c.cash_low))

    pvr_rtrn      = 0        # Profit vs Risk returns based on maximum spent
    profit_loss   = 0        # Profit-n-loss
    shorts        = 0        # Shorts value
    start         = c.portfolio.starting_cash
    cash_dip      = int(max(0, start - cash))

    if record_cash:
        record(Cash = int(c.portfolio.cash))  # Cash

    if record_leverage:
        record(Lvrg = c.account.leverage)     # Leverage

    if record_max_lvrg:
        if c.account.leverage > c.max_lvrg:
            c.max_lvrg = c.account.leverage
            record(MaxLv = c.max_lvrg)        # Maximum leverage

    if record_pnl:
        profit_loss = c.portfolio.pnl
        record(PnL = profit_loss)             # "Profit and Loss" in dollars

    for p in c.portfolio.positions:
        shrs = c.portfolio.positions[p].amount
        if shrs < 0:
            shorts += int(abs(shrs * data[p].price))

    if record_shorting:
        record(Shorts = shorts)               # Shorts value as a positve

    risk = int(max(cash_dip, shorts))
    if record_risk:
        record(Risk = risk)                   # Amount in play, maximum of shorts or cash used

    new_risk_hi = 0
    if risk > c.risk_hi:
        c.risk_hi = risk
        new_risk_hi = 1

        if record_risk_hi:
            record(RiskHi = c.risk_hi)        # Highest risk overall

    if record_pvr:      # Profit_vs_Risk returns based on max amount actually spent (risk high)
        if c.risk_hi != 0:     # Avoid zero-divide
            pvr_rtrn = 100 * (c.portfolio.portfolio_value - start) / c.risk_hi
            record(PvR = pvr_rtrn)            # Profit_vs_Risk returns

    q_rtrn = 100 * (c.portfolio.portfolio_value - start) / start
    if record_q_return:
        record(QRet = q_rtrn)                 # Quantopian returns to compare to pvr returns curve

    from pytz import timezone
    if logging:
        if log_method == 'risk_hi' and new_risk_hi \
          or log_method == 'daily' and c.date_prv != date \
          or c.date_end == date \
          or new_cash_low:
            qret   = 'QRet '    + '%.1f' % q_rtrn
            mxlv   = 'MaxLv '   + '%.1f' % c.max_lvrg   if record_max_lvrg else ''
            pvr    = 'PvR '     + '%.1f' % pvr_rtrn     if record_pvr      else ''
            pnl    = 'PnL '     + '%.0f' % profit_loss  if record_pnl      else ''
            csh    = 'Cash '    + '%.0f' % cash         if record_cash     else ''
            csh_lw = 'CshLw '   + '%.0f' % c.cash_low   if record_cash_low else ''
            shrt   = 'Shrt '    + '%.0f' % shorts       if record_shorting else ''
            risk   = 'Risk '    + '%.0f' % risk         if record_risk     else ''
            rsk_hi = 'RskHi '   + '%.0f' % c.risk_hi    if record_risk_hi  else ''
            minute = get_datetime().astimezone(timezone('US/Eastern')).time().minute
            log.info('{} {} {} {} {} {} {} {} {} {}'.format(
                    minute, mxlv, qret, pvr, pnl, csh, csh_lw, shrt, risk, rsk_hi))

    if c.date_end == date:    # Log on last day, like cash 125199  portfolio 126890
        log.info('cash {}  portfolio {}'.format(
                int(cash), int(c.portfolio.portfolio_value)))

    c.date_prv = date

There was a runtime error.

@garyha... period 2010-2011 is definitely.... an anomaly.... if you take out ....2011-2011 the algo performs.. poorly... can someone explain... why.. the sudden spike in that period... 2010-2011 in this algo...? thanks...

Clone Algorithm
7
Loading...
Backtest from to with initial capital
Total Returns
--
Alpha
--
Beta
--
Sharpe
--
Sortino
--
Max Drawdown
--
Benchmark Returns
--
Volatility
--
Returns 1 Month 3 Month 6 Month 12 Month
Alpha 1 Month 3 Month 6 Month 12 Month
Beta 1 Month 3 Month 6 Month 12 Month
Sharpe 1 Month 3 Month 6 Month 12 Month
Sortino 1 Month 3 Month 6 Month 12 Month
Volatility 1 Month 3 Month 6 Month 12 Month
Max Drawdown 1 Month 3 Month 6 Month 12 Month
import pandas
import numpy as np
from scipy import optimize
from sklearn.covariance import LedoitWolf
from pytz import timezone    # for track_orders()

def track_orders(context, data):  # Log orders created or filled.
    # https://www.quantopian.com/posts/track-orders
    #   modified here to log minute and cash

    #if 'orders' not in context:
    #    context.orders = {}

    '''
    To not overwhelm the logging window, start/stop dates can be entered in initialize()
    Example
    context.dates  = {
        'active': 0,
        'start' : ['2007-05-07', '2010-04-26'],
        'stop'  : ['2008-02-13', '2010-11-15']
    }
    '''
    if context.dates['start']:         # If the list has something in it
        date = str(get_datetime().date())
        if   date in context.dates['start']:    # See if there's a match to start
            context.dates['active'] = 1
        elif date in context.dates['stop']:     #   ... or to stop
            context.dates['active'] = 0
    else:
        context.dates['active'] = 1    # Set to active b/c no conditions

    if context.dates['active'] == 0:
        return    # Skip if off

    to_delete = []
    for id in context.orders:
        o   = get_order(id)
        sec = o.sid
        sym = sec.symbol
        bar_dt = get_datetime().astimezone(timezone('US/Eastern'))
        minute = (bar_dt.hour * 60) + bar_dt.minute - 570  # (-570 = 9:30a)
        if o.filled:        # Filled at least some, status 1 is Filled
            trade = 'Bot' if o.amount > 0 else 'Sold'
            log.info(' {}      {} {} {} at {}   cash {}'.format(minute,
                    trade, o.filled, sym, data[sec].price, int(context.portfolio.cash)))
            to_delete.append(o.id)
        else:
            log.info(' {}         {} {} unfilled'.format(
                    minute, o.sid.symbol, o.amount))

    for sec, oo_for_sid in get_open_orders().iteritems(): # Open orders
        sym = sec.symbol
        for o in oo_for_sid: # Orders per security
            bar_dt = get_datetime().astimezone(timezone('US/Eastern'))
            minute = (bar_dt.hour * 60) + bar_dt.minute - 570  # (-570 = 9:30a)
            if o.id in to_delete:
                continue
            if o.status == 2:                 # Cancelled
                log.info('    Cancelled {} {} order'.format(
                        trade, o.amount, sym, data[sec].price))
                to_delete.append(o.id)
            elif o.id not in context.orders:  # New
                context.orders[o.id] = 1
                trade = 'Buy' if o.amount > 0 else 'Sell'
                if o.limit:        # Limit order
                    log.info('   {} {} {} now {} limit {}'.format(
                            trade, o.amount, sym, data[sec].price, o.limit))
                elif o.stop:       # Stop order
                    log.info('   {} {} {} now {} stop {}'.format(
                            trade, o.amount, sym, data[sec].price, o.stop))
                else:              # Market order
                    log.info(' {}   {} {} {} at {}   cash {}'.format(minute,
                        trade, o.amount, sym, data[sec].price, int(context.portfolio.cash)))
    for d in to_delete:
        del context.orders[d]

def initialize(context):
    schedule_function(trade,
                      date_rule=date_rules.month_end(),
                      time_rule=time_rules.market_close(),
                      half_days=False)
    context.n = 300 # screened stocks number
    context.lookback = 252 # 252*n = n years
    context.bmark = symbol('SPY')
    context.bmarkp = 0 # Placeholder for later

    # for track_orders()
    context.orders = {}

    # track_orders() dates to start or stop logging.
    #   This is to target a date range and not overwhelm the logging window.
    #   These lists can be empty like []
    context.dates  = {
        'active': 0,
        'start' : ['2007-05-07', '2010-04-26'],
        'stop'  : ['2008-02-13', '2010-11-15']
    }

    #schedule_function(pvr, date_rules.every_day(), time_rules.market_close())

    # for pvr()
    c = context
    c.max_lvrg = 0
    c.risk_hi  = 0
    c.date_prv = ''
    c.cash_low = c.portfolio.starting_cash
    c.date_end = str(get_environment('end').date())
    log.info('{} to {}  {}  {}'.format(str(get_datetime().date()), c.date_end,
        int(c.cash_low), get_environment('data_frequency')))

def handle_data(context, data):
    track_orders(context, data)
    pvr(context, data)

    # Do custom record,    off using '0 and' to make room for pvr()
    if    0 and     (context.bmarkp != 0):
        spread = context.portfolio.returns - (data[context.bmark].close_price/context.bmarkp - 1)
        record(return_spread=spread)

def before_trading_start(context, data):
    #log.info("Cash Position: "+str(context.portfolio.cash))
    if (get_datetime().month % 3) == 0:
        dvd = get_fundamentals(query(fundamentals.valuation_ratios.dividend_yield)
                           .filter(fundamentals.valuation.market_cap >= 2000000000)
                           # Ensures that we get large market cap. thus liquid stocks
                           .order_by(fundamentals.valuation_ratios.dividend_yield.desc()).limit(context.n)
                          ).T
        # Remove the top three and bottom three to control outliers
        dvd = dvd.iloc[3:-3]
        update_universe(dvd.index)
        context.dvdy = dvd

def trade(context,data):
    if (get_datetime().month%3)==0:
        log.info(context.account.leverage)
        w = getCarried(context,data)

        # Close all current positions
        for stock in context.portfolio.positions:
            if stock not in w.index:
               order_target(stock, 0)

        #Order
        for i,n in enumerate(w):
            if w.index[i] in data:
                if w.index[i] in context.portfolio.positions: #If position already exists
                    order_target_percent(w.index[i], n)
                else:
                    order_percent(w.index[i], n)

def getCarried(context,data):
        # Structure a portfolio such that it maximizes dividend yield while lowering volatility
        # Covar estimation
        price_history = history(bar_count=context.lookback, frequency="1d", field='price')
        if(context.bmarkp == 0):
            context.bmarkp = data[context.bmark].close_price
        price_history = price_history[context.dvdy.index] # Fix the +1 issue of extra stock
        ret = (price_history/price_history.shift(1)-1)[1:].dropna(axis=1)
        n = len(ret.columns)
        context.dvdy = context.dvdy.loc[ret.columns] # subset that removes all na columns
        #OAS
        covar = LedoitWolf().fit(ret).covariance_

        #Constant Correlation Model
        #corr = ccmodel(ret, context, n)
        #std = np.dot(pandas.DataFrame(ret.std()),pandas.DataFrame(ret.std()).T)
        #covar = pandas.DataFrame(corr*std) # Rho %*% (sigma%*%sigma^T)

        # sample estimate
        #covar = ret.cov() # Normal Cov

        # Set Solver Constraint
        cons = (
             {'type': 'eq', 'fun': lambda x: 1-sum(x)},
             {'type': 'ineq', 'fun': lambda x: 0.03-x}, # 3 % at mo
             {'type': 'ineq', 'fun': lambda x: x},
             #{'type': 'ineq', 'fun': lambda x: 50-np.sqrt(sum(pow(x,2)))/sum(x)} # Leverage Constraint
        )
        ra = 5000 # Risk Aversion Coefficient
        w = optimize.minimize(objDivTradeOff, # Minimize so -1 makes it maximize (see below)
                                    x0=np.array([1/n]*n), #Initial guess of 1/N
                                    args=(context.dvdy,covar,ra), # Pass in arguments
                                    method='SLSQP', jac=False, #Jacobian vector
                                    constraints=cons, # constraints set as above
                                    options=({'maxiter': 1e4}))   #Ensure convergence
        #log.info('Expected Dividend Yield: '+str(np.dot(w.x,context.dvdy)*100)+'%')
        #log.info('Expected Volatility: '+str(np.sqrt(np.dot(w.x.T,np.dot(covar,w.x)))*100)+'%')
        # To save space in logging window, one line
        divyld = np.dot(w.x,context.dvdy)[0] * 100
        vltlty = np.sqrt(np.dot(w.x.T,np.dot(covar,w.x))) * 100

        w = pandas.Series(np.round(10000*w.x)/10000)
        #log.info('Number of Stocks: '+str(sum(abs(w)>0)))

        log.info('NumStcks, Exp. DivYd, Vy  {} {} {}'.format(
                str(sum(abs(w)>0)), '%.2f' % divyld, '%.2f' % vltlty ))

        w.index = ret.columns
        return w

def objDivTradeOff(x,divyield,covar,ra):
    return -1*(np.dot(x,divyield)-ra*np.dot(x.T,np.dot(covar,x))) # W'DivYield - lambda * W'CovarW

def jacDivTradeOff(x,divyield,covar,ra):
    return -1*(divyield - ra*2*np.dot(covar,x)) #First order derivative

def ccmodel(ret, context, n):
    # Assuming T x N Matrix
    norm = n*(n-1)/2
    m = sum(sum(np.triu(ret.corr(),k=1)))/norm
    a = pandas.DataFrame(np.array([m]*(n*n)).reshape(n,n), index=ret.columns, columns=ret.columns)
    for i in np.arange(n):
        a.iloc[i,i] = 1
    return a

def pvr(context, data):
    ''' Custom chart and/or log of profit_vs_risk returns and related information
    '''
    # # # # # # # # # #  Options  # # # # # # # # # #
    record_max_lvrg = 1          # Maximum leverage encountered
    record_leverage = 0          # Leverage (context.account.leverage)
    record_q_return = 0          # Quantopian returns (percentage)
    record_pvr      = 1          # Profit vs Risk returns (percentage)
    record_pnl      = 0          # Profit-n-Loss
    record_shorting = 1          # Total value of any shorts
    record_risk     = 0          # Risked, maximum cash spent or shorts in excess of cash at any time
    record_risk_hi  = 1          # Highest risk overall
    record_cash     = 0          # Cash available
    record_cash_low = 1          # Any new lowest cash level
    logging         = 1          # Also log to the logging window conditionally (1) or not (0)
    log_method      = 'risk_hi'  # 'daily' or 'risk_hi'

    c = context                          # For brevity
    new_cash_low = 0                     # To trigger logging in cash_low case
    date = str(get_datetime().date())    # To trigger logging in daily case
    cash = c.portfolio.cash

    if int(cash) < c.cash_low:    # New cash low
        new_cash_low = 1
        c.cash_low   = int(cash)
        if record_cash_low:
            record(CashLow = int(c.cash_low))

    pvr_rtrn      = 0        # Profit vs Risk returns based on maximum spent
    profit_loss   = 0        # Profit-n-loss
    shorts        = 0        # Shorts value
    start         = c.portfolio.starting_cash
    cash_dip      = int(max(0, start - cash))

    if record_cash:
        record(Cash = int(c.portfolio.cash))  # Cash

    if record_leverage:
        record(Lvrg = c.account.leverage)     # Leverage

    if record_max_lvrg:
        if c.account.leverage > c.max_lvrg:
            c.max_lvrg = c.account.leverage
            record(MaxLv = c.max_lvrg)        # Maximum leverage

    if record_pnl:
        profit_loss = c.portfolio.pnl
        record(PnL = profit_loss)             # "Profit and Loss" in dollars

    for p in c.portfolio.positions:
        shrs = c.portfolio.positions[p].amount
        if shrs < 0:
            shorts += int(abs(shrs * data[p].price))

    if record_shorting:
        record(Shorts = shorts)               # Shorts value as a positve

    risk = int(max(cash_dip, shorts))
    if record_risk:
        record(Risk = risk)                   # Amount in play, maximum of shorts or cash used

    new_risk_hi = 0
    if risk > c.risk_hi:
        c.risk_hi = risk
        new_risk_hi = 1

        if record_risk_hi:
            record(RiskHi = c.risk_hi)        # Highest risk overall

    if record_pvr:      # Profit_vs_Risk returns based on max amount actually spent (risk high)
        if c.risk_hi != 0:     # Avoid zero-divide
            pvr_rtrn = 100 * (c.portfolio.portfolio_value - start) / c.risk_hi
            record(PvR = pvr_rtrn)            # Profit_vs_Risk returns

    q_rtrn = 100 * (c.portfolio.portfolio_value - start) / start
    if record_q_return:
        record(QRet = q_rtrn)                 # Quantopian returns to compare to pvr returns curve

    from pytz import timezone
    if logging:
        if log_method == 'risk_hi' and new_risk_hi \
          or log_method == 'daily' and c.date_prv != date \
          or c.date_end == date \
          or new_cash_low:
            qret   = 'QRet '    + '%.1f' % q_rtrn
            mxlv   = 'MaxLv '   + '%.1f' % c.max_lvrg   if record_max_lvrg else ''
            pvr    = 'PvR '     + '%.1f' % pvr_rtrn     if record_pvr      else ''
            pnl    = 'PnL '     + '%.0f' % profit_loss  if record_pnl      else ''
            csh    = 'Cash '    + '%.0f' % cash         if record_cash     else ''
            csh_lw = 'CshLw '   + '%.0f' % c.cash_low   if record_cash_low else ''
            shrt   = 'Shrt '    + '%.0f' % shorts       if record_shorting else ''
            risk   = 'Risk '    + '%.0f' % risk         if record_risk     else ''
            rsk_hi = 'RskHi '   + '%.0f' % c.risk_hi    if record_risk_hi  else ''
            minute = get_datetime().astimezone(timezone('US/Eastern')).time().minute
            log.info('{} {} {} {} {} {} {} {} {} {}'.format(
                    minute, mxlv, qret, pvr, pnl, csh, csh_lw, shrt, risk, rsk_hi))

    if c.date_end == date:    # Log on last day, like cash 125199  portfolio 126890
        log.info('cash {}  portfolio {}'.format(
                int(cash), int(c.portfolio.portfolio_value)))

    c.date_prv = date

There was a runtime error.