Margin costs

Is there a way to create a function in an algorithm to chart the costs?

From https://www.quantopian.com/posts/stocks-on-the-move-by-andreas-clenow
... I understand this defines the costs of margin (negative cash, borrowing), the particular form of leverage that is margin:

LevCost(t) = (1+L)∙k∙A(0)∙(1 + r + α – fc%)^t - (1+L)∙k∙A(0)∙(1 + r + α – lc% – fc%)^t

1 response

One small step for margin awareness, not a giant leap for much, important nonetheless.

This backtest from here was incurring margin beyond the value of positions at times. No broker will do that unless you're putting up your yachts for collateral. In the code, the margin hack looks for those times near the end of day and adjusts positions to target less than 100% margin for something closer to tradable on planet earth. Results and some lines from the log output ...

                  Q Returns    PvR (Profit per maximum dollar risked)
His version:   2707%        237%
Limited margin:   1760%        256% (increase)

2013-04-01 12:58 margin_hack:138 INFO mratio -3.11%
2013-04-01 12:58 margin_hack:144 INFO YHOO order -967 of 31094
2013-04-01 12:58 margin_hack:144 INFO DAL order -967 of 31094
2013-04-01 12:58 margin_hack:144 INFO TSN order -992 of 31908
2013-04-01 12:58 margin_hack:144 INFO HRB order -992 of 31908
2013-04-01 12:58 margin_hack:144 INFO HPQ order -992 of 31908
2013-04-01 12:58 margin_hack:144 INFO PSX order -967 of 31094
2013-04-01 12:58 margin_hack:144 INFO MPC order -967 of 31094
2013-04-01 12:58 margin_hack:144 INFO NYX order -992 of 31908
2013-04-01 12:58 margin_hack:144 INFO BBY order -992 of 31908
2013-04-01 12:58 margin_hack:144 INFO BAC order -967 of 31094
2013-12-02 12:58 margin_hack:138 INFO mratio -28.15%
2013-12-02 12:58 margin_hack:144 INFO DAL order -11308 of 40165
2013-12-02 12:58 margin_hack:144 INFO FB order -10442 of 37089
2013-12-02 12:58 margin_hack:144 INFO MU order -11308 of 40165
2013-12-02 12:58 margin_hack:144 INFO LNG order -10442 of 37088
2013-12-02 12:58 margin_hack:144 INFO TSLA order -11308 of 40165
2013-12-02 12:58 margin_hack:144 INFO IEP order -10442 of 37088
2013-12-02 12:58 margin_hack:144 INFO LUV order -10442 of 37088
2013-12-02 12:58 margin_hack:144 INFO BBY order -11308 of 40165


PvR is turned off here for speed and to give mratio the stage.

39
Total Returns
--
Alpha
--
Beta
--
Sharpe
--
Sortino
--
Max Drawdown
--
Benchmark Returns
--
Volatility
--
 Returns 1 Month 3 Month 6 Month 12 Month
 Alpha 1 Month 3 Month 6 Month 12 Month
 Beta 1 Month 3 Month 6 Month 12 Month
 Sharpe 1 Month 3 Month 6 Month 12 Month
 Sortino 1 Month 3 Month 6 Month 12 Month
 Volatility 1 Month 3 Month 6 Month 12 Month
 Max Drawdown 1 Month 3 Month 6 Month 12 Month
'''
Original code from Stocks On The Move by Andreas Clenow, a version by VY
https://www.quantopian.com/posts/stocks-on-the-move-by-andreas-clenow#58d470074fde6b0be1277e29
and then modified by Blue regarding margin for https://www.quantopian.com/posts/margin-costs

The problem:
1. People are using margin without realizing it.
2. Even worse, that margin gets turned into apparent/unreal profit in Returns.
3. Even worse yet, sometimes the margin is beyond anything possible in the real world, beyond
what any broker would provide.

While I am no expert on margin, it seems to me that with some accounts, a broker might allow
up to 100% of longs value in margin.
The idea here is to compare cash to positions value near the end of the day and any time margin
is found to exceed positions_value, adjust positions to increase cash to something
somewhat in closer to viable real-world limits.

The original code is not mine. I only added margin stuff and PvR (off for speed).
'''

from quantopian.algorithm import attach_pipeline, pipeline_output
from quantopian.pipeline import Pipeline
from quantopian.pipeline.factors import CustomFactor    # there was a memory error, throwing stuff overboard
#from quantopian.pipeline.factors import CustomFactor, SimpleMovingAverage, Latest
from quantopian.pipeline.data.builtin import USEquityPricing
from quantopian.pipeline.data import morningstar

import numpy as np
from scipy import stats
import talib

def initialize(context):
context.market = sid(8554)
context.market_window = 200
context.atr_window = 20
context.talib_window = context.atr_window + 5
context.risk_factor = 0.003                     # 0.01 = less position, more % but more risk

context.momentum_window_length = 90
context.market_cap_limit = 700
context.rank_table_percentile = .30
context.significant_position_difference = 0.1
context.min_momentum = 0.000
context.leverage_factor = 1.0                   # 1=2154%. Guy's version is 1.4=3226%
context.use_stock_trend_filter = 0              # either 0 = Off, 1 = On
context.sma_window_length = 100                 # Used for the stock trend filter
context.use_market_trend_filter = 1             # either 0 = Off, 1 = On. Filter on SPY
context.use_average_true_range = 0              # either 0 = Off, 1 = On. Manage risk with individual stock volatility
context.average_true_rage_multipl_factor = 1    # Change the weight of the ATR. 1327%

attach_pipeline(make_pipeline(context, context.sma_window_length,
context.market_cap_limit), 'screen')

schedule_function(balance, date_rules.month_start(), time_rules.market_open(hours=1))

# Cancel all open orders at the end of each day.
schedule_function(cancel_oos, date_rules.every_day(), time_rules.market_close())

# Effort to keep margin more within limits
schedule_function(margin_hack, date_rules.every_day(), time_rules.market_close(minutes=2))

def margin_hack(context, data):
c = context

'''
Margin:
...is only when cash is negative, so ...
Here, mratio to be able to see cash/positions even when positive. Meanwhuile ....
When margin (negative cash) is beyond positions value, adjust to make scenario less impossible.
'''

#mratio = min(0, c.portfolio.cash) / max(1, c.portfolio.positions_value)

mratio = 0  # margin ratio, cash versus positions, needs more work if shorting involved
if c.portfolio.positions_value > 100: # avoid division infinity
mratio = c.portfolio.cash / c.portfolio.positions_value

mratio = min(.05, mratio)    # cap at x to see negatives in chart more in scope
record(mratio = mratio)

if mratio < 0:      # Excess mratio beyond positions value if negative
log.info('mratio {}%'.format('%.2f' % (100 * mratio)))

# Trim positions targeting reduction of mratio to under 100% of positions_value
for s in c.portfolio.positions:
amt    = c.portfolio.positions[s].amount
to_cut = max(int(mratio * amt) - 1, -amt)
log.info('{} order {} of {}'.format(s.symbol, to_cut, amt))
order(s, to_cut)

def slope(ts): ## new version
x = np.arange(len(ts))
log_ts = np.log(ts)
slope, intercept, r_value, p_value, std_err = stats.linregress(x, log_ts)
annualized_slope = (np.power(np.exp(slope), 250) - 1) * 100
return annualized_slope * (r_value ** 2)

def _slope(ts):
x = np.arange(len(ts))
slope, intercept, r_value, p_value, std_err = stats.linregress(x, ts)
annualized_slope = (1 + slope)**250
return annualized_slope * (r_value ** 2)

class MarketCap(CustomFactor):
inputs = [USEquityPricing.close, morningstar.valuation.shares_outstanding]
window_length = 1

def compute(self, today, assets, out, close, shares):
out[:] = close[-1] * shares[-1]

def make_pipeline(context,sma_window_length, market_cap_limit):
pipe = Pipeline()

# Now only stocks in the top N largest companies by market cap
market_cap = MarketCap()
top_N_market_cap = market_cap.top(market_cap_limit)

#Other filters to make sure a clean universe
is_primary_share = morningstar.share_class_reference.is_primary_share.latest

#### TREND FITLER ##############
#### Don't want to trade stocks that are below their sma_window_length(100) moving average price.
if context.use_stock_trend_filter:
latest_price = USEquityPricing.close.latest
sma = SimpleMovingAverage(inputs=[USEquityPricing.close], window_length=sma_window_length)
above_sma = (latest_price > sma)
initial_screen = (above_sma & top_N_market_cap & is_primary_share & is_not_adr)
log.info("Init: Stock trend filter ON")
else: #### TREND FITLER OFF  ##############
initial_screen = (top_N_market_cap & is_primary_share & is_not_adr)
log.info("Init: Stock trend filter OFF")

pipe.set_screen(initial_screen)

return pipe

context.selected_universe = pipeline_output('screen')
context.assets = context.selected_universe.index

def cancel_oos(context, data):
oo = get_open_orders()
for s in oo:
for o in oo[s]: cancel_order(o.id)

#record(lever=context.account.leverage,
#record(exposure=context.account.leverage)

def balance(context, data):
highs  = data.history(context.assets, "high", context.talib_window, "1d")
lows   = data.history(context.assets, "low", context.talib_window, "1d")
closes = data.history(context.assets, "price", context.market_window, "1d")

estimated_cash_balance = context.portfolio.cash
slopes = closes[context.selected_universe.index].tail(context.momentum_window_length).apply(slope)
slopes = slopes[slopes > context.min_momentum]
ranking_table = slopes[slopes > slopes.quantile(1 - context.rank_table_percentile)].sort_values(ascending=False)
#log.info( len(ranking_table.index))
# close positions that are no longer in the top of the ranking table
positions = context.portfolio.positions
for security in positions:
price = data.current(security, "price")
position_size = positions[security].amount
if data.can_trade(security) and security not in ranking_table.index:
order_target(security, 0, style=LimitOrder(price))
estimated_cash_balance += price * position_size
new_position_size = get_position_size(context, highs[security], lows[security], closes[security],security)
if significant_change_in_position_size(context, new_position_size, position_size):
estimated_cost = price * (new_position_size * context.leverage_factor - position_size)
order_target(security, new_position_size * context.leverage_factor, style=LimitOrder(price))
estimated_cash_balance -= estimated_cost

# Market history is not used with the trend filter disabled
# Removed for efficiency
if context.use_market_trend_filter:
market_history = data.history(context.market, "price", context.market_window, "1d")  ##SPY##
current_market_price = market_history[-1]
average_market_price = market_history.mean()
else:
average_market_price = 0

if (current_market_price > average_market_price) :  #if average is 0 then jump in
for security in ranking_table.index:
if data.can_trade(security) and security not in context.portfolio.positions:
new_position_size = get_position_size(context, highs[security], lows[security], closes[security],
security)
estimated_cost = data.current(security, "price") * new_position_size * context.leverage_factor
if estimated_cash_balance > estimated_cost:
order_target(security, new_position_size * context.leverage_factor, style=LimitOrder(data.current(security, "price")))
estimated_cash_balance -= estimated_cost

def get_position_size(context, highs, lows, closes, security):
try:
average_true_range = talib.ATR(highs.ffill().dropna().tail(context.talib_window),
lows.ffill().dropna().tail(context.talib_window),
closes.ffill().dropna().tail(context.talib_window),
context.atr_window)[-1] # [-1] gets the last value, as all talib methods are rolling calculations#
if not context.use_average_true_range: #average_true_range
average_true_range = 1 #divide by 1 gives... same initial number
context.average_true_rage_multipl_factor = 1

return (context.portfolio.portfolio_value * context.risk_factor)  / (average_true_range * context.average_true_rage_multipl_factor)
except:
log.warn('Insufficient history to calculate risk adjusted size for {0.symbol}'.format(security))
return 0

def significant_change_in_position_size(context, new_position_size, old_position_size):
return np.abs((new_position_size - old_position_size)  / old_position_size) > context.significant_position_difference

def pvr(context, data):
''' Custom chart and/or logging of profit_vs_risk returns and related information
'''
import time
from datetime import datetime
from pytz import timezone      # Python will only do once, makes this portable.
#   Move to top of algo for better efficiency.
c = context  # Brevity is the soul of wit -- Shakespeare [for readability]
if 'pvr' not in c:
c.pvr = {
'options': {
# # # # # # # # # #  Options  # # # # # # # # # #
'logging'         : 0,    # Info to logging window with some new maximums

'record_pvr'      : 1,    # Profit vs Risk returns (percentage)
'record_pvrp'     : 0,    # PvR (p)roportional neg cash vs portfolio value
'record_cash'     : 1,    # Cash available
'record_max_lvrg' : 1,    # Maximum leverage encountered
'record_risk_hi'  : 0,    # Highest risk overall
'record_shorting' : 0,    # Total value of any shorts
'record_max_shrt' : 0,    # Max value of shorting total
'record_cash_low' : 1,    # Any new lowest cash level
'record_q_return' : 0,    # Quantopian returns (percentage)
'record_pnl'      : 0,    # Profit-n-Loss
'record_risk'     : 0,    # Risked, max cash spent or shorts beyond longs+cash
'record_leverage' : 0,    # Leverage (context.account.leverage)
# # # # # # # # #  End options  # # # # # # # # #
},
'pvr'        : 0,      # Profit vs Risk returns based on maximum spent
'cagr'       : 0,
'max_lvrg'   : 0,
'max_shrt'   : 0,
'risk_hi'    : 0,
'days'       : 0.0,
'date_prv'   : '',
'date_end'   : get_environment('end').date(),
'cash_low'   : 1e99,
'cash'       : c.portfolio.starting_cash,
'start'      : c.portfolio.starting_cash,
'pstart'     : c.portfolio.portfolio_value, # Used if restart
'begin'      : time.time(),                 # For run time
'log_summary': 126,                         # Summary every x days
'run_str'    : '{} to {}  \${}  {} US/Eastern'.format(get_environment('start').date(), get_environment('end').date(), int(c.portfolio.starting_cash), datetime.now(timezone('US/Eastern')).strftime("%Y-%m-%d %H:%M"))
}
if c.pvr['options']['record_pvrp']: c.pvr['options']['record_pvr'] = 0 # if pvrp is active, straight pvr is off
if get_environment('arena') not in ['backtest', 'live']: c.pvr['log_summary'] = 1 # Every day when real money
log.info(c.pvr['run_str'])
p = c.pvr ; o = c.pvr['options']
def _pvr(c):
p['cagr'] = ((c.portfolio.portfolio_value / p['start']) ** (1 / (p['days'] / 252.))) - 1
ptype = 'PvR' if o['record_pvr'] else 'PvRp'
log.info('{} {} %/day   cagr {}   Portfolio value {}   PnL {}'.format(ptype, '%.4f' % (p['pvr'] / p['days']), '%.3f' % p['cagr'], '%.0f' % c.portfolio.portfolio_value, '%.0f' % (c.portfolio.portfolio_value - p['start'])))
log.info('  Profited {} on {} activated/transacted for PvR of {}%'.format('%.0f' % (c.portfolio.portfolio_value - p['start']), '%.0f' % p['risk_hi'], '%.1f' % p['pvr']))
log.info('  QRet {} PvR {} CshLw {} MxLv {} RskHi {} MxShrt {}'.format('%.2f' % q_rtrn, '%.2f' % p['pvr'], '%.0f' % p['cash_low'], '%.2f' % p['max_lvrg'], '%.0f' % p['risk_hi'], '%.0f' % p['max_shrt']))
def _minut():
dt = get_datetime().astimezone(timezone('US/Eastern'))
return str((dt.hour * 60) + dt.minute - 570).rjust(3)  # (-570 = 9:31a)
date = get_datetime().date()
if p['date_prv'] != date:
p['date_prv'] = date
p['days'] += 1.0
do_summary = 0
if p['log_summary'] and p['days'] % p['log_summary'] == 0 and _minut() == '100':
do_summary = 1              # Log summary every x days
if do_summary or date == p['date_end']:
p['cash'] = c.portfolio.cash
elif p['cash'] == c.portfolio.cash and not o['logging']: return  # for speed

shorts = sum([z.amount * z.last_sale_price for s, z in c.portfolio.positions.items() if z.amount < 0])
q_rtrn       = 100 * c.portfolio.returns  #100 * (c.portfolio.portfolio_value - p['start']) / p['start']
cash         = c.portfolio.cash
new_risk_hi  = 0
new_max_lv   = 0
new_max_shrt = 0
new_cash_low = 0               # To trigger logging in cash_low case
cash_dip     = int(max(0, p['pstart'] - cash))
risk         = int(max(cash_dip, -shorts))

if o['record_pvrp'] and cash < 0:   # Let negative cash ding less when portfolio is up.
cash_dip = int(max(0, p['start'] - cash * p['start'] / c.portfolio.portfolio_value))
# Imagine: Start with 10, grows to 1000, goes negative to -10, should not be 200% risk.

if int(cash) < p['cash_low']:             # New cash low
new_cash_low = 1
p['cash_low']  = int(cash)            # Lowest cash level hit
if o['record_cash_low']: record(CashLow = p['cash_low'])

if c.account.leverage > p['max_lvrg']:
new_max_lv = 1
p['max_lvrg'] = c.account.leverage    # Maximum intraday leverage
if o['record_max_lvrg']: record(MaxLv   = p['max_lvrg'])

if shorts < p['max_shrt']:
new_max_shrt = 1
p['max_shrt'] = shorts                # Maximum shorts value
if o['record_max_shrt']: record(MxShrt  = p['max_shrt'])

if risk > p['risk_hi']:
new_risk_hi = 1
p['risk_hi'] = risk                   # Highest risk overall
if o['record_risk_hi']:  record(RiskHi  = p['risk_hi'])

# Profit_vs_Risk returns based on max amount actually spent (risk high)
if p['risk_hi'] != 0: # Avoid zero-divide
p['pvr'] = 100 * (c.portfolio.portfolio_value - p['start']) / p['risk_hi']
ptype = 'PvRp' if o['record_pvrp'] else 'PvR'
if o['record_pvr'] or o['record_pvrp']: record(**{ptype: p['pvr']})

if o['record_shorting']: record(Shorts    = shorts)            # Shorts value as a positve
if o['record_leverage']: record(Lvrg = c.account.leverage)     # Leverage
if o['record_cash']:     record(Cash = cash)                   # Cash
if o['record_risk']:     record(Risk = risk)   # Amount in play, maximum of shorts or cash used
if o['record_q_return']: record(QRet = q_rtrn) # Quantopian returns to compare to pvr returns curve
if o['record_pnl']:      record(PnL  = c.portfolio.portfolio_value - p['start']) # Profit|Loss

# margin hack
#margin = 100 * min(0, cash) / max(1, c.portfolio.positions_value)
#if margin < 0:
#    log.info('margin {}%'.format('%.1f' % margin))

if o['logging'] and (new_risk_hi or new_cash_low or new_max_lv or new_max_shrt):
csh     = ' Cash '   + '%.0f' % cash
risk    = ' Risk '   + '%.0f' % risk
qret    = ' QRet '   + '%.1f' % q_rtrn
shrt    = ' Shrt '   + '%.0f' % shorts
lv      = ' Lv '     + '%.1f' % c.account.leverage
pvr     = ' PvR '    + '%.1f' % p['pvr']
rsk_hi  = ' RskHi '  + '%.0f' % p['risk_hi']
csh_lw  = ' CshLw '  + '%.0f' % p['cash_low']
mxlv    = ' MxLv '   + '%.2f' % p['max_lvrg']
mxshrt  = ' MxShrt ' + '%.0f' % p['max_shrt']
pnl     = ' PnL '    + '%.0f' % (c.portfolio.portfolio_value - p['start'])
log.info('{}{}{}{}{}{}{}{}{}{}{}{}'.format(_minut(), lv, mxlv, qret, pvr, pnl, csh, csh_lw, shrt, mxshrt, risk, rsk_hi))
if do_summary: _pvr(c)
if get_datetime() == get_environment('end'):    # Summary at end of run
_pvr(c)
elapsed = (time.time() - p['begin']) / 60  # minutes
log.info( '{}\nRuntime {} hr {} min'.format(p['run_str'], int(elapsed / 60), '%.1f' % (elapsed % 60)))

'''
def handle_data(context, data):
pvr(context, data)
'''

There was a runtime error.