Back to Community
Optimize debugging???

Hello,

I've been at this for a while and cannot figure out why my algo is not purchasing anything. I've given it a list with weights which should buy but it seems the optimize function is not working. Any thoughts on where I'm going wrong. Thanks in advance!

Clone Algorithm
8
Loading...
Backtest from to with initial capital
Total Returns
--
Alpha
--
Beta
--
Sharpe
--
Sortino
--
Max Drawdown
--
Benchmark Returns
--
Volatility
--
Returns 1 Month 3 Month 6 Month 12 Month
Alpha 1 Month 3 Month 6 Month 12 Month
Beta 1 Month 3 Month 6 Month 12 Month
Sharpe 1 Month 3 Month 6 Month 12 Month
Sortino 1 Month 3 Month 6 Month 12 Month
Volatility 1 Month 3 Month 6 Month 12 Month
Max Drawdown 1 Month 3 Month 6 Month 12 Month
"""
This is a template algorithm on Quantopian for you to adapt and fill in.
"""
import quantopian.algorithm as algo
from quantopian.pipeline import Pipeline
from quantopian.pipeline.data.builtin import USEquityPricing
from quantopian.pipeline.filters import QTradableStocksUS
import quantopian.optimize as opt
from quantopian.pipeline.experimental import risk_loading_pipeline

from quantopian.pipeline.data.morningstar import Fundamentals
#from quantopian.pipeline.domain import US_EQUITIES
import pandas as pd
import numpy as np
import datetime as dt

def initialize(context):
    """
    Called once at the start of the algorithm.
    """
    # Create our dynamic stock selector.
    algo.attach_pipeline(make_pipeline(), 'pipeline')
    algo.attach_pipeline(risk_loading_pipeline(), 'risk_loading_pipeline')
    
    context.max_leverage = 1.00
    context.max_pos_size = 0.50
    context.max_turnover = 0.95
    
    # Rebalance every day, 1 hour after market open.
    algo.schedule_function(
        rebalance,
        algo.date_rules.every_day(),
        algo.time_rules.market_open(hours=3),
    )
    '''
    algo.schedule_function(
        re_try,
        algo.date_rules.month_start(days_offset = 1),
        algo.time_rules.market_open(hours=3),
    )
    
    algo.schedule_function(
        exit,
        algo.date_rules.month_end(),
        algo.time_rules.market_open(hours=3),
    )
    '''
    # Record tracking variables at the end of each day.
    algo.schedule_function(
        record_vars,
        algo.date_rules.every_day(),
        algo.time_rules.market_close(),
    )
    
    set_commission(commission.PerShare(cost=0.000)) 

def make_pipeline():
    # Base universe set to the QTradableStocksUS
    base_universe = QTradableStocksUS()

    # Factor of yesterday's close price.
    market_cap = Fundamentals.market_cap.latest
    pe_ratio = Fundamentals.pe_ratio.latest 
    pb_ratio = Fundamentals.pb_ratio.latest
    value_score = Fundamentals.value_score.latest
    earning_yield = Fundamentals.earning_yield.latest
    roic = Fundamentals.roic.latest
        
    pipe = Pipeline(
        columns={
            'earning_yield': earning_yield,
            'roic' : roic
        },
        screen=(base_universe & earning_yield.notnan() & roic.notnan())
    )
    return pipe

def before_trading_start(context, data):
    """
    Called every day before market open.
    """
    context.output = algo.pipeline_output('pipeline')
    context.risk_loading_pipeline = algo.pipeline_output('risk_loading_pipeline')
    a = 0
    for i in context.portfolio.positions:
        a +=1
        
    if a < 20:
        context.output['score'] = 0.0
        context.my_list = []
        df = context.output
        
        # These are the securities that we are interested in trading each day.
        context.security_list = context.output.index
        df_1 = context.output.sort_values(by=['earning_yield'], ascending = True)
        df_2 = context.output.sort_values(by=['roic'], ascending = True)
        
        df_1['score_1'] = 0.0 
        df_2['score_2'] = 0.0
                     
        for i in range(1,len(df_1)):
            df_1['score_1'][i] = i 
    
        for i in range(1,len(df_2)):
            df_2['score_2'][i] = i 
                
        for i in range(len(df)):
            for j in range(len(df_1)):
                if df.index[i] == df_1.index[j]:
                    for k in range(len(df_2)):
                        if df.index[i] == df_2.index[k]:
                            df['score'][i] = float(df_1['score_1'][j]) + float(df_2['score_2'][k])
                    
        df = context.output.sort_values(by=['score'], ascending = False)

        context.my_list.append(df.index[:30])
        dlist = df.index[:30]
        dl = pd.DataFrame({'Symbol':dlist})
            
        context.weight = 1.0/(len(dl))              
        context.long_weights = pd.Series(context.weight, context.my_list)
        context.weights = pd.concat([context.long_weights])
        
        print "hello", context.weights
        drop_list = []
        count = 0
        for i in context.risk_loading_pipeline.index:
            if i not in dlist:
                drop_list.append(i)
            if i in dlist:
                count +=1

        context.risk_loading_pipeline = context.risk_loading_pipeline.drop(drop_list)
        
    
def rebalance(context, data):
    
    alpha = context.risk_loading_pipeline
    target_weights = opt.TargetWeights(context.weights)
     
    constrain_pos_size = opt.PositionConcentration.with_equal_bounds(-context.max_pos_size, context.max_pos_size)
    max_leverage = opt.MaxGrossExposure(context.max_leverage)        
    dollar_neutral = opt.DollarNeutral()  
    #max_turnover = opt.MaxTurnover(context.max_turnover)
    
    constrain_sector_style_risk = opt.experimental.RiskModelExposure(  
        risk_model_loadings=context.risk_loading_pipeline,  
        version=opt.Newest)
    
    algo.order_optimal_portfolio(objective=target_weights,
                                 constraints=[constrain_pos_size, 
                                              constrain_sector_style_risk,
                                              max_leverage,
                                              dollar_neutral,
                                              #max_turnover,
                                              ]
                                              )    
    
    '''
    #Long Positioning
    a = 0
    for i in context.portfolio.positions:
        a +=1
    
    for i in context.risk_loading_pipeline.index:
        if data.can_trade(i) and a < 30:
            if i not in context.portfolio.positions:
                order_target_percent(i, 0.03)'''
  
    pass

def re_try(context, data):
    #Retry for Long positions
    for i in context.portfolio.positions:
        a = (context.portfolio.positions[i].amount*context.portfolio.positions[i].cost_basis)/context.portfolio.portfolio_value
        if a < 0.030 and a > 0:
            order_target_percent(i, 0.03) 
    
    pass

def exit(context, data):
    
    #print "Final Filtered List:", context.my_list
    
    for i in context.portfolio.positions:
        order_target_percent(i, 0.0)             

    pass

def record_vars(context, data):
    """
    Plot variables at the end of each day.
    """
    record(leverage=context.account.leverage)
    long_ = 0 
    short = 0
    for i in context.portfolio.positions:
        if context.portfolio.positions[i].amount > 0:
            long_ += 1
        if context.portfolio.positions[i].amount < 0:
            short +=1
    record(longs = long_, shorts= short) 
    pass


def handle_data(context, data):
    """
    Called every minute.
    """
    pass
There was a runtime error.
2 responses

@Lahiru Fernando A first step I typically use in troubleshooting is to set breakpoints in the IDE. (see the docs https://www.quantopian.com/docs/user-guide/environments/algo-ide#debugger) . Setting a breakpoint at the order_optimal_portfolio method allows one to inspect variables and, in this case, the inputs to that method.

Inspecting 'context.weights' (which are the target weights) it appears all the weights are positive. There aren't any short weights. This isn't a problem except one of the constraints is to be dollar neutral, or an equal amount of long and short positions.

What's happening? The optimizer will try changing the target weights to satisfy all the constraints including the dollar neutral constraint. It will also try to minimize the 'distance' from the original weights. In effect, the optimizer keeps decreasing the weights until it finds a solution. The first solution it finds is all weights equal to zero. This isn't probably what one wants but it does satisfy all the constraints. This result is dollar neutral ( equal long and short values). The result doesn't go over our max position size. The result doesn't go over our max leverage. And the result doesn't have any exposure to the common risk factors. The optimizer is simply doing what it was asked to do.

So, assuming this isn't the solution one really want's, what are some options? It's best to explicitly get pretty close to a solution and then use the optimizer to simply 'tweak' the results. In this case, ensure the target weights going into the optimizer include both long and short values and are pretty close to being dollar neutral from the beginning.

Good luck.

Disclaimer

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.

Hello Dan,

Thank you for the explanation. That makes a lot of sense and now I have a better understanding of the optimizer function as well!