Back to Community
Backtest failing to start before start date of 2017!

Has anyone had any issues today with running backtests? This is a code that runs fine without any issues but suddenly if I try to run it starting earlier than 2018 I get the below error.

Something went wrong. Sorry for the inconvenience. Try using the built-in debugger to analyze your code. If you would like to help, send us an email.

but earlier start dates such as 1/1/2019 runs ok?! its seems like a data issue !!>>

8 responses

To be precise it fails as I pull the date before 11/01/2016

Very strange !!!

Sorry to see you are getting errors. If you are still seeing this problem perhaps attach a backtest. It would be easier to help troubleshoot by seeing the code. Maybe as a start, try commenting out sections of code to narrow down the offending line(s)?

Disclaimer

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.

Dan, Thanks for offering to help. It seems like something broke yesterday !!! Perhaps has to do something with the release of some of the new datasets

Here is a sample code. It fails to start before 2016 but works fine in 2017. I stripped this off a piece of perfectly working algo that suddenly fails to execute before a certain date. It seems like something happened to the filter model! My other filter that's in the code too fails during the same date ranges!

Clone Algorithm
3
Loading...
Backtest from to with initial capital
Total Returns
--
Alpha
--
Beta
--
Sharpe
--
Sortino
--
Max Drawdown
--
Benchmark Returns
--
Volatility
--
Returns 1 Month 3 Month 6 Month 12 Month
Alpha 1 Month 3 Month 6 Month 12 Month
Beta 1 Month 3 Month 6 Month 12 Month
Sharpe 1 Month 3 Month 6 Month 12 Month
Sortino 1 Month 3 Month 6 Month 12 Month
Volatility 1 Month 3 Month 6 Month 12 Month
Max Drawdown 1 Month 3 Month 6 Month 12 Month
from pandas import Series, DataFrame
import pandas as pd
#import statsmodels
import statsmodels.api
import statsmodels.api as sm

import scipy.stats as stats
# import datetime as dt
from datetime import timedelta
import numpy as np
import talib
import math
from scipy.stats import zscore

## quantopian libs
from quantopian.algorithm import attach_pipeline, pipeline_output
from quantopian.pipeline import Pipeline, CustomFactor
from quantopian.pipeline.data.builtin import USEquityPricing
from quantopian.pipeline.classifiers.morningstar import Sector
from quantopian.pipeline.data import Fundamentals
from quantopian.pipeline.factors import Latest , Returns, RollingLinearRegressionOfReturns, SimpleMovingAverage, SimpleBeta
#from brokers.ib import VWAPBestEffort, RelativeOrder, IBExchange
#from quantopian.pipeline.data import factset
from quantopian.pipeline.filters import Q3000US, Q1500US

from operator import itemgetter

def initialize(context):
    attach_pipeline(make_pipeline(), 'my_pipeline')
    
    #set_symbol_lookup_date('2016-1-30')
    context.benchmarkSecurity = symbol('IWM')
    context.spysym = symbol('SPY')
    context.no_trade_sym = [ symbol('TLT')]
     
    
    
#############################################################################################################################
#############################################################################################################################
def make_pipeline():
    """
    A function to create our dynamic stock selector (pipeline). Documentation on
    pipeline can be found here: https://www.quantopian.com/help#pipeline-title
    """
    vsma = SimpleMovingAverage(inputs=[USEquityPricing.volume], window_length=3)
    
    # Equities that listed as common stock (as opposed to, say, preferred stock).
    # This is our first string column. The .eq method used here produces a Filter returning
    # True for all asset/date pairs where security_type produced a value of 'ST00000001'.    
    common_stock = Fundamentals.security_type.latest.eq('ST00000001')
    
    # Equities with a null entry for the balance_sheet.limited_partnership field.
    # This is an alternative way of checking for LPs.    
    not_lp_balance_sheet = Fundamentals.limited_partnership.latest.isnull()
    
    # Equities whose company name ends with 'LP' or a similar string.
    # The .matches() method uses the standard library `re` module to match
    # against a regular expression.    
    not_lp_name = ~Fundamentals.standard_name.latest.matches('.* L[\\. ]?P\.?$')
    LP = ~Fundamentals.standard_name.latest.matches(r'.* L[.]?P[.]?$')
    no_llc =  ~Fundamentals.standard_name.latest.matches(r'.* L[.]?C?$')
    #not_partners_name = morningstar.balance_sheet.general_partnership_capital.latest.eq(0)
    
    # Must have market cap data
    have_data = Fundamentals.market_cap.latest.notnull()
    
    # Equities whose exchange id does not start with OTC (Over The Counter).
    # startswith() is a new method available only on string-dtype Classifiers.
    # It returns a Filter.    
    not_otc = ~Fundamentals.exchange_id.latest.startswith('OTC')
    
    # Equities whose symbol (according to morningstar) ends with .WI
    # This generally indicates a "When Issued" offering.
    # endswith() works similarly to startswith().    
    not_wi = ~Fundamentals.symbol.latest.endswith('.WI')
    not_wi_ = ~Fundamentals.symbol.latest.endswith('_WI')
    #not_ADR = ~morningstar.share_class_reference.symbol.latest.endswith('LTD')
    
    # Equities that morningstar lists as primary shares.
    # NOTE: This will return False for stocks not in the morningstar database.    
    primary_share = Fundamentals.is_primary_share.latest
    #not_aq_target = ~IsAnnouncedAcqTarget()
     
    not_adr = ~Fundamentals.is_depositary_receipt.latest
    
    not_foreign =  ~Fundamentals.legal_name.latest.endswith('PLC') & \
                      ~Fundamentals.legal_name.latest.endswith('LTD') & \
                      ~Fundamentals.legal_name.latest.endswith('AG') & \
                      ~Fundamentals.legal_name.latest.endswith('ADR')
                
    tradable_filter = (common_stock & not_lp_name  & not_lp_balance_sheet & LP &
                       have_data & not_otc & not_wi & not_wi_ & primary_share & no_llc & 
                      Fundamentals.market_cap.latest.top(2500) & not_foreign )
                    
   
    pipe = Pipeline(
                    columns={
                             'fcf':Fundamentals.free_cash_flow.latest,
                             'operating_cash_flow': Fundamentals.operating_cash_flow.latest,
                           
                             'name': Fundamentals.short_name.latest,
                             'alt_name': Fundamentals.legal_name.latest,
                           
                             'primary_share':Fundamentals.is_primary_share.latest,
                             'sec_type': Fundamentals.security_type.latest,
                             'market_cap': Fundamentals.market_cap.latest,
                             'sector': Fundamentals.morningstar_sector_code.latest,
                             
                             'debt': Fundamentals.total_debt.latest,
                             'symbol': Fundamentals.symbol.latest,
                             'industry': Fundamentals.morningstar_industry_code.latest,
                           
                            'entval': Fundamentals.enterprise_value.latest,
                      
                              'adr': Fundamentals.is_depositary_receipt.latest,
                          
                             'price': USEquityPricing.close.latest,
                             'volume':  USEquityPricing.volume.latest,
                        
                           
                              'pb_ratio': Fundamentals.pb_ratio.latest,
                              'pe_ratio': Fundamentals.pe_ratio.latest,
                              'ps_ratio':  Fundamentals.ps_ratio.latest,
                              'fcf_ratio': Fundamentals.priceto_cash_ratio.latest,
       
                             
                            },
    screen=Q1500US())
    return pipe
#############################################################################################################################
#############################################################################################################################
def before_trading_start(context, data):
   
    df = pipeline_output( 'my_pipeline')
    print(df.shape)
   
#############################################################################################################################

def handle_data(context, data):
    pass
There was a runtime error.

@ kamran

Something is odd with the behavior of Fundamentals.priceto_cash_ratio.latest. If you delete that line from the pipe definition it seems to work. Check if you experience the same. We're looking at it internally to see what's going on.

Attached is a stripped down version which highlights the issue. It runs starting in 2017 but generates an error if started in 2016. Actually it seems to start around October 27, 2016.

Again, sorry for the inconvenience.

Clone Algorithm
2
Loading...
Backtest from to with initial capital
Total Returns
--
Alpha
--
Beta
--
Sharpe
--
Sortino
--
Max Drawdown
--
Benchmark Returns
--
Volatility
--
Returns 1 Month 3 Month 6 Month 12 Month
Alpha 1 Month 3 Month 6 Month 12 Month
Beta 1 Month 3 Month 6 Month 12 Month
Sharpe 1 Month 3 Month 6 Month 12 Month
Sortino 1 Month 3 Month 6 Month 12 Month
Volatility 1 Month 3 Month 6 Month 12 Month
Max Drawdown 1 Month 3 Month 6 Month 12 Month
"""
This is a template algorithm on Quantopian for you to adapt and fill in.
"""
import quantopian.algorithm as algo
from quantopian.pipeline import Pipeline

from quantopian.pipeline.data import Fundamentals

def initialize(context):
    algo.attach_pipeline(make_pipeline(), 'pipeline')


def make_pipeline():
    pipe = Pipeline(
        columns={'fcf_ratio':Fundamentals.priceto_cash_ratio.latest},
    )
    return pipe

def before_trading_start(context, data):
    context.output = algo.pipeline_output('pipeline')
    log.info("it works")

There was a runtime error.

I removed the field and I can run it now. It's strange as I had removed a bunch of these fields before and it also failed a few times.

As I mentioned this broke yesterday for some reason. I am not sure if it's related to any of the new data sets. Also, we had a failed data load on Monday. Don't know if any of it is related but thought to mention it,

Thanks again for your help.

Dan here is another issue!

if you add the below screen to the pipe (with a new data column) the backtest fails again starting before 2008! It runs ok after 1/1/2008 !!!

def make_pipeline():
pipe = Pipeline(
columns={'fcf':Fundamentals.free_cash_flow.latest,},
screen= Fundamentals.limited_partnership.latest.isnull()
)
return pipe

Additionally, its late Saturday afternoon and the Friday's data is still not loaded in the system. Something is going on with the data at Q lately!

This also is failing to start again starting 2003!

Also, this field is an issue too

Fundamentals.limited_partnership.latest,

Would be great to get to the bottom of this issue.

Thanks