Back to Community
Upgrading to Python 3 - Update

Hey everyone,

An update on our migration from Python 2.7 to Python 3.5. Since our [initial announcement][1] we have been feverishly converting things over to Python 3. Everything has progressed smoothly.

As an interim step, we had added a feature, in both the IDE and notebooks, to run either Python 2.7 or 3.5. Hopefully, this helped in troubleshooting and keeping any existing code running. Our next step however, is to phase out this feature and remove the Python 2.7 environment entirely from Quantopian. The plan is to remove Python 2.7 on March 10, 2020.

What does this mean for you? For most users the transition will be a non-event. If you have any code currently written in 2.7, simply go into the IDE or notebook and select Python 3.5. This will verify it runs in 3.5. Also, ensure any new algos and notebooks are coded using Python 3.5. In the future, most old algorithms and notebooks should run. However, the auto-conversion tool will still be available so you can easily convert any of those old algorithms to Python 3.5. Also note that new contest entries must be coded in Python 3.5. However, existing contest entries coded in Python 2.5 will continue to run 'as is' and won't require changes.

If you do encounter errors when updating Python 2.7 code , you may want to check these common issues.

print statements now require parenthesis.

# Python 2 could print like this  
print 'hello world'  
# Python 3 requires parenthesis like this  
print('hello world')  

.iteritems(), .itervalues(), and .iterkeys()are no longer supported for iterating dictionaries. Replace with .items(), .values(), and .keys(). Check out [this explanation][2] for specific details.

# In Python 2 iterating over a dictionary, such as `context.portfolio`, could be done  
for stock, position in context.portfolio.iteritems():  

# In Python 3 iterating over a dictionary should be done  
for stock, position in context.portfolio.items():  

Division is always 'true' division in Python 3 and returns a real number. This actually often created problems in Python 2 so it may be a welcome relief for most. However, this can cause some inadvertent errors. We have seen this show up as "ValueError: At based indexing on an integer index can only have integer indexers". This happens because ones code calculated an index through division of two integers. In Python 2 the result would always be an integer. In Python 3 it will be a real number. Using a real number as an index can often create issues. If you get some sort of "non-integer index" error then look for possible issues with division.

# In Python 2 dividing two integers would result in an integer. 'result1' will equal 0 and 'result2' will equal .5.  
# result1' will not equal  'result2'  
# In Python dividing two integers would result in an real number. 'result1' and result2' will both equal .5  
result1 = 1 / 2  
result2 = 1.0 / 2.0  

If you are having issues converting any of your algorithms, please feel free to reach out to our support team.

[1]: https://www.quantopian.com/posts/upgrading-to-python-3
[2]: https://www.python.org/dev/peps/pep-0469/

Disclaimer

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.

8 responses

Any updates on the daily contest results? Still stuck at feb 14th...

Hi Dan,

fetch_csv seems to be failing in Python 3.5. I get "TypeError: string argument expected, got 'bytes'". Can this be fixed?

@Chris That error is typically the result of the import file not including encoding information (eg utf-8). Does your code work if it's run under Python 2.7?

For starters, the fetch_csv method will work in Python 3.5. Attached is a VERY simple algo which uses it and it runs without errors. The format of the file is below. It's a Google sheets file which does play nicely with fetch because Google includes the encoding information.

symbol  date    score  
A   2018-06-04  0.9191565439  
AAL 2018-06-04  0.717061033  
AAP 2018-06-04  0.02735539376  
AAPL    2018-06-04  0.9298507041  
ABBV    2018-06-04  0.3560910446

Not all download sources include the encoding information and therefore may not play so nicely. As a test, maybe copy the data into a Google sheet and then import that new sheet?

That said, the fetch_csv method isn't well supported anymore and we are recommending using self serve data and pipeline instead (https://www.quantopian.com/docs/user-guide/tools/self-serve).

Clone Algorithm
10
Loading...
Total Returns
--
Alpha
--
Beta
--
Sharpe
--
Sortino
--
Max Drawdown
--
Benchmark Returns
--
Volatility
--
Returns 1 Month 3 Month 6 Month 12 Month
Alpha 1 Month 3 Month 6 Month 12 Month
Beta 1 Month 3 Month 6 Month 12 Month
Sharpe 1 Month 3 Month 6 Month 12 Month
Sortino 1 Month 3 Month 6 Month 12 Month
Volatility 1 Month 3 Month 6 Month 12 Month
Max Drawdown 1 Month 3 Month 6 Month 12 Month
def initialize(context):  
    # set_symbol_lookup_date('2018-05-01')  
    file_url = 'https://docs.google.com/spreadsheets/d/e/2PACX-1vQ2wjr1uvpYxGZssaBFmpFripOXUIWD357hGCKLP0hd0eZRDOUkRH9c5_Dck4FuUs2SoteIbFUtjbsK/pub?gid=1510738953&single=true&output=csv'  
    fetch_csv(file_url,  
          date_column='date',  
          date_format='%Y-%m-%d',  
          mask=False,  
          timezone='EST'  
          )

def before_trading_start(context, data):  
    pass

def handle_data(context, data):  
    pass
There was a runtime error.

Thanks, Dan. Once I patched the content type, it worked in Python 3.5.

Hi @Chris

I have the same issue.
How did you patch/initialize content type?

Hi @ Dan
I tried it in my old algorithm (written in 2.7, it doesn't work now in 3.5), but it doesn't seem to support "pre_func" on fetch_csv.
Some idea? Where did I go wrong?
I am a rookie and it takes me a long time to build it. If anyone can help me it will be appreciated.
I enclose only the small part.
Thanks Michele

def add_fields(df):  
    prices_op = df['VIX Open']  
    prices_op_1 = df['VIX Open'].shift(1)  
    prices_cl_1 = df['VIX Close'].shift(1)  
    prices_cl_2 = df['VIX Close'].shift(2)  
    prices_lw_1 = df['VIX Low'].shift(1)  
    prices_hi_1 = df['VIX High'].shift(1)  
    ma120 = prices_op.rolling(window=120,center=False).mean()  
    #ma200 = prices_op.rolling(window=200,center=False).mean()  
    #macd = ma60 - ma200  
    var1 = 100*(prices_cl_1/prices_cl_2 - 1)  
    var2 = 100*(prices_op/prices_cl_1 - 1)  
    df['MA120'] = ma120  
    #df['MA200'] = ma200  
    #df['MACD'] = macd  
    df['OPEN_1'] = prices_op_1  
    df['CLOSE_1'] = prices_cl_1  
    df['CLOSE_2'] = prices_cl_2  
    df['LOW_1'] = prices_lw_1  
    df['HIGH_1'] = prices_hi_1  
    df['var1'] = var1  
    df['var2'] = var2  
    #df['CUMVAR'] = var.rolling(window = 5).sum().shift(0)  
    #log.info('\n%s ' % df.tail())  
    return df




def initialize(context):  
    vixurl = 'http://www.cboe.com/publish/scheduledtask/mktdata/datahouse/vixcurrent.csv'  
    fetch_csv(vixurl,  
              symbol='Vix',  
              skiprows=1,  
              date_column='Date',  
              pre_func=add_fields)  


def before_trading_start(context, data):  
    pass  
def handle_data(context, data):  
    pass  

Yes, can someone please help with how to solve the content type issue?

The workaround of fetch CSV from Google Sheets is broken too since recent changes and it seems like there are no plans to fix it.

No, I don't want to spend more time figuring out the self serve data dashboard, importing my data, etc. I don't know how Pipeline works, and I don't want to learn. I have dozens of algos I wrote in 2019, I want to test them out of sample to see how well they did, and if you're making me spend more time on it, I might as well as migrate away and find a better platform.

All this experience has told me is that I cannot rely on Quantopian for anything, and it's better to get away from this platform (and stop sharing algos with a third party).