Back to Community
Multiple stocks context initialisation function problem

Hi,

I'm relatively new to python and quantopian.

I'm having difficultly writing a function that initialises context variables for each stock that I'm trading. Any help will be appreciated

My code is as follows:


def initalise_context(sid, context):  
    historical_close = history(bar_count=365, frequency='1d', field='close_price')  

    daily_close = historical_close[sid] 

    context.daily_sma30[sid] = ta.SMA(np.array(daily_close), timeperiod=30)  


def initialize(context):  
    context.stocks = symbols('AAPL','MSFT')  
    context.max_notional = 100000.1  
    context.min_notional = 0

    context.daily_sma30 = [] 

def handle_data(context, data):  
    if ( context.daily_sma30 == None ):  
        for sid in context.stocks:  
            initalise_context(sid, context)  

I'm getting a problem saying that

Error   Runtime exception: TypeError: list indices must be integers, not __builtins__.Security  

So, how can I initialise this type of array? What else am I doing wrong?

Thanks in advance for your help.
Andrew

13 responses

Hi Andrew,

The error you're seeing is telling you that, when you do

context.daily_sma30[sid] = ta.SMA(np.array(daily_close), timeperiod=30)  

You're trying to set a value you in your daily_sma30 list with a Security object as a key rather than an integer.
A simpler example of what you're doing would be something like:

In [1]: mylist = [1,2,3]

In [2]: mylist['foo'] = 4  
---------------------------------------------------------------------------  
TypeError Traceback (most recent call last)  
 in ()  
----> 1 mylist['foo'] = 4

TypeError: list indices must be integers, not str  

In python, lists indices have to have integers, and you can only set into a list at a given index if it's already of the right length. For example, with the same list, if you try to add a key at index 0 (the first index) it will work.

In [3]: mylist[0] = 5  
In [4]: mylist  
Out[4]: [5, 2, 3]  

But if you try to add a key at index 6, it will fail, because the list isn't that long

In [6]: mylist[6] = 5  
---------------------------------------------------------------------------  
IndexError Traceback (most recent call last)  
 in ()  
----> 1 mylist[6] = 5

IndexError: list assignment index out of range  

Since, in your case, you want to be storing a map from arbitrary keys to values, what you want to use is a dicitonary. The way to initialize an empty dictionary is with curly braces instead of square brackets.

So in summary, the fix for your issue is to change

context.daily_sma30 = []  

to

context.daily_sma30 = {}  

Also, if you're new to python, I'd wholeheartedly recommend taking a look at the standard docs on the Core Data Structures. Understanding how and when to use list vs dict vs tuple vs set is a big part of using Python effectively. If you're feeling ambitious you could also take a look at Pandas, the library we use for tabular data structures. Among other things, the history() API for loading trailing windows of data is built on top of pandas, and lots of our internal systems use Pandas for storing and manipulating tabular and time series data.

Hope that helps,
-Scott

Disclaimer

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.

Thanks for your help Scott!

Having changed the [] to {} to make context.daily_sma30 an associated array (dictionary item) I'm now getting this error:

Error   Runtime exception: KeyError: Security(24, symbol='AAPL', security_name='APPLE INC', exchange='NASDAQ GLOBAL SELECT MARKET', start_date=datetime.datetime(1993, 1, 4, 0, 0, tzinfo=), end_date=datetime.datetime(2014, 10, 1, 0, 0, tzinfo=), first_traded=None)  

It seems to be happening on line in the handle_data function (not in the original code snippet)

daily_sma30 = context.daily_sma30[sid]  

This is the first time that I try to use context.daily_sma30 dictionary item. In theory, sid should equal 24 (for Apple Inc) but I don't understand why this error is being produced.

Within the intialise_context funciton I've added a line:

context.daily_sma30.insert(sid,ta.SMA(np.array(daily_close), timeperiod=30))  

to make and index of 24 (Apple Inc) to be a list producted from the ta-lib SMA call.

Any clues what"s going on now?

Sorry to be a pain!

Thanks in advance.
Andrew

Andrew,

I think I understand where the issue might be but it'd be great if you could post what your code looks like now? Or if you'd like to keep it private you can send me an invitation to collaborate and I could help you out there.

Seong

Disclaimer

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.

And I realized that I didn't include my email: [email protected]. Feel free to reply here or send me a email!

Seong

Thanks Seong, I've invited you.

Andrew,

Check out the algorithm you've invited me to now. I've changed and commented a few things and let's talk in the chat there when you see this

Seong

I'm also new to Python and am attempting to do something very similar for my first algo on Quantopian. Would you consider sharing your solution or at least the initialise portion?

I'd like to initialize each stock differently based on whether or not it is a new day or not. If its not a new day, i only want to pull 1 day of history and update the last entry of my array. My goal is to set up a few indicators in the context of each stock, for example: context.ADX, context.MACD & context.RSI. Based on whether or not the stock is trending and the respective indicators, I'll individually determine if each stock is a buy, sell, hold or none of the above.

Any help would be greatly appreciated and i'd love to see where your ideas led and share mine.

[email protected]
-Austin

Austin,

I'm not able to share Andrew's solution unless he chooses to, but if you'd like, you could post what you have here and myself and others will be able to help you out (and we can help out anyone else who might have the same questions). In the meanwhile I can post a few code snippets that might be useful:

Pulling 1 day of history

price_of_stock = history(1, "1d", "price")["sid name here"][-1]  

Updating the last element of an array (list)

#: Initialize a new list  
new_list = [1,2,3,4]  
#: Update the last element  
new_list[-1] = 5  
#: new_list is now [1,2,3,5]  

Feel free to ask any more questions on this!

Seong

Thanks for your help Seong, it aided me greatly.

Austin, I've no problem sharing my code with Seongs corrections. Hope it helps:

import math  
import talib as ta  
import pandas as pd  
import numpy as np

def initalise_context(sid, context):  
    historical_close = history(bar_count=365, frequency='1d', field='close_price')  

    daily_close = historical_close[sid]  
    #: Instead of insert use this notation which essentially says:  
    #: In this dictionary, update this key (sid) with this value (ta.SMA(np.array(daily_close), timeperiod=30)) and if it doesn't exist  
    #: Create a new key (sid) with this value (ta.SMA(np.array(daily_close), timeperiod=30))  
    context.daily_sma30[sid] = ta.SMA(np.array(daily_close), timeperiod=30)  


def process_order(sid):  
    return  


def initialize(context):  
    context.stocks = [symbol('AAPL'),symbol('MSFT')]  
    context.max_notional = 100000.0  
    context.min_notional = 0

    context.day = None  
    context.daily_sma30 = {}  

def handle_data(context, data):  
    #: Instead of asking context.daily_sma30 == None, ask if the length of it is 0  
    #: Because None only applies if the object is actually None, where in this case it isn't because 'context.daily_sma30 = {}' assigns  
    #: context.daily_sma30 to a dictionary object  
    if len(context.daily_sma30) == 0:  
        for sid in context.stocks:  
            initalise_context(sid, context)  
    for sid in context.stocks:  
        close = data[sid].close_price  
        daily_sma30 = context.daily_sma30[sid]  
        if get_datetime().day == context.day:  
            return  
        context.day = get_datetime().day  
        initalise_context(sid, context)  

Good Luck Austin and thanks again Seong!
Andrew

Hi everyone,

I want to do something similar to Andrew but my question is not as python specific. Let's say I want to build an algorithm that buys when the all time historic closing price of a security has been exceeded. For example, security XYZ's highest closing price ever is $10. I need to be able to know that and every minute check/ update that value so I know when to buy. It's similar in that the original question where Andrew wants to pre-calculate SMA for a security for the previous year before running his algorithm. I want the highest closing price of all time --- even including years prior to the quantopian dataset. It's a similar class of problems... Can I programatically generate aggregated historical data before my algorithm runs?

I know I can use fetcher to get this data from a csv file.... But can I use quandl or any other service to generate a csv file that quantopian can read in that will give me the all-time highest closing price of a security? Or do I have to do this offline on my own? I just got the research product beta but I haven't played with it enough to know if that's what can do this for me. Maybe that's where I should look?

Thanks!

Hello Brett,

I've attached some code that may solve your problem. The trick here is that 'history' returns trailing daily highs and the high for the current day, up to the current backtest minute.

Grant

(Note that pandas does not need to be imported. I thought I'd end up using it, but didn't need it.)

import pandas as pd

def initialize(context):  
    context.stocks = [ sid(19662),  # XLY Consumer Discrectionary SPDR Fund  
                       sid(19656),  # XLF Financial SPDR Fund  
                       sid(19658),  # XLK Technology SPDR Fund  
                       sid(19655),  # XLE Energy SPDR Fund  
                       sid(19661),  # XLV Health Care SPRD Fund  
                       sid(19657),  # XLI Industrial SPDR Fund  
                       sid(19659),  # XLP Consumer Staples SPDR Fund  
                       sid(19654),  # XLB Materials SPDR Fund  
                       sid(19660) ] # XLU Utilities SPRD Fund  
def handle_data(context, data):  
    high_daily = history(2520,'1d','high')  
    peak = high_daily.max(axis=0)  
    peak_XLY = peak[context.stocks[0]]  
    record(peak_XLY = peak_XLY)  
Clone Algorithm
7
Loading...
Total Returns
--
Alpha
--
Beta
--
Sharpe
--
Sortino
--
Max Drawdown
--
Benchmark Returns
--
Volatility
--
Returns 1 Month 3 Month 6 Month 12 Month
Alpha 1 Month 3 Month 6 Month 12 Month
Beta 1 Month 3 Month 6 Month 12 Month
Sharpe 1 Month 3 Month 6 Month 12 Month
Sortino 1 Month 3 Month 6 Month 12 Month
Volatility 1 Month 3 Month 6 Month 12 Month
Max Drawdown 1 Month 3 Month 6 Month 12 Month
import pandas as pd

def initialize(context):
    context.stocks = [ sid(19662),  # XLY Consumer Discrectionary SPDR Fund   
                       sid(19656),  # XLF Financial SPDR Fund  
                       sid(19658),  # XLK Technology SPDR Fund  
                       sid(19655),  # XLE Energy SPDR Fund  
                       sid(19661),  # XLV Health Care SPRD Fund  
                       sid(19657),  # XLI Industrial SPDR Fund  
                       sid(19659),  # XLP Consumer Staples SPDR Fund   
                       sid(19654),  # XLB Materials SPDR Fund  
                       sid(19660) ] # XLU Utilities SPRD Fund
 
def handle_data(context, data):
    
    high_daily = history(2520,'1d','high')
    peak = high_daily.max(axis=0)
    
    peak_XLY = peak[context.stocks[0]]
    
    record(peak_XLY = peak_XLY)
There was a runtime error.

Thanks Grant.

I'm looking to find a way to do almost exactly what Austin mentioned above. Has anyone created or found an example that shows how to precalculate the indicators for multiple stocks for comparison?