Back to Community
How to use the Fetcher from Quandl Dataset to backtest the strategy via Quantopian?

I want to call multiple Chinese stocks from Quandl using my strategy in the following algo. Can someone give me a hand to allow me to download multiple Chinese stocks using external data set like Quandl and test their performance in Quantopian? Thanks!

The stocks I want to fetch into the algorithm are listed below:
000023.SZ
000020.SZ
000019.SZ
000018.SZ
000014.SZ
000011.SZ
000010.SZ
000008.SZ

Here is my strategy:

from pandas import TimeSeries  
from numpy import NaN

RESISTANCE = 80

def initialize(context):  
    """Initialize context object. It's passed to the handle_data function."""  
    # algo securities and (prev_ma5, prev_ma30, prev_long_trend) placeholders  
    context.secs = {sid(8554): (None, None, None),  
                    sid(33652): (None, None, None)}  
@batch_transform(refresh_period=1, window_length=25)  
def get_df(datapanel, field):  
    # We are looking for the min and the max price to return. Just because it's interesting  
    # we also are logging the current price.  
    return datapanel[field]

def SMMA(tseries, period):  
    """  
    Calculate smoothed moving average as described here:  
        https://mahifx.com/indicators/smoothed-moving-average-smma  
    :param tseries: time series  
    :param period: smoothing period  
    """  
    result = TimeSeries([NaN for i in tseries], tseries.index)  
    # first value: SUM (CLOSE, N)/N  
    result[period] = tseries[:period].sum()/period  
    for i in xrange(period+1, len(tseries)):  
        # second and subsequent values:  
        # SMMA (i) = (SUM1 – SMMA1 + CLOSE (i))/ N  
        result[i] = (tseries[i-period:i].sum() - result[period] + tseries.iget(i))/period  
    return result

def handle_data(context, data):  
    """  
    The main proccessing function.  
    Called whenever a market event occurs for any of algorithm's securities.

    :param context: context object  
    :param data: Object contains all the market data for algorithm securities  
                 keyed by security id. It represents a snapshot of algorithm's  
                 universe as of when this method is called.  
    :returns: None  
    """  
    # calculate parameters, based on historical data  
    high = get_df(data, 'high')  
    if not high:  
        # less than 25 days of data  
        return  
    low = get_df(data, 'low')  
    close = get_df(data, 'close_price')  
    ratio = (close - low)/(high - low)  
    positions = context.portfolio.positions

    for security, (prev_ma5, prev_ma30, prev_long_trend) in context.secs.iteritems():  
        sec_data = data[security]  
        up = SMMA(ratio[security.sid] * 30, 4)[3:]  
        mid = SMMA(up, 3)[2:]  
        ground = SMMA(mid, 2)[1:]  
        long_trend = 9 * up.iget(-1) - 3 * mid.iget(-1) - 2 * ground.iget(-1)

        if get_open_orders(security):  
            # skip the security if it has active orders  
            continue  
        if security in positions:  
            # check exit conditions for open positions  
            if prev_long_trend != None and prev_long_trend >= RESISTANCE \  
                   and long_trend < RESISTANCE:  
                position = positions[security]  
                log.debug('%s: prev_long_trend=%f, long_trend=%f' % (security.symbol,  
                                                                     prev_long_trend,  
                                                                     long_trend))  
                log.info("selling %d shares of %s" % (position.amount, position.sid.symbol))  
                order(position.sid, -position.amount)  
        else:  
            # check enter conditions if no open positions for this security  
            if prev_ma5 != None and prev_ma30 != None and prev_ma5 <= prev_ma30 \  
                     and sec_data.mavg(5) > sec_data.mavg(30):

                log.debug('%s: prev_ma5=%f, prev_ma30=%f, ma5=%f, ma30=%f' % \  
                          (security.symbol, prev_ma5, prev_ma30,  
                           sec_data.mavg(5), sec_data.mavg(30)))

                # calculate amount of cash for the order  
                cash = context.portfolio.cash/(len(context.secs) - len(positions))  
                # and amount of shares to buy  
                shares = int(cash/sec_data.price)  
                log.info("buying %d shares of %s" % (shares, security.symbol))  
                order(security, shares)

        context.secs[security] = (sec_data.mavg(5), sec_data.mavg(30), long_trend)  

Here is the fetcher written by David Edwards

https://www.quantopian.com/posts/use-the-fetcher-for-any-quandl-dataset

I want to combine his fetcher with my strategy to use external data via Quantopian.

5 responses

Hi Qifeng,

Here's a start on the fetcher piece for you (I've also invited you to a collab on this algo). I created a Quandl "superset" with the pricing time-series data for the Shenzen listed securities you are looking for (please double check I found the right tickers). I then pull the daily price series for these in to Quantopian using a single call to fetch_csv. For now I just have this working to print out the prices and record one example stock's price series in a custom plot (000023.sz).

One thing that I need to clarify however is that while you can pull the pricing time-series for this stocks in to Quantopian, you cannot simulate placing orders for them. The order() method will only work for sids in Quantopian's security universe. When pulling in data for securities that cannot be mapped against our investible universe its possible to treat these non-tradeable securities as 'signals' -- that's what my example code is doing.

So I'm not sure how useful it is for you to be able to have access to pricing without being able to simulate trading these stocks.

Let me know if you have any questions or if I can help you further!
Jess

Clone Algorithm
40
Loading...
Backtest from to with initial capital
Total Returns
--
Alpha
--
Beta
--
Sharpe
--
Sortino
--
Max Drawdown
--
Benchmark Returns
--
Volatility
--
Returns 1 Month 3 Month 6 Month 12 Month
Alpha 1 Month 3 Month 6 Month 12 Month
Beta 1 Month 3 Month 6 Month 12 Month
Sharpe 1 Month 3 Month 6 Month 12 Month
Sortino 1 Month 3 Month 6 Month 12 Month
Volatility 1 Month 3 Month 6 Month 12 Month
Max Drawdown 1 Month 3 Month 6 Month 12 Month
# Put any initialization logic here.  The context object will be passed to
# the other methods in your algorithm.
def preview(df):
    
    log.info('\n %s' % df.head())
    return df

def rename_col(df):
    df = df.rename(columns={'SZ_000023 - Adjusted Close': 'SZ_000023'})
    df = df.rename(columns={'SZ_000019 - Adjusted Close': 'SZ_000019'})
    df = df.rename(columns={'SHE_000018 - Close': 'SHE_000018'})
    df = df.rename(columns={'SHE_000014 - Close': 'SHE_000014'})
    df = df.rename(columns={'SZ_000011 - Adjusted Close': 'SZ_000011'})
    df = df.rename(columns={'SZ_000020 - Adjusted Close': 'SZ_000020'})
    df = df.rename(columns={'SZ_000010 - Adjusted Close': 'SZ_000010'})
    df = df.rename(columns={'SZ_000008 - Adjusted Close': 'SZ_000008'})
    
    df = df.fillna(method='ffill')
    #df = df[['price', 'sid']]
    # Correct look-ahead bias in mapping data to times   
    df = df.tshift(1, freq='b')
    log.info(' \n %s ' % df.head())
    return df

def initialize(context):
    context.stock = sid(24)
    #'http://www.quandl.com/api/v1/datasets/YAHOO/SZ_000023.csv?&trim_start=2001-04-29&trim_end=2014-04-22&sort_order=desc', \
    fetch_csv( 'http://www.quandl.com/api/v1/datasets/JESS_STAUTH/8NR.csv?&auth_token=HT9PvUsSJbbpxkZ5E2xE&trim_start=2002-01-01&trim_end=2014-04-22&sort_order=desc',\
          pre_func = preview,
          post_func = rename_col,
          symbol = 'shenzen_securities',
          date_column = 'Date')
    context.chinese_stocks = [
        'SZ_000023',
        'SZ_000020',
        'SZ_000019',
        'SHE_000018',
        'SHE_000014',
        'SZ_000011',
        'SZ_000010',
        'SZ_000008']

# Will be called on every trade event for the securities you specify. 
def handle_data(context, data):
    
    if 'shenzen_securities' in data:
        
        shenzen = data['shenzen_securities']
        if 'SZ_000023' in shenzen:
            record(sample_stock = shenzen['SZ_000023'])
        for stock in context.chinese_stocks:
            if stock in shenzen:
                print shenzen[stock]
There was a runtime error.
Disclaimer

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.

Hi, Jess

Thank you so much for providing me this wonderful solution. But it would be less useful without simulated placing orders for the stocks we chose. Anyway, thank you again for your help. I am just wondering if we can do this in Zipline not via the Quantopian? Can you give me a live example for that? For now, I just know some strategies in Pyalgotrade, and it has tremendous limitations. So I want to transfer to Zipline.

Best,
Qifeng Wu

@Jessica is that 8 chinese stock running as one/ or one index?? or just one stock? and I've notice your using .. context.stock = sid(24) in your code.. but cant see apple in the chart only spy and shenzen... puzzled....??

@Jesicca in quandl... its YAHOO/SZ_000023.csv where.. did you get this 8NR.csv at line 28... ish....

Hi John,

First, I included Apple as a context stock just to make sure I had at least one stock in my universe so that handle_data would be called. This might be updated in our API now, but I think at the time I wrote this if you only included fetcher stocks your universe might be empty and skip some days - so that is possibly legacy code.

Then secondly, this bit:

www.quandl.com/api/v1/datasets/JESS_STAUTH/8NR.csv?&auth_token=HT9PvUsSJbbpxkZ5E2xE&trim_start=2002-01-01&trim_end=2014-04-22&sort_order=desc'

Is the API call to the superset of stocks I built on Quandl. I just checked and it still works, if you copy/paste that url to your browser you should get a csv with the data for those 8 chinese stocks listed and for the time range indicated. I haven't played with supersets on Quandl lately, so it is also possible they have a better way of building stock universes now.

Hope that helps, Jess