Back to Community
help w/ fetcher

I'm trying to get a fetcher file into my algo, but it is a no-go.

The file looks like this:

day,my_data
5/4/2015,1
5/5/2015,2
5/6/2015,3
5/7/2015,4
5/8/2015,5

And my code is:

def initialize(context):  
    fetch_csv('https://www.dropbox.com/s/i1teg8xcokks6fl/fetcher_test.csv?dl=0',  
               date_column = 'day',  
               date_format = '%m/%d/%y',  
               symbol = 'my_data')  
    context.spy = sid(8554)  
def handle_data(context, data):  
    pass  

I am trying to run the algo on minute bars, with a start date of 5/4/2015 and an end date of 5/8/2015.

I get the error:

Exception: Error parsing remote CSV data.
There was a runtime error on line 5.

Grant

10 responses

Edit: Solved - see Ian's comment below

I'm not a dropbox user, but if you copy your link into a browser it displays a page that asks for a login/ignore. Is there a public link that dropbox provides so that the link when entered into a browser just displays the file (without the login/ignore page)?

Edit: Make sure you are not currently logged into dropbox to test this.

Thanks Rob,

I get the same thing with Firefox, but I can bypass the link asking me to login, and subsequently download the file. I just pasted the link into my Chrome browser (after logging out of dropbox), and it does ask me to login, but if I bypass this, I can download.

Maybe there is an option in dropbox to avoid the login request? Maybe this is getting in the way?

Kinda confusing, since the Quantopian error makes it sound like the file is accessible, but just can't be parsed.

Grant

Grant,

Try this url and see if it works. (I just copied your file to a test server of mine).

Edit: URL removed

Works! So I wonder what the trick is with dropbox? Or I could use an alternative service? Any recommendations? Thanks, Grant

Clone Algorithm
9
Loading...
Backtest from to with initial capital
Total Returns
--
Alpha
--
Beta
--
Sharpe
--
Sortino
--
Max Drawdown
--
Benchmark Returns
--
Volatility
--
Returns 1 Month 3 Month 6 Month 12 Month
Alpha 1 Month 3 Month 6 Month 12 Month
Beta 1 Month 3 Month 6 Month 12 Month
Sharpe 1 Month 3 Month 6 Month 12 Month
Sortino 1 Month 3 Month 6 Month 12 Month
Volatility 1 Month 3 Month 6 Month 12 Month
Max Drawdown 1 Month 3 Month 6 Month 12 Month
# Backtest ID: 554f715b63ae800e4ba40ade
There was a runtime error.

Great. You could use Google Sheets (free with a Google account) and output as a CSV file. Just click "Share" and generate a public link (viewable by anyone without a sign-in) and then you have to tweak the url it gives you to generate CSV output. Try this link I just made of your file (and note the "/export?format=csv" addition):

https://docs.google.com/spreadsheets/d/17jKqzzDfpmPb7TlXMuvc7aWOspIuBdzw_35f2OGCE6I/export?format=csv

Hey Grant,

You can replace the 'www.dropbox.com' domain portion of any dropbox share URL with the domain 'dl.dropboxusercontent.com', i.e.,

https://dl.dropboxusercontent.com/s/i1teg8xcokks6fl/fetcher_test.csv?dl=0

Cheers,
Ian

Disclaimer

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.

Ian - much easier!

You may still want to consider using Google Sheets as a handy way to manually enter data or automatically integrate multiple data sources into one fetcher file since it allows for XML/HTML/CSV imports from other sources. I'm considering doing this based on Grant's suggestion here to "remote control" a running algorithm.

I use a public folder in Dropbox which means I can right-click any file and choose "copy public link". It really is extremely convenient. You have to be a paying user for that feature though.

Thanks all for the help. Another question. The code below results in the error:

24 Error Runtime exception: AttributeError: 'SIDData' object has no attribute 'custom'

It will run if I use the standard attribute, 'price', but I'd like to use some custom ones. Is this possible? Or is there an alternative approach?

Grant

import pandas

def rename_col(df):  
    # df = df.rename(columns={'my_data': 'price'})  
    df = df.rename(columns={'my_data': 'custom'})  
    return df

def initialize(context):  
    # fetch_csv('https://www.dropbox.com/s/i1teg8xcokks6fl/fetcher_test.csv?dl=0',  
    #            date_column = 'day',  
    #            date_format = '%m/%d/%y',  
    #            symbol = 'my_data')  
    fetch_csv('http://205.186.154.106/q/fetcher_test.csv',  
               date_column = 'day',  
               date_format = '%m/%d/%y',  
               symbol = 'my_data',  
               post_func=rename_col)  
    context.spy = sid(8554)  
def handle_data(context, data):  
    # print data['my_data'].price  
    print data['my_data'].custom  

Drop box has terminated the public link. I try to use the google sheet, but it does not work either. I get the following error:
AttributeError: 'DataFrame' object has no attribute 'Date'

This is my link:
https://docs.google.com/spreadsheets/d/13-EFqu0D-8Q9L_9fjWaf8vr2SWgFm0_vgUeQqWarZg0/export?format=csv

Can someone help me with this?