Back to Community
Status non-tradeable Futures/Indices in blotter... (new feature)

The more I work with fetcher the more I'm convinced that this is real poor mans solution to trading with VIX premia curves, WTI prices, Indices that do not have a tradeable ETF (Like HUI for making an algo to trade NUGT and DUST). As I really believe that this would make a huge difference to the variety of algorithms one could create (as impactful as fundamental data I think) I wanted to ask the dear Quantopianimals when this is to be expected on the roadmap. Just to know so I can determine whether its worth it to learn zipline and/or R to experiment with these algo's. I rather not...
It would be awesome if these non-tradeables would be recognisable, I think Yahoo uses a ^ in front: ^HUI, and futures have their own structure: Oil delivery Dec 2015: CLZ15.

Would be helpful if the community, the Quantopitribe, would concur (+1), or Heaven forbid, disagree (-1).

10 responses

Peter, seems like you have a lot packed in here! Could you tell me more about what you're looking for? And maybe with an example?

Disclaimer

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.

What precisely is the problem with using fetcher?

(I get that it's not optimal, but it seems far more convenient to me than using Zipline offline.)

It's too easy to cause look-ahead bias, it's tedious to massage CSVs from random data sources, the data is rarely available at a higher frequency than daily, you have to twist to get to to work in paper-trading at all by forward-filling the last row (thanks Alisa!), and generally it's super annoying.

Definitely would love to have proper "realtime" index data.

Yes, I agree with those points, but I don't see how they're solved by using Zipline on your own computer or using R.

The more built-in minute data the merrier of course.

Hi Alisa, Let me recap the main issues with fetcher:

Fetcher does not adhere to schedule function: the fetcher data is called every day before trading and most of my alhorithms do things near the closing. When it's called before the open I always work with yesterday's data.

Secondly it's too easy to make mistakes with shifting data in time, I bet most Algos which use fetcher have a look ahead bias in back testing and loose that in real live trading, hence the backtest and live performances are far apart.

Thirdly there are only 5 calls allowed and one call can only be bound to one Symbol. This excludes Algos where one needs more (VIX complex)

When it comes to indices: indices are tickers in Bloomberg and in my terminal I can easily create a strategy with NUGT and DUST based on the behaviour of HUI, the underlying index. Even the simplest MA crossover strategy I cannot create in Quantopian as I only can get the HUI of yesterday....

Same for the VIXs. I would love to create an algo trading changes in the VIX complex, wether it has contango or backwardation structures and how much and with which velocity. If I would be able to call the VIX futures as normal symbols in DATA that would solve it.

So either include a set of indices as callable SID in DATA, or as a patch allow algo developers to call fetcher in a scheduled function or determine the time fetcher will be called as before trading is the most unhelpful time.

Ditto on the bias. The algo I posted last night had one. The results got better and better the earlier in the day I ran my schedule_function until I got very suspicious. Sure enough, I was using the VIX daily close during the morning of.

I'll second (third?) the request for index and even better ETF NAV values.

I want to restate some of the ideas from this thread to make sure I understand them.

  • Peter, Simon, and Kevin are all looking for a live (minutely) feed of a CBOE index, VIX
  • Peter (and others?) are looking for a live (minutely) feed of a NYSE index, HUI

I call them out separately because of the different providers. We've looked at VIX before but we haven't pulled the trigger yet because of the cost of buying the historical and live data from CBOE, plus the en. I've never priced out the cost of the NYSE indices. It's good to hear the requests for them, though, because requests like this help drive the priorities.

Other things from the thread:

  • Peter is looking for prices for commodity futures. This one is actually a work in progress! We've started work on supporting trading futures. The price data will come as we work on that feature.
  • Simon and Peter point out that it's too easy to get look-ahead bias using Fetcher. I agree with that. I haven't found a solution to it that I like yet. The power of Fetcher is that it lets you bring in anything. The challenge is that when you are flexible enough to bring in anything, it's hard to manage. . .
  • The concept of minutely-fetcher is a great one. I want to build it, but we haven't gotten to the point where it is a next-priority yet.
Disclaimer

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.

Kevin would like to have the NAV of ETFs as well. You might be able to kill two birds with this, as having the NAV of many of the index ETFs is a pretty good proxy for the underlying index (although sadly not for VIX).

I think almost every problem can be solved by be able to call fetch_now(fetcher_id) at the time you actually need it as we can use external apps do to the actual databuilding, or use Kimono, or Quandl or Yahoo directly. To make sure the script doesnt timeout in 50 secs, the developer could run a schedule_function at T-1 to get the data and do the work at T-zero or if you know that your fetcher is responsive one could do it it one go.

Moving that higher in the roadmap would allow the algo developers to solve for most issues. except for lookahead bias off course. As I'm planning to run most of my investments through Q at one point I'm more concerned about the fetcher on-demand then the look ahead bias as the look ahead bias will be caught out while paper trading (and comparing it with backtesting the same period).

[I develop, backtest*100, put live in paper-trading, leave for a month, then back test the papertrading period and see if the pattern is the same. If so: great, if not.... back to start.... If all good: put live]