Back to Posts
Listen to Thread

I've encountered several problems with the batch transform, get_data, as I've implemented it in the attached algorithm:

  1. Sporadically, get_data will return data with NaNs. Why the NaNs? And what is the recommended way of dealing with them?
  2. It is inefficient. Any recommendations on speeding up my code? The bottleneck is the call to get_data. Note that I'm just want to return the data in a numpy ndarray--I don't need the fancy pandas stuff.

Grant

Clone Algorithm
5
Loading...
Backtest from to with initial capital ( data)
Cumulative performance:
Algorithm Benchmark
Custom data:
Week
Month
All
Total Returns
--
Alpha
--
Beta
--
Sharpe
--
Sortino
--
Information Ratio
--
Benchmark Returns
--
Volatility
--
Max Drawdown
--
Returns 1 Month 3 Month 6 Month 12 Month
Alpha 1 Month 3 Month 6 Month 12 Month
Beta 1 Month 3 Month 6 Month 12 Month
Sharpe 1 Month 3 Month 6 Month 12 Month
Sortino 1 Month 3 Month 6 Month 12 Month
Information Ratio 1 Month 3 Month 6 Month 12 Month
Volatility 1 Month 3 Month 6 Month 12 Month
Max Drawdown 1 Month 3 Month 6 Month 12 Month
This backtest was created using an older version of the backtester. Please re-run this backtest to see results using the latest backtester. Learn more about the recent changes.
There was a runtime error.

Hi Grant,

It's curious that you get nans, those should be cleaned by default. They might be a result of using a universe, can you try without that?

In terms of speed, you might be interested in recent efforts to refactor the batch_transform. This leads to a 100x speed-up: https://github.com/quantopian/zipline/pull/136

Also, I think it might be a good idea to add a get_data function to quantopian that just returns the DataPanel. That way you wouldn't have to create a single-line function all the time.

Thomas

Thanks Thomas,

I'll see if I can reproduce the problem with a definite list of sids, however, I'm interested in using set_universe.

Glad to hear that you are working on speeding up the batch transform...please keep me posted.

Regarding adding a get_data function to Quantopian, I wouldn't put it at the top of the priority list. In fact, I'd rather not return the entire DataPanel, but just the data I need for a given algorithm, in a numpy ndarray (or another data structure directly useable in the algorithm).

By the way, perhaps you guys are already thinking along these lines, but it'd be nice if we could eliminate some/all of the loops in Quantopian. For example, the order function could accept vectors of sids and corresponding number of shares, rather than having to loop though the sids. This is pretty much the paradigm in MATLAB and it tends to result in more readable code.

Grant

Hello Thomas,

If the algorithm spits out NaNs and I capture the offending sids, I can re-run it, listing the sids explicitly (see attached). Then, the backtester automatically resets the dates of the backtest and avoids the NaNs (as best I can tell). I figure that securities are popping in and out of existence, or maybe the set of securities delivered by set_universe is changing, and NaNs result.

I suppose I can just test for NaNs and skip the tic if there are any in the data...or I can eliminate columns from the data matrix that contains NaNs (perhaps the best approach).

Grant

Clone Algorithm
5
Loading...
Backtest from to with initial capital ( data)
Cumulative performance:
Algorithm Benchmark
Custom data:
Week
Month
All
Total Returns
--
Alpha
--
Beta
--
Sharpe
--
Sortino
--
Information Ratio
--
Benchmark Returns
--
Volatility
--
Max Drawdown
--
Returns 1 Month 3 Month 6 Month 12 Month
Alpha 1 Month 3 Month 6 Month 12 Month
Beta 1 Month 3 Month 6 Month 12 Month
Sharpe 1 Month 3 Month 6 Month 12 Month
Sortino 1 Month 3 Month 6 Month 12 Month
Information Ratio 1 Month 3 Month 6 Month 12 Month
Volatility 1 Month 3 Month 6 Month 12 Month
Max Drawdown 1 Month 3 Month 6 Month 12 Month
This backtest was created using an older version of the backtester. Please re-run this backtest to see results using the latest backtester. Learn more about the recent changes.
There was a runtime error.

Thanks for checking that.

Yes, the problem are actually not the nans -- there is no way to avoid having missing data (e.g. if a security didn't trade on that event). By default, however, the batch_transform already removes those (using datapanel.dropna()) so I was surprised that it didn't do that in this case. In any case, I opened an issue here: https://github.com/quantopian/zipline/issues/140 to get to the bottom of this.

Thanks for the repro example Grant. That was immensely helpful. I don't know what's wrong yet, but I sure can make it happen here.

Following up on this problem led me down a rabbit hole of a bunch of bugs and code changes, but in the end I think they're all irrelevant. I mention them only to tell you why it took me so long!

I think the NaNs you are seeing are expected. I took your first shared example, for instance, and tracked down the NaNs I saw there, The cause? A two-month trading suspension of RGA because of a spin-off activity from MetLife. It feels like a one-off, but there are one-offs everywhere!

When we built set_universe we tried to avoid NaNs because they were a pain. We just can't avoid them, it seems - they are everywhere.

You probably have read the line in the help document where it says "If a stock is not present for the full quarter (due to an IPO, merger, bankruptcy, or other corporate action) it is excluded from the universe for that quarter. This behavior will be modified in the future to minimize any survivorship bias." We were trying to avoid NaNs, but we knew we couldn't forever.

I think your early question, "what is the recommended way of dealing with them?" is the key one. I don't have any specific advice for you yet. Let's see what other quants do with the problem, and we'll see how to add it to the product.

Thanks Dan...Grant

Log in to reply to this thread.
Not a member? Sign up!