Back to Community
Proper way to remove NaN Data?

I've simplified my issue to hopefully make it easier to debug.

basically when I try to use set_universe() for some dates sometimes I get results back which contain NaN values. I'm not sure how to properly eliminate these from my dataset.

For my example I've used 'GOOG' which I know will return NaN. If you change this to 'GOOG_L' then the algo will run, however it pops up again when I try to use set_universe because I can't account for all securities.

I've tried using the dropna() method, however that seems to provide mixed results depending on where I use it, so I think I may not fully understand the how/when to use this method.

any ideas?

Clone Algorithm
5
Loading...
Backtest from to with initial capital
Total Returns
--
Alpha
--
Beta
--
Sharpe
--
Sortino
--
Max Drawdown
--
Benchmark Returns
--
Volatility
--
Returns 1 Month 3 Month 6 Month 12 Month
Alpha 1 Month 3 Month 6 Month 12 Month
Beta 1 Month 3 Month 6 Month 12 Month
Sharpe 1 Month 3 Month 6 Month 12 Month
Sortino 1 Month 3 Month 6 Month 12 Month
Volatility 1 Month 3 Month 6 Month 12 Month
Max Drawdown 1 Month 3 Month 6 Month 12 Month
# Backtest ID: 5455393addcd7d090a33a62a
There was a runtime error.
4 responses

oops. Looks like the backtest needs to succeed before the source code is available.

Clone Algorithm
5
Loading...
Backtest from to with initial capital
Total Returns
--
Alpha
--
Beta
--
Sharpe
--
Sortino
--
Max Drawdown
--
Benchmark Returns
--
Volatility
--
Returns 1 Month 3 Month 6 Month 12 Month
Alpha 1 Month 3 Month 6 Month 12 Month
Beta 1 Month 3 Month 6 Month 12 Month
Sharpe 1 Month 3 Month 6 Month 12 Month
Sortino 1 Month 3 Month 6 Month 12 Month
Volatility 1 Month 3 Month 6 Month 12 Month
Max Drawdown 1 Month 3 Month 6 Month 12 Month
# Backtest ID: 54553ae8ddcd7d090a33b736
There was a runtime error.

Yes. I used GOOG as just an example. I can work around that one security, but it seems like there should be a more robust solution so that when I cast a wider net they each get taken care of without having to track them down individually.

If you just want to drop any stocks with NaN values from the history dataframe you can pass 'axis=1' to DataFrame.dropna. The default behavior is to drop any rows with NaN values, passing axis=1 will drop columns with NaNs.

Clone Algorithm
15
Loading...
Backtest from to with initial capital
Total Returns
--
Alpha
--
Beta
--
Sharpe
--
Sortino
--
Max Drawdown
--
Benchmark Returns
--
Volatility
--
Returns 1 Month 3 Month 6 Month 12 Month
Alpha 1 Month 3 Month 6 Month 12 Month
Beta 1 Month 3 Month 6 Month 12 Month
Sharpe 1 Month 3 Month 6 Month 12 Month
Sortino 1 Month 3 Month 6 Month 12 Month
Volatility 1 Month 3 Month 6 Month 12 Month
Max Drawdown 1 Month 3 Month 6 Month 12 Month
# Backtest ID: 5455872655115808ff73e032
There was a runtime error.

That certainly does the trick. Thank you Mr. Edwards. I knew there had to be a way.