Back to Community
ValueError: Cannot convert NA to integer

Hi Community,

I've been really struggling to finish this backtest but apparently it seems there's an error on my code. I have checked Nans (even though i have explicitly rejected them on my make_pipeline function) and i have also check the data type introduced into my portfolio weights (context.weights variable) but i can't find out the solution. I would really appreciate you feedback.

Thanks a lot!!

Axel

Clone Algorithm
4
Loading...
Backtest from to with initial capital
Total Returns
--
Alpha
--
Beta
--
Sharpe
--
Sortino
--
Max Drawdown
--
Benchmark Returns
--
Volatility
--
Returns 1 Month 3 Month 6 Month 12 Month
Alpha 1 Month 3 Month 6 Month 12 Month
Beta 1 Month 3 Month 6 Month 12 Month
Sharpe 1 Month 3 Month 6 Month 12 Month
Sortino 1 Month 3 Month 6 Month 12 Month
Volatility 1 Month 3 Month 6 Month 12 Month
Max Drawdown 1 Month 3 Month 6 Month 12 Month
# Backtest ID: 5a9f2c8d1db0ad406d69a01f
There was a runtime error.
9 responses

That's a bug where the values to opt are all 0.0. Python complains about NA and integer and Q would like to know I think, catch that and make it user-friendly.

The easiest solution is to use try ... except, allows the backtest to continue and provides the opportunity to log info and do something, anything else.
The positive results here are simply from the minus sign flipping long short.

Clone Algorithm
4
Loading...
Backtest from to with initial capital
Total Returns
--
Alpha
--
Beta
--
Sharpe
--
Sortino
--
Max Drawdown
--
Benchmark Returns
--
Volatility
--
Returns 1 Month 3 Month 6 Month 12 Month
Alpha 1 Month 3 Month 6 Month 12 Month
Beta 1 Month 3 Month 6 Month 12 Month
Sharpe 1 Month 3 Month 6 Month 12 Month
Sortino 1 Month 3 Month 6 Month 12 Month
Volatility 1 Month 3 Month 6 Month 12 Month
Max Drawdown 1 Month 3 Month 6 Month 12 Month
# Backtest ID: 5a9f727672fd4346a77f1c7a
There was a runtime error.

Hi Blue,

First of all, thanks for your response. Despite not crashing, it seems that my closing orders are not being proceeded, so i understand that on that day no closing orders will be placed. I don't quite get the nature of the problem, i mean, what's happening on that specifid date to crash? I have used the same logic for a bigger timespan and it only crashes on that date apparently. Another solution that has come to me is to append my closing _positions_list to my trading_positions list, in order to don't have a unique list with values of 0.0, but it seems that has got the same problem. I am desperate to understand the nature of the problem. However, thanks for your contribution, hope that both can find a solution.

Axel

Thanks for reporting that, it's a pandas bug: https://github.com/pandas-dev/pandas/issues/14687

Try emailing [email protected] and with good fortune it might make its way to a developer. There could be nothing they can do or they might be able to capture the error and at least avoid the crash. Do want them to know in any case.

Expected any of the keys (securities) that are current positions would be closed since the values are all 0.0 when that happens.

Thanks @Blue, i've posted the ticket so let's cross fingers!

Any response?

Hi @Axel, @Blue,
For me, as someone who struggles constantly with python in general and with pandas in particular, i am in awe of people such as yourselves who can (not only) use pandas effectively and/ but can also go so far as to find bugs in it!!!! I'm sure i will thank you even more profusely when my python skills improve! Cheers, all the best, TonyM.

I'm getting this error message as well. Would upgrading pandas in the IDE fix it? try-except workaround works, but not ideal. Looks like the pandas version in the IDE is 0.18.1, which is about 3 years old?

@Q Support team,

Could you advice please?

@Blue Seahawk,

I'm using the below in 'rebalance' to remove any NaNs or Null values in my alpha-factor pandas series, but I still get this error message. Do you know why, or if there's a better way to do this? Why is it trying to convert to an integer anyway, when all other values are floats?? You don't reckon it's a bug in the Optimize API, rather than the pandas bug?

pipeline_data = pipeline_data[pd.notnull(pipeline_data)]  

Looks like that was from values not 0.0 yet very close like 4.351e-7.
Joakim thoughtfully used Quantopian's collaboration feature to share affected code with me.
Is np.isclose() involved somehow? I was seeing np errors showing up in debug where the object was just a pd Series.

Thanks for your help on this @Blue, really appreciate it!