Either I can't do math or Quantopian can't

So I have this toy algorithm that is a stripped down version of something I am working on and I have noticed a strange behavior. In the month of July 2014 there it calculates a change in unemployment rate of -0.2 and then does a two sets of comparisons, both simple if unr_diff < -0.2 and in one case it results in false and in the other it says true. What is going on?

This is the relevant part of the code:

log.info("unr_diff: %.2f" % unr_diff)
log.info("unr_diff < -0.2: %s" % unr_diff < -0.2)
if unr_diff < -0.2:
outlook += 0.5
log.info("(unr_diff < -0.2) outlook + .5")


This is the relevant part of the log:

2014-07-07trade_unr:52 INFO unr_diff: -0.20
2014-07-07trade_unr:57 INFO (unr_diff < -0.2) outlook + .5

0
Total Returns
--
Alpha
--
Beta
--
Sharpe
--
Sortino
--
Max Drawdown
--
Benchmark Returns
--
Volatility
--
 Returns 1 Month 3 Month 6 Month 12 Month
 Alpha 1 Month 3 Month 6 Month 12 Month
 Beta 1 Month 3 Month 6 Month 12 Month
 Sharpe 1 Month 3 Month 6 Month 12 Month
 Sortino 1 Month 3 Month 6 Month 12 Month
 Volatility 1 Month 3 Month 6 Month 12 Month
 Max Drawdown 1 Month 3 Month 6 Month 12 Month
def rename_col(df):
df = df.rename(columns={'Value':'value'})
df = df.rename(columns={'VALUE':'value'})
return df

def initialize(context):
context.outlook = 0
# Reported December 1st 2013
context.unr_prev_value = 7.0

# Reported January 1st 2014
context.current_unr = 6.7

date_rule=date_rules.month_start(days_offset=3),
)

fetch_csv('https://www.quandl.com/api/v1/datasets/FRED/UNRATE.csv',
symbol='UNR',
date_column='DATE',
date_format='%m/%d/%y',
post_func=rename_col,
)

def handle_data(context, data):
if context.outlook > 0:
order(sid(8554), 1)

if 'value' in data['UNR']:
#if 'index' in data['PMI']:
log.info("-- new external data --")

future_unr = data['UNR']['value']

current_unr = context.current_unr

log.info("previous UNR: %s" % context.unr_prev_value)
log.info("-")
log.info("current  UNR: %s" % current_unr)
log.info("-")
log.info("future   UNR: %s" % future_unr)
log.info("-")

unr_diff = (current_unr - context.unr_prev_value)# * 10

log.info("unr delta: %.2f" % unr_diff)
log.info("-")

outlook = 0
log.info("unr_diff: %.2f" % unr_diff)
log.info("unr_diff < -0.2: %s" % unr_diff < -0.2)

if unr_diff < -0.2:
outlook += 0.5
log.info("(unr_diff < -0.2) outlook + .5")

context.outlook = outlook
log.info("Outlook: %s" % outlook)

context.unr_prev_value = current_unr
context.current_unr = future_unr

There was a runtime error.
8 responses

Hi Andrew, try putting "unr_diff < -0.2" within parentheses in your log statement.

log.info("unr_diff < -0.2: %s" % (unr_diff < -0.2))

You're right I was logging that incorrectly. It still does not explain the strange behavior though. After the correction the log appears as below:

2014-07-07 trade_unr:52 INFO unr_diff: -0.20
2014-07-07 trade_unr:53 INFO unr_diff < -0.2: True
2014-07-07 trade_unr:57 INFO (unr_diff < -0.2) outlook + .5


When you are logging the unr_diff variable, you are providing a %.2f format. The real float value probably has something after the first 2 decimals.

For example:

Real value = -0.200008
Logged as "-0.20"
unr_diff < -0.2 == TRUE

I hope I understood your problem correctly! ;) And sorry if not.

The values it is subtracting only go out to a tenth, but I printed out to the hundredth in the logs just to make sure.

To leave no stone un-turned I went ahead and tried printing out to the 20th decimal place and you were right about the float value being different. My question then is why when subtracting two numbers such as 6.1 and 6.3 do I get -0.20000000000000017764? That seems like it could bite some people when they are trying to make decisions on calculations and there are mysterious remainders appearing on the end.

2014-07-07 trade_unr:39 INFO previous UNR: 6.3
2014-07-07 trade_unr:41 INFO current  UNR: 6.1
2014-07-07 trade_unr:43 INFO future   UNR: 6.2
2014-07-07 trade_unr:48 INFO unr delta: -0.20000000000000017764
2014-07-07 trade_unr:53 INFO unr_diff < -0.2: True
2014-07-07 trade_unr:57 INFO (unr_diff < -0.2) outlook + .5


Alright I figured it out and am going to post an answer to my question in case someone else ends up here. Basically it boils down to the fact that the result of 6.1 - 6.3 is a double precision float which is stored as binary. Due to the constraints of storing a number as binary some numbers can't be stored in their exact values. For more information read this: http://floating-point-gui.de/basic/

So the way to deal with this in the case that you know you only want to deal with a number out to the 10th or 100th decimal place you have to round. See below:

$ipython Python 2.7.9 (default, Feb 10 2015, 03:29:19) Type "copyright", "credits" or "license" for more information. IPython 2.3.0 -- An enhanced Interactive Python. ? -> Introduction and overview of IPython's features. %quickref -> Quick reference. help -> Python's own help system. object? -> Details about 'object', use 'object??' for extra details. In [1]: a = 6.1 In [2]: b = 6.3 In [3]: c = a - b In [4]: c Out[4]: -0.20000000000000018 In [5]: d = round(c,2) In [6]: d Out[6]: -0.2 In [7]: c < -0.2 Out[7]: True In [8]: d < -0.2 Out[8]: False  I am not sure rounding is robust either though. You could do something like: def lt(a,b): return ((a<b) & ~(numpy.isclose(a,b))) > 0  Yeah rounding worked in this case because I know the numbers being processed will not be more precise than a 1/10th, but you're right it is not robust. I talked to a Python guru friend and he recommended just doing fixed point arithmetic and using the decimal python library. https://docs.python.org/2/library/decimal.html $ ipython
Python 2.7.9 (default, Feb 10 2015, 03:29:19)

IPython 2.3.0 -- An enhanced Interactive Python.
?         -> Introduction and overview of IPython's features.
%quickref -> Quick reference.
help      -> Python's own help system.
object?   -> Details about 'object', use 'object??' for extra details.

In [1]: from decimal import *

In [2]: getcontext().prec = 4

In [3]: a = Decimal(6.1)

In [4]: a
Out[4]: Decimal('6.0999999999999996447286321199499070644378662109375')

In [5]: b = Decimal(6.3)

In [6]: a - b
Out[6]: Decimal('-0.2000')

In [7]: c = a - b

In [8]: c < -.2
Out[8]: False


Yup fixed point works well. Most of the data feeds actually send prices and whatnot as fixed decimal.