Back to Community
Either I can't do math or Quantopian can't

So I have this toy algorithm that is a stripped down version of something I am working on and I have noticed a strange behavior. In the month of July 2014 there it calculates a change in unemployment rate of -0.2 and then does a two sets of comparisons, both simple if unr_diff < -0.2 and in one case it results in false and in the other it says true. What is going on?

This is the relevant part of the code:

log.info("unr_diff: %.2f" % unr_diff)  
log.info("unr_diff < -0.2: %s" % unr_diff < -0.2)  
if unr_diff < -0.2:  
    outlook += 0.5  
    log.info("(unr_diff < -0.2) outlook + .5")  

This is the relevant part of the log:

2014-07-07trade_unr:52 INFO unr_diff: -0.20  
2014-07-07trade_unr:53 INFO False  
2014-07-07trade_unr:57 INFO (unr_diff < -0.2) outlook + .5  
2014-07-07trade_unr:60 INFO Outlook: 0.5  
Clone Algorithm
0
Loading...
Total Returns
--
Alpha
--
Beta
--
Sharpe
--
Sortino
--
Max Drawdown
--
Benchmark Returns
--
Volatility
--
Returns 1 Month 3 Month 6 Month 12 Month
Alpha 1 Month 3 Month 6 Month 12 Month
Beta 1 Month 3 Month 6 Month 12 Month
Sharpe 1 Month 3 Month 6 Month 12 Month
Sortino 1 Month 3 Month 6 Month 12 Month
Volatility 1 Month 3 Month 6 Month 12 Month
Max Drawdown 1 Month 3 Month 6 Month 12 Month
def rename_col(df):
    df = df.rename(columns={'Value':'value'})
    df = df.rename(columns={'VALUE':'value'})
    return df

def initialize(context):
    context.outlook = 0
    # Reported December 1st 2013
    context.unr_prev_value = 7.0       
    
    # Reported January 1st 2014
    context.current_unr = 6.7           

    schedule_function(trade_unr,
                      date_rule=date_rules.month_start(days_offset=3),
                     )
    
    fetch_csv('https://www.quandl.com/api/v1/datasets/FRED/UNRATE.csv',
        symbol='UNR',
        date_column='DATE',
        date_format='%m/%d/%y',
        post_func=rename_col,
    )

    
def handle_data(context, data):
    if context.outlook > 0:
        order(sid(8554), 1)
    
def trade_unr(context, data):
    if 'value' in data['UNR']:  
    #if 'index' in data['PMI']:     
        log.info("-- new external data --")
            
        future_unr = data['UNR']['value']
        
        current_unr = context.current_unr
        
        log.info("previous UNR: %s" % context.unr_prev_value)
        log.info("-")
        log.info("current  UNR: %s" % current_unr)
        log.info("-")
        log.info("future   UNR: %s" % future_unr)
        log.info("-")

        unr_diff = (current_unr - context.unr_prev_value)# * 10

        log.info("unr delta: %.2f" % unr_diff)
        log.info("-")

        outlook = 0
        log.info("unr_diff: %.2f" % unr_diff)
        log.info("unr_diff < -0.2: %s" % unr_diff < -0.2)
        
        if unr_diff < -0.2: 
            outlook += 0.5
            log.info("(unr_diff < -0.2) outlook + .5")
        
        context.outlook = outlook
        log.info("Outlook: %s" % outlook)
        
        context.unr_prev_value = current_unr
        context.current_unr = future_unr
There was a runtime error.
8 responses

Hi Andrew, try putting "unr_diff < -0.2" within parentheses in your log statement.

log.info("unr_diff < -0.2: %s" % (unr_diff < -0.2))

You're right I was logging that incorrectly. It still does not explain the strange behavior though. After the correction the log appears as below:

2014-07-07 trade_unr:52 INFO unr_diff: -0.20  
2014-07-07 trade_unr:53 INFO unr_diff < -0.2: True  
2014-07-07 trade_unr:57 INFO (unr_diff < -0.2) outlook + .5  
2014-07-07 trade_unr:60 INFO Outlook: 0.5  

When you are logging the unr_diff variable, you are providing a %.2f format. The real float value probably has something after the first 2 decimals.

For example:

Real value = -0.200008
Logged as "-0.20"
unr_diff < -0.2 == TRUE

I hope I understood your problem correctly! ;) And sorry if not.

The values it is subtracting only go out to a tenth, but I printed out to the hundredth in the logs just to make sure.

To leave no stone un-turned I went ahead and tried printing out to the 20th decimal place and you were right about the float value being different. My question then is why when subtracting two numbers such as 6.1 and 6.3 do I get -0.20000000000000017764? That seems like it could bite some people when they are trying to make decisions on calculations and there are mysterious remainders appearing on the end.

2014-07-07 trade_unr:39 INFO previous UNR: 6.3  
2014-07-07 trade_unr:40 INFO -  
2014-07-07 trade_unr:41 INFO current  UNR: 6.1  
2014-07-07 trade_unr:42 INFO -  
2014-07-07 trade_unr:43 INFO future   UNR: 6.2  
2014-07-07 trade_unr:44 INFO -  
2014-07-07 trade_unr:48 INFO unr delta: -0.20000000000000017764  
2014-07-07 trade_unr:49 INFO -  
2014-07-07 trade_unr:52 INFO unr_diff: -0.20000000000000017764  
2014-07-07 trade_unr:53 INFO unr_diff < -0.2: True  
2014-07-07 trade_unr:57 INFO (unr_diff < -0.2) outlook + .5  
2014-07-07 trade_unr:60 INFO Outlook: 0.5  

Alright I figured it out and am going to post an answer to my question in case someone else ends up here. Basically it boils down to the fact that the result of 6.1 - 6.3 is a double precision float which is stored as binary. Due to the constraints of storing a number as binary some numbers can't be stored in their exact values. For more information read this: http://floating-point-gui.de/basic/

So the way to deal with this in the case that you know you only want to deal with a number out to the 10th or 100th decimal place you have to round. See below:

$ ipython
Python 2.7.9 (default, Feb 10 2015, 03:29:19)  
Type "copyright", "credits" or "license" for more information.

IPython 2.3.0 -- An enhanced Interactive Python.  
?         -> Introduction and overview of IPython's features.
%quickref -> Quick reference.
help      -> Python's own help system.  
object?   -> Details about 'object', use 'object??' for extra details.

In [1]: a = 6.1

In [2]: b = 6.3

In [3]: c = a - b

In [4]: c  
Out[4]: -0.20000000000000018

In [5]: d = round(c,2)

In [6]: d  
Out[6]: -0.2

In [7]: c < -0.2  
Out[7]: True

In [8]: d < -0.2  
Out[8]: False  

I am not sure rounding is robust either though. You could do something like:

def lt(a,b):  
    return ((a<b) & ~(numpy.isclose(a,b))) > 0  

Yeah rounding worked in this case because I know the numbers being processed will not be more precise than a 1/10th, but you're right it is not robust.

I talked to a Python guru friend and he recommended just doing fixed point arithmetic and using the decimal python library. https://docs.python.org/2/library/decimal.html

$ ipython
Python 2.7.9 (default, Feb 10 2015, 03:29:19)  
Type "copyright", "credits" or "license" for more information.

IPython 2.3.0 -- An enhanced Interactive Python.  
?         -> Introduction and overview of IPython's features.
%quickref -> Quick reference.
help      -> Python's own help system.  
object?   -> Details about 'object', use 'object??' for extra details.

In [1]: from decimal import *

In [2]: getcontext().prec = 4

In [3]: a = Decimal(6.1)

In [4]: a  
Out[4]: Decimal('6.0999999999999996447286321199499070644378662109375')

In [5]: b = Decimal(6.3)

In [6]: a - b  
Out[6]: Decimal('-0.2000')

In [7]: c = a - b

In [8]: c < -.2  
Out[8]: False  

Yup fixed point works well. Most of the data feeds actually send prices and whatnot as fixed decimal.