Back to Community
Mispricing Data

Hello!

I ran into this issue and would really like to understand where the problem lies.

It occurs to me that the data "recorded" at run-time is correct, while the same data, retrieved "historically", is way mispriced.

Am I missing something?

Thanks for any clarification.

P.S. The info is in the log of the Backtest

2018-02-20 14:45 before_trading_start:26 INFO 2018-02-16 00:00:00+00:00 75.06
Freq: C, Name: Equity(21652 [IYR]), dtype: float64 2018-02-21 14:45 before_trading_start:26 INFO 2018-02-20 00:00:00+00:00 74.13
Freq: C, Name: Equity(21652 [IYR]), dtype: float64 2018-02-22 14:45 before_trading_start:26 INFO 2018-02-21 00:00:00+00:00 72.9
Freq: C, Name: Equity(21652 [IYR]), dtype: float64 2018-02-23 14:45 before_trading_start:26 INFO 2018-02-22 00:00:00+00:00 73.57
Freq: C, Name: Equity(21652 [IYR]), dtype: float64 2018-02-26 14:45 before_trading_start:26 INFO 2018-02-23 00:00:00+00:00 74.83
Freq: C, Name: Equity(21652 [IYR]), dtype: float64 2019-01-02 14:45 before_trading_start:17 INFO 2018-02-16 00:00:00+00:00 72.579
2018-02-20 00:00:00+00:00 71.680
2018-02-21 00:00:00+00:00 70.491
2018-02-22 00:00:00+00:00 71.138
2018-02-23 00:00:00+00:00 72.357
Freq: C, Name: Equity(21652 [IYR]), dtype: float64 End of logs.

Here's a better comparison:

Run-time:
2018-02-16 00:00:00+00:00 75.06
2018-02-20 00:00:00+00:00 74.13
2018-02-21 00:00:00+00:00 72.9
2018-02-22 00:00:00+00:00 73.57
2018-02-23 00:00:00+00:00 74.83

Historical:
2018-02-16 00:00:00+00:00 72.579
2018-02-20 00:00:00+00:00 71.680
2018-02-21 00:00:00+00:00 70.491
2018-02-22 00:00:00+00:00 71.138
2018-02-23 00:00:00+00:00 72.357

Clone Algorithm
1
Loading...
Backtest from to with initial capital
Total Returns
--
Alpha
--
Beta
--
Sharpe
--
Sortino
--
Max Drawdown
--
Benchmark Returns
--
Volatility
--
Returns 1 Month 3 Month 6 Month 12 Month
Alpha 1 Month 3 Month 6 Month 12 Month
Beta 1 Month 3 Month 6 Month 12 Month
Sharpe 1 Month 3 Month 6 Month 12 Month
Sortino 1 Month 3 Month 6 Month 12 Month
Volatility 1 Month 3 Month 6 Month 12 Month
Max Drawdown 1 Month 3 Month 6 Month 12 Month
import quantopian.algorithm as algo
import datetime

def initialize(context):
    
    context.iyr = symbol('IYR')
    context.stop = 5
    context.hist = 1
    
    
def before_trading_start(context, data):
    dt = get_datetime()
    
    if dt.year == 2019:
        if context.hist == 1:
            h = data.history(context.iyr, 'close', 250, '1d')
            log.info(h.iloc[31:36])
            context.hist = 0
            
     
    
    if context.stop == 0:
        return
    
    price = data.history(context.iyr, 'close', 1, '1d')
    log.info(price)
    context.stop = context.stop - 1
There was a runtime error.
9 responses

Try this way:

def initialize(context):  
    context.sec = symbol('IYR')  
    context.stop = 5  

def before_trading_start(context, data):  
    price = data.current(context.sec, 'close')  
    log.info(price)  
    context.stop = context.stop - 1 

    if context.stop == 0:  
        prices = data.history(context.sec, 'close', 250, '1d')  
        log.info(prices.iloc[-5:])  
'''
START  
02/11/2019  
END  
02/18/2019  
'''

2019-02-11 05:45 before_trading_start:7 INFO 84.14
2019-02-12 05:45 before_trading_start:7 INFO 84.35
2019-02-13 05:45 before_trading_start:7 INFO 83.84
2019-02-14 05:45 before_trading_start:7 INFO 84.25
2019-02-15 05:45 before_trading_start:7 INFO 84.52

2019-02-15 05:45 before_trading_start:12 INFO
2019-02-08 00:00:00+00:00 84.14
2019-02-11 00:00:00+00:00 84.35
2019-02-12 00:00:00+00:00 83.84
2019-02-13 00:00:00+00:00 84.25
2019-02-14 00:00:00+00:00 84.52

Yes it works, but why is the price different when you retrieve data after a long time, and not after few days?

I have tried it on different dates:

START
02/11/2017
END
02/18/2017

2017-02-13 05:45 before_trading_start:7 INFO 78.51
2017-02-14 05:45 before_trading_start:7 INFO 78.77
2017-02-15 05:45 before_trading_start:7 INFO 78.38
2017-02-16 05:45 before_trading_start:7 INFO 78.229
2017-02-17 05:45 before_trading_start:7 INFO 78.635

2017-02-17 05:45 before_trading_start:12 INFO
2017-02-10 00:00:00+00:00 78.510
2017-02-13 00:00:00+00:00 78.770
2017-02-14 00:00:00+00:00 78.380
2017-02-15 00:00:00+00:00 78.229
2017-02-16 00:00:00+00:00 78.635

START
02/11/2015
END
02/18/2015

2015-02-11 05:45 before_trading_start:7 INFO 80.06
2015-02-12 05:45 before_trading_start:7 INFO 79.9
2015-02-13 05:45 before_trading_start:7 INFO 80.76
2015-02-17 05:45 before_trading_start:7 INFO 80.32
2015-02-18 05:45 before_trading_start:7 INFO 80.1

2015-02-18 05:45 before_trading_start:12 INFO
2015-02-10 00:00:00+00:00 80.06
2015-02-11 00:00:00+00:00 79.90
2015-02-12 00:00:00+00:00 80.76
2015-02-13 00:00:00+00:00 80.32
2015-02-17 00:00:00+00:00 80.10

Maybe I did not explain myself correctly.

Yes, your algo shows exact data, but that is because you are retrieving data right after you retireve your data run-time.
Try this and see if you understand what I mean.. This one retrieves data after 1 year.. so the algo requests the data after one year.
I have trouble writing it out.

See the version below, please.

def initialize(context):
context.sec = symbol('IYR')
context.stop1 = 5
context.stop2 = 200

def before_trading_start(context, data):
if context.stop1 != 0:
price = data.current(context.sec, 'close')
log.info(price)
context.stop1 = context.stop1 - 1

context.stop2 = context.stop2 - 1  
if context.stop2 == 0:  
    prices = data.history(context.sec, 'close', 250, '1d')  
    log.info(prices.iloc[51:55])  

This is as simple as it gets.
You can check online wherever you want (investing.com, bloomberg ...) that the data returned is NOT correct.

Thoughts?

def initialize(context):

context.iyr = symbol('IYR')  

def before_trading_start(context, data):
h = data.history(context.iyr, 'close', 250, '1d')
log.info(h.head(20))

Longer period:

def initialize(context):  
    context.sec = symbol('IYR')  
    context.stop = 20  

def before_trading_start(context, data):  
    price = data.current(context.sec, 'close')  
    log.info(price)  
    context.stop = context.stop - 1 

    if context.stop == 0:  
        prices  = data.history(context.sec, 'close', 250, '1d')  
        log.info(prices.iloc[-20:])  


START
01/21/2015
END
02/18/2015

2015-01-21 05:45 before_trading_start:7 INFO 81.02
2015-01-22 05:45 before_trading_start:7 INFO 80.95
2015-01-23 05:45 before_trading_start:7 INFO 82.49
2015-01-26 05:45 before_trading_start:7 INFO 82.37
2015-01-27 05:45 before_trading_start:7 INFO 83.13
2015-01-28 05:45 before_trading_start:7 INFO 83.01
2015-01-29 05:45 before_trading_start:7 INFO 82.43
2015-01-30 05:45 before_trading_start:7 INFO 82.58
2015-02-02 05:45 before_trading_start:7 INFO 81.18
2015-02-03 05:45 before_trading_start:7 INFO 81.0
2015-02-04 05:45 before_trading_start:7 INFO 81.77
2015-02-05 05:45 before_trading_start:7 INFO 81.42
2015-02-06 05:45 before_trading_start:7 INFO 82.35
2015-02-09 05:45 before_trading_start:7 INFO 80.18
2015-02-10 05:45 before_trading_start:7 INFO 79.79
2015-02-11 05:45 before_trading_start:7 INFO 80.06
2015-02-12 05:45 before_trading_start:7 INFO 79.9
2015-02-13 05:45 before_trading_start:7 INFO 80.76
2015-02-17 05:45 before_trading_start:7 INFO 80.32
2015-02-18 05:45 before_trading_start:7 INFO 80.1

2015-02-18 05:45 before_trading_start:12 INFO
2015-01-20 00:00:00+00:00 81.02
2015-01-21 00:00:00+00:00 80.95
2015-01-22 00:00:00+00:00 82.49
2015-01-23 00:00:00+00:00 82.37
2015-01-26 00:00:00+00:00 83.13
2015-01-27 00:00:00+00:00 83.01
2015-01-28 00:00:00+00:00 82.43
2015-01-29 00:00:00+00:00 82.58
2015-01-30 00:00:00+00:00 81.18
2015-02-02 00:00:00+00:00 81.00
2015-02-03 00:00:00+00:00 81.77
2015-02-04 00:00:00+00:00 81.42
2015-02-05 00:00:00+00:00 82.35
2015-02-06 00:00:00+00:00 80.18
2015-02-09 00:00:00+00:00 79.79
2015-02-10 00:00:00+00:00 80.06
2015-02-11 00:00:00+00:00 79.90
2015-02-12 00:00:00+00:00 80.76
2015-02-13 00:00:00+00:00 80.32
2015-02-17 00:00:00+00:00 80.10

But again, you are retrieving the data after 20 days even if the backtest lasts 4 years.

Try retrieving the data after one year and the data will be wrong.

Now I see what you called "mispricing".

def initialize(context):  
    context.sec = symbol('IYR')  
    context.stop = 20  

def before_trading_start(context, data):  
    price = data.current(context.sec, 'close')  
    log.info(price)  
    context.stop = context.stop - 1 

    if context.stop == 0:  
        prices  = data.history(context.sec, 'close', 250, '1d')  
        log.info(prices.iloc[-20:])  

START
12/03/2015
END
01/01/2016

2015-12-03 05:45 before_trading_start:7 INFO 74.72
2015-12-04 05:45 before_trading_start:7 INFO 73.63
2015-12-07 05:45 before_trading_start:7 INFO 74.84
2015-12-08 05:45 before_trading_start:7 INFO 74.64
2015-12-09 05:45 before_trading_start:7 INFO 74.47
2015-12-10 05:45 before_trading_start:7 INFO 74.03
2015-12-11 05:45 before_trading_start:7 INFO 73.56
2015-12-14 05:45 before_trading_start:7 INFO 73.25
2015-12-15 05:45 before_trading_start:7 INFO 73.43
2015-12-16 05:45 before_trading_start:7 INFO 74.24
2015-12-17 05:45 before_trading_start:7 INFO 75.69
2015-12-18 05:45 before_trading_start:7 INFO 75.32
2015-12-21 05:45 before_trading_start:7 INFO 74.37
2015-12-22 05:45 before_trading_start:7 INFO 74.76
2015-12-23 05:45 before_trading_start:7 INFO 75.12
2015-12-24 05:45 before_trading_start:7 INFO 74.969872
2015-12-28 05:45 before_trading_start:7 INFO 74.97
2015-12-29 05:45 before_trading_start:7 INFO 75.27
2015-12-30 05:45 before_trading_start:7 INFO 75.995
2015-12-31 05:45 before_trading_start:7 INFO 75.62

2015-12-31 05:45 before_trading_start:12 INFO
2015-12-02 00:00:00+00:00 73.775
2015-12-03 00:00:00+00:00 72.699
2015-12-04 00:00:00+00:00 73.894
2015-12-07 00:00:00+00:00 73.696
2015-12-08 00:00:00+00:00 73.528
2015-12-09 00:00:00+00:00 73.094
2015-12-10 00:00:00+00:00 72.630
2015-12-11 00:00:00+00:00 72.324
2015-12-14 00:00:00+00:00 72.501
2015-12-15 00:00:00+00:00 73.301
2015-12-16 00:00:00+00:00 74.733
2015-12-17 00:00:00+00:00 74.368
2015-12-18 00:00:00+00:00 73.430
2015-12-21 00:00:00+00:00 73.815
2015-12-22 00:00:00+00:00 74.170
2015-12-23 00:00:00+00:00 74.970
2015-12-24 00:00:00+00:00 74.970
2015-12-28 00:00:00+00:00 75.270
2015-12-29 00:00:00+00:00 75.995
2015-12-30 00:00:00+00:00 75.620

Are you aware of Quantopian split and dividend price adjustment?
The data returned for equities will be adjusted for splits, mergers, and dividends as of the current simulation time.
This may help.

Thanks a lot!!!!