Quantopian Miscalculation?

I don't quite understand the way Quantopian simulate the trade. Hope someone can help me.

Please look at the following algorithm and backtest result. The algorithm is very simple: buy XIV with all cash available. The backtest ran from 20140716 to 20140718. 20140717 was the day MH17 was shotdown in Ukraine, as a result price dropped significantly.

From the log, I can see the prices on each day are:
20140716 46.80
20140717 42.33
20140718 45.4716

On 16th, the algorithm try to buy 1000000/46.8 -> 23167 shares. If you click 'Transaction Details' page. The transaction on 16th is BUY 21367 shares of XIV @ 42.36, which seems to be the price of 17th, after MH17 was shot down.

Why? The price algo used to calculate order and the price system used to fulfill the order has 1 days gap. Just like we calculate the orders today at market close and place the order tomorrow at market close.

Why?

Ideally, the backtester should use the same price of 46.8 to fulfill the order immediately. I understand the price will move after the moment of calculation. But that should be accounted by slippage.

3
Backtest from to with initial capital
Total Returns
--
Alpha
--
Beta
--
Sharpe
--
Sortino
--
Max Drawdown
--
Benchmark Returns
--
Volatility
--
 Returns 1 Month 3 Month 6 Month 12 Month
 Alpha 1 Month 3 Month 6 Month 12 Month
 Beta 1 Month 3 Month 6 Month 12 Month
 Sharpe 1 Month 3 Month 6 Month 12 Month
 Sortino 1 Month 3 Month 6 Month 12 Month
 Volatility 1 Month 3 Month 6 Month 12 Month
 Max Drawdown 1 Month 3 Month 6 Month 12 Month
# For this example, we're going to write a simple momentum script.
# When the stock goes up quickly, we're going to buy;
# when it goes down we're going to sell.
# Hopefully we'll ride the waves.

# To run an algorithm in Quantopian, you need two functions:
# initialize and handle_data.
def initialize(context):
# The initialize function sets any data or variables that
# you'll use in your algorithm.
# For instance, you'll want to define the security
# (or securities) you want to backtest.
# You'll also want to define any parameters or values
# you're going to use later.
# It's only called once at the beginning of your algorithm.

# In our example, we're looking at Apple.
# If you re-type this line you'll see
# the auto-complete that is available for security.
context.security = symbol('XIV')

# The handle_data function is where the real work is done.
# This function is run either every minute
# (in live trading and minute backtesting mode)
# or every day (in daily backtesting mode).
def handle_data(context, data):
# We've built a handful of useful data transforms for you to use,
# such as moving average.

# To make market decisions, we're calculating the stock's
# moving average for the last 5 days and its current price.
current_price = data[context.security].price
log.info("XIV current price: %f" % current_price)

# Another powerful built-in feature of the Quantopian backtester is the
# portfolio object.  The portfolio object tracks your positions, cash,
# cost basis of specific holdings, and more.  In this line, we calculate
# the current amount of cash in our portfolio.
cash = context.portfolio.cash

# Here is the meat of our algorithm.
# If the current price is 1% above the 5-day average price
# AND we have enough cash, then we will order.
# If the current price is below the average price,
# then we want to close our position to 0 shares.
if cash > current_price:

# Need to calculate how many shares we can buy
number_of_shares = int(cash/current_price)

order(context.security, +number_of_shares)

'''
elif current_price < average_price:

# Sell all of our shares by setting the target position to zero
order_target(context.security, 0)
log.info("Selling %s" % (context.security.symbol))
'''

# You can use the record() method to track any custom signal.
# The record graph tracks up to five different variables.
# Here we record the Apple stock price.
record(stock_price=data[context.security].price)
There was a runtime error.
20 responses

Hi Alan,

How orders work in daily and minutely mode is that, at the fastest, orders are executed at the next bar. So when you submit an order in daily mode on October 1st, the fastest it'll get executed is October 2nd at the closing. The same applies in minutely mode: 9:31 AM submit 9:32 execution (at the fastest).

However you can create your own custom slippage models to fit your needs. I believe Grant has created one on this thread (https://www.quantopian.com/posts/backtest-results-different-in-minute-and-daily-mode) to run on daily bars and execute on the same day.

Disclaimer

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.

Alan, does this version look more like what you were expecting? All I did was switch to minute mode and add a check for open orders.

0
Backtest from to with initial capital
Total Returns
--
Alpha
--
Beta
--
Sharpe
--
Sortino
--
Max Drawdown
--
Benchmark Returns
--
Volatility
--
 Returns 1 Month 3 Month 6 Month 12 Month
 Alpha 1 Month 3 Month 6 Month 12 Month
 Beta 1 Month 3 Month 6 Month 12 Month
 Sharpe 1 Month 3 Month 6 Month 12 Month
 Sortino 1 Month 3 Month 6 Month 12 Month
 Volatility 1 Month 3 Month 6 Month 12 Month
 Max Drawdown 1 Month 3 Month 6 Month 12 Month
# For this example, we're going to write a simple momentum script.
# When the stock goes up quickly, we're going to buy;
# when it goes down we're going to sell.
# Hopefully we'll ride the waves.

# To run an algorithm in Quantopian, you need two functions:
# initialize and handle_data.
def initialize(context):
# The initialize function sets any data or variables that
# you'll use in your algorithm.
# For instance, you'll want to define the security
# (or securities) you want to backtest.
# You'll also want to define any parameters or values
# you're going to use later.
# It's only called once at the beginning of your algorithm.

# In our example, we're looking at Apple.
# If you re-type this line you'll see
# the auto-complete that is available for security.
context.security = symbol('XIV')

# The handle_data function is where the real work is done.
# This function is run either every minute
# (in live trading and minute backtesting mode)
# or every day (in daily backtesting mode).
def handle_data(context, data):
# We've built a handful of useful data transforms for you to use,
# such as moving average.
if get_open_orders():
return
# To make market decisions, we're calculating the stock's
# moving average for the last 5 days and its current price.
current_price = data[context.security].price
log.info("XIV current price: %f" % current_price)

# Another powerful built-in feature of the Quantopian backtester is the
# portfolio object.  The portfolio object tracks your positions, cash,
# cost basis of specific holdings, and more.  In this line, we calculate
# the current amount of cash in our portfolio.
cash = context.portfolio.cash

# Here is the meat of our algorithm.
# If the current price is 1% above the 5-day average price
# AND we have enough cash, then we will order.
# If the current price is below the average price,
# then we want to close our position to 0 shares.
if cash > current_price:

# Need to calculate how many shares we can buy
number_of_shares = int(cash/current_price)

order(context.security, +number_of_shares)

'''
elif current_price < average_price:

# Sell all of our shares by setting the target position to zero
order_target(context.security, 0)
log.info("Selling %s" % (context.security.symbol))
'''

# You can use the record() method to track any custom signal.
# The record graph tracks up to five different variables.
# Here we record the Apple stock price.
record(stock_price=data[context.security].price)
There was a runtime error.

Hi Seong,
I understand your argument. That is also what I guess.

For minute level test, this problem is not severe. But for day level, the back test result is totally off. And it can hardly be addressed by slippage model only.

A more reasonable solution is to update the definition of bar, no matter minute bar or day bar. A bar should have open/close/volume at the minimum. The order calculated based on close price of bar1 should be fulfilled by open price of bar2. If you want to make it more realistic, then take volume into consideration as well.

I point this out and urge for a solution cause the back test results here is unreliable. If my back test is good, I turn on the live trade. I may get totally different result.

Hi David,
Minute level would be a solution. I can run the calculation at 15:58 each day. So I can based on a price very close to market close to calculate and fulfill the order. But it has a couple issues:
1. Speed. This is quite obvious.
2. I need a trading calendar see if I should calculate at 12:58 for those half-day trading dates like the day after Thanks giving.
3. Most importantly, I cannot trust any day level back test result in this website.

To demonstrate this problem, I created two back tests:
The first one is a clone of what Florent contributed a week ago @ https://www.quantopian.com/posts/system-based-on-easy-volatility-investing-by-tony-cooper-at-double-digit-numerics
The second one, I changed her code slightly. I ran the test in minute mode. But only calculate once a day @ 15:58pm. This make sure I can use price close to last price to calculate and fulfill the orders within the same day rather than the end of next trade day.
You can see how big the difference could be:

8
Backtest from to with initial capital
Total Returns
--
Alpha
--
Beta
--
Sharpe
--
Sortino
--
Max Drawdown
--
Benchmark Returns
--
Volatility
--
 Returns 1 Month 3 Month 6 Month 12 Month
 Alpha 1 Month 3 Month 6 Month 12 Month
 Beta 1 Month 3 Month 6 Month 12 Month
 Sharpe 1 Month 3 Month 6 Month 12 Month
 Sortino 1 Month 3 Month 6 Month 12 Month
 Volatility 1 Month 3 Month 6 Month 12 Month
 Max Drawdown 1 Month 3 Month 6 Month 12 Month
import pandas
import math
import numpy

def rename_col(df):
df = df.rename(columns={'Close': 'price'})
df = df.fillna(method='ffill')
df = df[['price', 'sid']]
return df

def initialize(context):
date_column='Date',
date_format='%m/%d/%Y',
symbol='vix',
usecols=['Close'],
pre_func=preview,
post_func=rename_col)
context.spy = sid(8554)
context.xiv = sid(40516)
context.vxx = sid(38054)
pass

# see a snapshot of your CSV for debugging
def preview(df):
return df

# would prefer if this was a batch_transform of a batch_transform rather than using rolling stuff
#@batch_transform(window_length=20, refresh_period=1)
def get_vrp(prices, spy_sid, current_vix):
spy = prices[spy_sid]
log_spy = numpy.log(spy)
spy_log_returns = log_spy.diff()[1:]
hvol = pandas.stats.moments.rolling_std(spy_log_returns, 2) * math.sqrt(254) * 100
last_hvol = hvol.iloc[-1]
log.info('hvol = %f vix = %f' % (last_hvol, current_vix.price))
vrp = current_vix.price - hvol
svrp = pandas.stats.moments.rolling_median(vrp, 1)
last_smoothed_vrp = svrp[-1:].ix[0]
record(hvol=last_hvol,vix_price=current_vix.price,vrp=last_smoothed_vrp)
return last_smoothed_vrp

def close_gap(sid, current, target):
order(sid, target-current)

def handle_data(context, data):

prices = history(20, '1d', 'price')
vrp = get_vrp(prices, context.spy, data['vix'])

if (vrp is not None):
# this is convoluted
record(vrp = vrp)
target_xiv = 0
target_vxx = 0
capital = context.portfolio.positions_value + context.portfolio.cash
current_xiv = context.portfolio.positions[context.xiv].amount
current_vxx = context.portfolio.positions[context.vxx].amount

# this is the threshold at which we enter XIV and harvest VRP
if (vrp > 2):
target_xiv = capital / data[context.xiv]['price']
# this is the threshold at which we enter VXX, which usually bleeds due to VIX contango
elif (vrp < 0.5):
target_vxx = capital / data[context.vxx]['price']

if (target_xiv > 0):
# enter in only if we have none, to prevent pyramiding
if (current_xiv == 0):
close_gap(context.xiv, current_xiv, target_xiv)
else:
# always close out
close_gap(context.xiv, current_xiv, target_xiv)

if (target_vxx > 0):
# enter in only if we have none, to prevent pyramiding
if (current_vxx == 0):
close_gap(context.vxx, current_vxx, target_vxx)
else:
# always close out
close_gap(context.vxx, current_vxx, target_vxx)


There was a runtime error.

Here is the second test I mentioned:

7
Backtest from to with initial capital
Total Returns
--
Alpha
--
Beta
--
Sharpe
--
Sortino
--
Max Drawdown
--
Benchmark Returns
--
Volatility
--
 Returns 1 Month 3 Month 6 Month 12 Month
 Alpha 1 Month 3 Month 6 Month 12 Month
 Beta 1 Month 3 Month 6 Month 12 Month
 Sharpe 1 Month 3 Month 6 Month 12 Month
 Sortino 1 Month 3 Month 6 Month 12 Month
 Volatility 1 Month 3 Month 6 Month 12 Month
 Max Drawdown 1 Month 3 Month 6 Month 12 Month
import pandas
import math
import numpy

def rename_col(df):
df = df.rename(columns={'Close': 'price'})
df = df.fillna(method='ffill')
df = df[['price', 'sid']]
return df

def initialize(context):
date_column='Date',
date_format='%m/%d/%Y',
symbol='vix',
usecols=['Close'],
pre_func=preview,
post_func=rename_col)
context.spy = sid(8554)
context.xiv = sid(40516)
context.vxx = sid(38054)
pass

# see a snapshot of your CSV for debugging
def preview(df):
return df

# would prefer if this was a batch_transform of a batch_transform rather than using rolling stuff
#@batch_transform(window_length=20, refresh_period=1)
def get_vrp(prices, spy_sid, current_vix):
spy = prices[spy_sid]
log_spy = numpy.log(spy)
spy_log_returns = log_spy.diff()[1:]
hvol = pandas.stats.moments.rolling_std(spy_log_returns, 2) * math.sqrt(254) * 100
last_hvol = hvol.iloc[-1]
log.info('hvol = %f vix = %f' % (last_hvol, current_vix.price))
vrp = current_vix.price - hvol
svrp = pandas.stats.moments.rolling_median(vrp, 1)
last_smoothed_vrp = svrp[-1:].ix[0]
record(hvol=last_hvol,vix_price=current_vix.price,vrp=last_smoothed_vrp)
return last_smoothed_vrp

def close_gap(sid, current, target):
order(sid, target-current)

def handle_data(context, data):
exchange_time = pandas.Timestamp(get_datetime()).tz_convert('US/Eastern')
if exchange_time.hour != 15 or exchange_time.minute != 58:
return

prices = history(20, '1d', 'price')
vrp = get_vrp(prices, context.spy, data['vix'])

if (vrp is not None):
# this is convoluted
record(vrp = vrp)
target_xiv = 0
target_vxx = 0
capital = context.portfolio.positions_value + context.portfolio.cash
current_xiv = context.portfolio.positions[context.xiv].amount
current_vxx = context.portfolio.positions[context.vxx].amount

# this is the threshold at which we enter XIV and harvest VRP
if (vrp > 2):
target_xiv = capital / data[context.xiv]['price']
# this is the threshold at which we enter VXX, which usually bleeds due to VIX contango
elif (vrp < 0.5):
target_vxx = capital / data[context.vxx]['price']

if (target_xiv > 0):
# enter in only if we have none, to prevent pyramiding
if (current_xiv == 0):
close_gap(context.xiv, current_xiv, target_xiv)
else:
# always close out
close_gap(context.xiv, current_xiv, target_xiv)

if (target_vxx > 0):
# enter in only if we have none, to prevent pyramiding
if (current_vxx == 0):
close_gap(context.vxx, current_vxx, target_vxx)
else:
# always close out
close_gap(context.vxx, current_vxx, target_vxx)


There was a runtime error.

1250.53% vs -4.7%.
How should I understand the difference?

Hi Alan,

I'm looking for the 12:58 clause in your minutely backtest but can't seem to find it. Do you mind replicating it using schedule_function() and give that a shot? I believe right now it's executing every minute.

There is no logic for 12:58. As I said, I have to maintain a trading calendar if I want to do that. Without this, every year, I lost 3 trading days. But that won't change the performance upside down. Cause from the result, we didn't find any spike on those 3 days each year.
What I added on top of Florent's code is just:

    exchange_time = pandas.Timestamp(get_datetime()).tz_convert('US/Eastern')
if exchange_time.hour != 15 or exchange_time.minute != 58:
return



Even though handle_data will be called each minute, but given the code above, calculation will be fired only once a day shortly before market close. If anyone want to try Florent's idea, I think that is what he/she will do.
I am not familiar with the schedule_function(). So I don't want to bother. Even if it can allow be to trade 2 minutes before market close for those 3 days, it won't change the comparison above upside down. I would like to focus on the major difference.If you like, I can rerun the test for a period without half-trading day to demonstrate the difference again.

In backtesting, daily mode is used to get a sense of the strategy, to begin developing your idea. Once you have a concept, switch to minute mode to closely simulate the live trading environment. Using minute mode doesn't mean that you trade every minute - only that the algo receives minutely data. You can use schedule_function to schedule specific dates and times for trading. This eliminates the speed issues and need for maintaining the trading calendar - it's already built in.

When you're ready to deploy your algorithm to live trading, you are required to run a full backtest in minute mode to simulate the performance. This way you can directly see how the algorithm behaves. If you want to test your strategy before deploying with real money, you can paper trade it with Quantopian or connect your IB demo account.

Disclaimer

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.

Alan,

In this particular case, part of the performance discrepancy between the two backtests you've posted probably has to do with how fetch_csv handles time series data in daily vs minute mode. Basically, running that algorithm in minute mode allows the algorithm to examine the current day's closing VXMT value at the start of the day (which is impossible in real life, unless you're Marty McFly). The minute mode version of that algorithm would need to use post_func to shift the dates up by 1 day so the minute mode algorithm can't see into the future.

Matt,
I totally understand the look ahead issue. That is exactly why I let the calculation happens at 15:58. Even though there is still two minutes look ahead. It shouldn't be that big.
Correct, shift the data by one day avoid the look ahead bias. But that creates a bigger issue. Instead of 2 minutes look ahead bias, that uses one day old data. If you ask me to guess where the price is at 15:58pm today. I would say it is closer to close price today rather than the close price of yesterday.
Do you agree?

Alisa,
I agree with your opinion that minute level back test is way more accurate than the day level. What I want to say is, day level back test could have been much closer to reality if back tester execute the order in more widely accepted ways, either use the close price of current cycle or use open price of next cycle. When you see the last price of a stock, what price do you think your market order would be fulfilled? Without knowledge of BBO and book depth, your best guess is the last price you saw rather than the close price of next cycle, particularly when cycle is long like a day.
If we look at the discussions on this site, most back tests were ran at day level. Algorithms were discussed based on these day level back tests. Like the result I copied from Florent above, everyone is so excited. But it is so far away from the minute level test. That make the day level test result hard to compare.

Alan, we're in complete agreement. In the early iterations of the backtester, daily mode trades and fills orders at the end of day close price. An order submitted on Monday EOD is filled on Tuesday EOD. In hindsight, perhaps an improved simulation would be to submit orders at the end of the day and fill them at the next day's open. As we go forward, I would like to see the daily mode deprecated and use minute mode simulation. This would give more accurate pricing and fills, remove confusion, and align the behavior directly with live trading.

Why not make Daily just simply a subset of Minute that feeds data only at 15:59 (with fill at 16:00)? Thus maybe side-step the industry "Daily" OHLCV concept (rename it to something other than "Daily" if that seems appropriate), and just feed only one minute per day, the idea is to obtain a quick rough idea. Tons of issues evaporate (datetime, extreme slippage et al). Any down-side?

With schedule_function, Minute tests are still slow so that's not a solution, some might not realize this, to be clear, stay with me here, it's due to the fact that all 390 data frames are still being rounded up from the database, packaged, transmitted and the information for all securities fed in calls to handle_data(), 24 Gigabytes--with a G--per backtesting-year of frame content (with set_universe on max at 300 sids), even if only 1 minute of the day is specified in schedule_function. That 1 minute applies only to the new user-specified function that schedule_function was told to run, not handle_data() which still receives everything on all 390 minutes. Maybe the schedule_function func could be changed to be allowed to be handle_data().

Cool. Both Anony and Gary feed in very good point.
I think it really depends on how you want to trade finally.
If you want your calculation based on precisely the close price of day, then it is better to test it with Anony's suggestion, let your order get executed with the open price of next day.
If you want your order fulfilled within the same day to avoid the big price move over night due to earning announcement etc, then go Gary's suggestions, use 15:59 price to calculate and place the order. This way suffered from the look ahead bias mentioned by Matt if your calculation use EOD price of some user provided daily data. But hopefully the real 15:59 price is not far from the close price. Then such test is still a good approximation.
I would suggest Quantopian to make a small enhancement to support Gary's way. Currently using either schedule_function or my simply code snippet can achieve this. But under the hood, a lot of data were processed unnecessarily. That makes the test slow for user and expensive for Quantopian.
If the user choose to run the calculation at 15:55 for his daily test, should we fulfill the order use price at 15:55 or close price of day or price of 15:56? That is open for debate. Each can serve different purpose. It would be nice to provide it as an option. If no option provided, I think using open price of 15:56 minute bar would be the best approximation.

It discussed about using customized SlippageModel to choose different prices to fulfilled the order. Today close, next open and next close etc. That should be able to address the issue I raised. I would strongly suggest Quantopian integrate this as a standard option for the daily test. That would save people a lot of time to duplicate the code. More importantly, avoid people running daily test and fulfill the order with next day close price. As @Andrew Blount mentioned, waiting until the next period's close is not how "real trading" works.

Thing with OHLC bars is that they aren't technically point-in-time. It's a range. You can conceptually break EOD data into multiple points like

* denotes a price event

*Day1.Close
Post-close->Pre-open
Compute stats based on close. Orders can only execute on next day prices.
*Day2.Open
Post-open
Can do calcs on open prices. Can set limit orders, MOC.
*Day2.High Day2.Low
Pre-close
Since we don't know time ordering of high/low, you can only send MarkClose orders.
If an existing limit order had an attached stop and/or target, you'd always assume the most pessimistic outcome.
Example: Limit order was hit, then stop was hit.
In reality, this might not be true since Target might have been hit first.
*Day.Close
Post-close->Pre-open
Once again, we have close. Can compute orders for next open.


Breaking up the Daily into discrete portions allows a bit more granularity while still only using daily data. With the above breakdown, it's possible to trade the close based on open price of the same day without lookahead bias. If you have minutely data, you can always add another column like price_at_noon and pre_close, which would create additional discrete portions/events.

The backtester could also breakup the day into two timeframes, market-open -> 5-minutes-before-close and 5-minutes-before-close -> close. You'd do this because it's possible you care about the daily range so far, which skipping to just the 15:55 minute bar wouldn't tell you. Precomputing new OHLC data is a lot faster than sending in every minute bar.

I'm not actually sure if the point of daily mode was to quasi-replicate minutely backtests with faster EOD or to test a different time frame.

You sound experienced. I'm a neophyte, do not understand why there could be a look-ahead problem in this case and do not know how Q works at the database level so I have a question.
I gather that along with daily Close prices, there are stored Open, High, Low and Volume numbers for the day and those are all used for daily mode.
So I'm wondering whether the following could make sense:

Use the daily OHL and V pre-stored information for daily tests, not Close though.
For C, the close price, it is replaced with the minute-close 15:59 price.
Only that assembled set is fed as 'data' in daily mode (only one frame per day).
Fills can happen sort of at 16:00.
The same daily mode fast execution.
In Quantopian code, it is like minute mode (a lot of the same minute code is active) except it skips 389 out of 390 minutes -ish.

To put it another way, minute OHL and V for 15:59 are replaced by daily mode values.


The 15:59 price is likely to be closer to the 16:00 price a minute later than a price 24 hours later, resulting in daily tests nudging closer to minute tests.

Any downside?

There's no look ahead there. My concern with the minute-at-15:59 was that it couldn't just be the minute bar. It'd have to be the session up until then, but you might have had that in mind and I misunderstood. I still think you'd need the actual close transmitted to you at some point, depending on how volatile the close is it might result in a new daily high. Some algos might need that info after the fact, even if it's not something you could trade on the day of.