Back to Community
How to get VIX/VXV/VXMT/VXST/VVIX into algos with history.

This is not a new technique, I first posted this technique in February 2015. However, since then, there's been post after post about how getting it through pipeline has issues, fetch_csv doesn't work in live trading, or it has lookahead bias in backtests.

Attached is the technique I used in my algos which I live-traded for over a year (but recently stopped for compliance reasons). It should be easily adaptable to other time series that you want to have the time-series of inside your scheduled functions or callbacks, so you don't have to jam all the business logic into a single pipeline custom factor.

Although this example is only getting the last day of vix_vals via [-1], there are History days prior, so you can easily do pandas time-series transformations on the data to get moving averages, standard deviations, etc...

Note that algos which use this are not eligible for the contest, or the fund, or anything, but for personal trading, it worked fine for me for months, notwithstanding bugs, restarts, disconnections and Q2.

Good luck,

Simon.

Clone Algorithm
244
Loading...
Backtest from to with initial capital
Total Returns
--
Alpha
--
Beta
--
Sharpe
--
Sortino
--
Max Drawdown
--
Benchmark Returns
--
Volatility
--
Returns 1 Month 3 Month 6 Month 12 Month
Alpha 1 Month 3 Month 6 Month 12 Month
Beta 1 Month 3 Month 6 Month 12 Month
Sharpe 1 Month 3 Month 6 Month 12 Month
Sortino 1 Month 3 Month 6 Month 12 Month
Volatility 1 Month 3 Month 6 Month 12 Month
Max Drawdown 1 Month 3 Month 6 Month 12 Month
# Backtest ID: 5854283464337e62585041a5
There was a runtime error.
21 responses

Thanks Simon

Thanks for sharing the code, Simon.

It's worth noting that a file that ingests a signal like this would be considered for an allocation, particularly because signals like this are being added as data sources all the time. On the other hand if your algorithm is ingesting only a buy/sell list, and all of the business logic is outside of Quantopian, that is much less likely to be considered for an allocation. We don't have a way to maintain a point-in-time dataset and do out-of-sample testing.

Disclaimer

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.

@Simon Could u kindly explain why you need to add a bar ? # add today, and shift all previous data up to today. This
# should result in the same data frames as in backtest
Let's say Fetcher function return's data until 2016-12-21 and today is 2016-12-22, I think no bar should be added. A simple

def reformat_quandl(df):  
    df = df.fillna(method='ffill')  
    df = df.tshift(1, freq=tradingcalendar.trading_day)  
    return df  

will do the job to avoid look ahead bias

That won't work in live trading, last I checked.

@Simon I checked my code and used

vixUrl = 'http://www.cboe.com/publish/scheduledtask/mktdata/datahouse/vixcurrent.csv'  
fetch_csv(vixUrl, symbol='VIX', skiprows=1,date_column='Date', post_func=reformat_quandl)  

Everything works fine so far . The orders all execute correctly

Cool. freq=tradingcalendar.trading_day ensures that it will correctly shift over trading holidays?

@Simon Thanks for reminding me that. I am not sure about that. The non-holidays seems works. Do you suggest It won't work for holidays in live trading ? Maybe I 'll know it soon at Dec 26?

Guess we'll find out!

Thank you for sharing the code!

Thank you very much for the code, very helpful!

Pardon my ignorance, but any idea why VXST will fail to unpack? I removed the comment #s and I looked at the raw CSV and it all seemed fine - does it have to do with the discontinuation of the VXST futures/options?

Hi Simon,

Thought the Q has some VIX class or factor, but I still find your methods here quite interesting. I've tried use your method to get the VXNs but fails (see my attached algo. I doubt the data format in the csv file is not float but sting. I wrote to CBOE. But they told me the data format is correct.

Maybe you have an idea?

Cheers

Thomas

Clone Algorithm
18
Loading...
Backtest from to with initial capital
Total Returns
--
Alpha
--
Beta
--
Sharpe
--
Sortino
--
Max Drawdown
--
Benchmark Returns
--
Volatility
--
Returns 1 Month 3 Month 6 Month 12 Month
Alpha 1 Month 3 Month 6 Month 12 Month
Beta 1 Month 3 Month 6 Month 12 Month
Sharpe 1 Month 3 Month 6 Month 12 Month
Sortino 1 Month 3 Month 6 Month 12 Month
Volatility 1 Month 3 Month 6 Month 12 Month
Max Drawdown 1 Month 3 Month 6 Month 12 Month
# Backtest ID: 58be4798dd287861d83110c8
There was a runtime error.

In live trading I have found the Quandl data to be useless, it is updated too late.
So I was looking at how to compute some of these ratios directly from the CBOE site.
Suppose I want the contango ratio Vx1/Vx2. Looking at http://www.cboe.com/delayedquote/futures-quotes
today (3/13/2017 at 2:38 Pacific time) I see:
VIX/H7 last=12.10 settle=12.13 exp=3/22
VIX/J7 last=13.85 settle=13.88 exp=4/19
VIX/K7 last=14.81 settle=14.88 exp=5/17

I assume to get the "most current" contango ratio I should use the "last" values
Vx1 = H7 = 12.10
Vx2= J7 = 13.85
So the contango ratio = 12.1/13.85
On 3/22 H7 will expire so I will start using J7/K7, Is all this correct?

It would seem that since the time value of the option depends on the sqrt of the time to expiration,
that the contango ration will be discontinuous when the option expires. It will rise sharply as the
age of Vx1 goes from zero to 30 days.
Quandl uses the settle date, is there any point in doing that instead of using last?
I tried doing it this way and comparing to Quandl, the values are close, but not exact.
(I feel like I should know this stuff before trying it but the definitions are confusing). Thanks

The price of the VIX future will converge to the VIX as the maturity date draws near; to perform an apples-to-apples comparison of VIX futures with the spot VIX, you would need to pick a model for calculating a constant-maturity VIX future. I've done it with linear interpolation, but other interpolations are possible.

Probably what you really want is something like the spot VIX compared with a constantly-changing blend of VX1 and VX2 targeting 30 days in the future or something. In my trading, I was just comparing the VIX with the VXV, which is the spot VIX calculation but of underlying options of different expirations. Another caveat to be aware of; the further VX futures are actually futures on effectively a different VIX, since the options that go into calculating the VIX today may not be the options that go into calculating the VIX by the time the VX future matures. Depends how much you care about all of this.

There are CSV files for VIX Futures by month here: http://cfe.cboe.com/market-data/historical-data#VX
Is there a function available for downloading the correct CSV for VX1 and VX2?

Such a function would be easy to write, if you calculate when you want to roll the futures. I think futures data is integrated into Quantopian now, though I hear their VX contracts only go back to 2012.

It might be complex to write if rolling by volume, since the data has to be downloaded first.
Their VX contracts start on July 2012. I have backtests with the Quantopian futures data and it works well, but like the other method, it does not work in live trading. I may have to use VXV as an approximation for now until Quantopian futures is fully implemented.

I've set up a Linux server in the Amazon cloud to automatically collect VX futures data from http://vixcentral.com/historical/?days=3000 and save it to a CSV file that is accessible from the web. The server uses Ruby and code from https://gist.github.com/sandys/3910840 to convert the HTML table to CSV. I then use Eric K's code to fetch and date shift the data. Hopefully this allows for getting the values of VX1 and VX2 for live trading without relying on Quandl or Quantopian futures.

can other people access this as well? I would love to access it to triangulate my algo's and to get them to trade more accuaterely

@Peter

Sure, but the server will be taken down once Quantopian futures data works in live trading. I'm still looking for a way to get VX data that is rolled based on contract volume.

from zipline.utils import tradingcalendar

def reformat_noko(df):  
    df = df.fillna(method='ffill')  
    df = df.tshift(1, freq=tradingcalendar.trading_day)  
    return df  

def initialize(context):  
    nokoUrl = 'http://52.15.233.150/noko.csv'  
    fetch_csv(nokoUrl, symbol='vx', date_column='Date', date_format='%Y-%m-%d', post_func=reformat_noko)

def ordering_logic(context, data):  
    v1_close = data.current('vx','F1')  
    v2_close = data.current('vx','F2')  

update 9/9: Server shutting down due to Quantopian's upcoming end of support for live trading

Hey would anyone be willing to answer a question on data shifting? I'm having a hard time figuring
out exactly what is going on.

From my understanding when you do df.tshift(1) on the VIX data it is supposed to shift everything one day to
prevent forward looking bias, right?

So does that mean when you are live trading you should remove the df.tshift(1) to get current values?
Or is it perfectly fine live trading with the df.tshift(1) implemented?

I didn't use tshift, not sure where you got that from. tshift will make it hard to work properly with trading holidays.

Generally, fetch_csv provides data to your algo if it has the same date as the current trading day. This means that you want to shift (somehow) yesterday's data forward to "today" in order to make sure that you aren't using today's closing price in the morning, which is a very easy-to-make error.

However, you furthermore need to make sure you shift forward to "today" in live trading, a date which will not exist at all in your dataframe. The easy way is tshift, using business days, but again, that will only work if yesterday was not a trading holiday. It's possible that there is a way to tshift using the trading calendar itself, but I am not certain, it's been a couple of years now since I was actively programming quantopian stuff.

Whatever you do, I would recommend not doing things differently for backtest vs live trading. Pick a technique that is going to work consistently throughout.