Inferring latent states using a Gaussian Hidden Markov Model

Example inspired by a scikit-learn example. I used this during my talk at the NYSE meetup earlier this week.

The idea is to predict hidden states in the daily price fluctuations and trading volume using a Hidden Markov Model (see the graphic). This model essentially assumes the existence of discrete hidden states. Each hidden state is associated with a certain probability of moving to another state at the next time point (thus, the current state is dependent on the previous one -- that's the Markov property). In addition, each hidden state is associated with emitting an observable event (in this case, fluctuation and volume) with a certain probability. This example tries to infer the hidden state transition probabilities, the observable events emitted, and then try to predict the hidden state of the current market.

Since we are continuously recomputing the HMM I set the previously learned means as a prior for the next model. So we are using the observed states we already learned for the next model.

Finally, this is so far just an analysis. Turning this into a trading strategy would require inferring what specific states mean and then place orders in response to that. This might not be quite as easy (you might want to look at the inferred means for this). But the Markov assumption that the current state depends on the previous will almost certainly be violated at least for the price fluctuations -- it is a well known fact that returns have almost 0 autocorrelation. Volume does, however, have autocorrelation so it might work better there. In addition, it's not clear that there are discrete states underneath. The state space could be continuous. If that's the case, a Kalman filter might be interesting to explore.

564
Total Returns
--
Alpha
--
Beta
--
Sharpe
--
Sortino
--
Max Drawdown
--
Benchmark Returns
--
Volatility
--
 Returns 1 Month 3 Month 6 Month 12 Month
 Alpha 1 Month 3 Month 6 Month 12 Month
 Beta 1 Month 3 Month 6 Month 12 Month
 Sharpe 1 Month 3 Month 6 Month 12 Month
 Sortino 1 Month 3 Month 6 Month 12 Month
 Volatility 1 Month 3 Month 6 Month 12 Month
 Max Drawdown 1 Month 3 Month 6 Month 12 Month
from sklearn.hmm import GaussianHMM
import numpy as np

# Define batch transform which will be called
# periodically with a continuously updated array
# of the most recent trade data.
@batch_transform(refresh_period=100, window_length=300)
def HMM(data, means_prior=None):
# data is _not_ an event-frame, but an array
# of the most recent trade events

# Create scikit-learn model using the means
# from the previous model as a prior
model = GaussianHMM(3,
covariance_type="diag",
n_iter=10,
means_prior=means_prior,
means_weight=0.5)

# Extract variation and volume
diff = data.variation[3951].values
volume = data.volume[3951].values
X = np.column_stack([diff, volume])

# Estimate model
model.fit([X])

return model

# Put any initialization logic here  The context object will be passed to
# the other methods in your algorithm.
def initialize(context):
# some sids to look at
context.sid = sid(3951)
context.means_prior = None

def handle_data(context, data):
c = context

# add the day's price range to the list for this sid
data[c.sid]['variation'] = (data[c.sid].close_price - data[c.sid].open_price)

# Pass event frame to batch_transform
# Will _not_ directly call the transform but append
# data to a window until full and then compute.
model = HMM(data, means_prior=c.means_prior)

if model is None:
return

# Remember mean for the prior
c.means_prior = model.means_

data_vec = [data[c.sid].variation, data[c.sid].volume]
state = model.predict([data_vec])

log.info(state)
This backtest was created using an older version of the backtester. Please re-run this backtest to see results using the latest backtester. Learn more about the recent changes.
There was a runtime error.
Disclaimer

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.

16 responses

Sorry to revive an old thread, but is their a video accompanying the NYSE link? I would love to see it!
Thanks,
Brandon

Hi Thomas, I agree with Brandon, it would be nice to look at the video, or at least to have access to the material of the presentations.

Hi guys,

Unfortunately there was no video recording of this talk. The notebook I used also has less descriptions than what can be found here. I'm happy to try and do a better job explaining what's going on (but note that there are good resources on HMMs online as well).

Related, Jess and I have been talking about how to extend this strategy so if something results from that we'll update it and hopefully provide a better description.

Thomas

Hello, I'm getting this compiling error on your alto.

Nonexistent property: variation on line 54.

Hi Marc,

I updated the backtest to use the more modern features available on Quantopian. To retrain the model monthly I'm using the new schedule_func (https://www.quantopian.com/help#ide-schedulefunction) and instead of a batch_transform I'm using history.

Let me know if you have any questions.

Thomas

564
Total Returns
--
Alpha
--
Beta
--
Sharpe
--
Sortino
--
Max Drawdown
--
Benchmark Returns
--
Volatility
--
 Returns 1 Month 3 Month 6 Month 12 Month
 Alpha 1 Month 3 Month 6 Month 12 Month
 Beta 1 Month 3 Month 6 Month 12 Month
 Sharpe 1 Month 3 Month 6 Month 12 Month
 Sortino 1 Month 3 Month 6 Month 12 Month
 Volatility 1 Month 3 Month 6 Month 12 Month
 Max Drawdown 1 Month 3 Month 6 Month 12 Month
from sklearn.hmm import GaussianHMM
import numpy as np

# Define batch transform which will be called
# periodically with a continuously updated array
# of the most recent trade data.

# Put any initialization logic here  The context object will be passed to
# the other methods in your algorithm.
def initialize(context):
# some sids to look at
context.sid = sid(3951)
context.means_prior = None
context.model = None
schedule_function(hmm_train,
date_rules.month_start())

def hmm_train(context, data):
df_close = history(300, '1d', 'close_price')
df_open = history(300, '1d', 'open_price')
df_volume = history(300, '1d', 'volume')
# add the day's price range to the list for this sid
df_variation = (df_close - df_open)

# Create scikit-learn model using the means
# from the previous model as a prior
model = GaussianHMM(3,
covariance_type="diag",
n_iter=10,
means_prior=context.means_prior,
means_weight=0.5)

# Extract variation and volume
diff = df_variation.values
volume = df_volume.values
X = np.column_stack([diff, volume])

log.info('Retraining model')
# Estimate model
model.fit([X])

# Update model
context.model = model

# Remember mean for the prior
context.means_prior = model.means_

def handle_data(context, data):
c = context

if context.model is None:
return

variation = (data[c.sid].close_price - data[c.sid].open_price)
data_vec = [variation, data[c.sid].volume]
state = context.model.predict([data_vec])

log.info(state)
There was a runtime error.

Hi Thomas, your algo doesnt seem to load anymore, could you post it again?

Peter, did you use the last one I posted in this thread? That one works fine for me. Can you post the error you are getting?

Hi Thomas, it looks like you have a scheduled function for training. On live trading, would this execute in parallel with the algorithm, or does the algorithm wait until training is complete?

Grant, the algorithm would wait for the training to complete.

Ok, by the way, is there any support for long processes that execute outside of market hours?
If I started a task at market close, could it execute overnight or over the weekend?

Currently that's not implemented as we shut down the algos over night and restart them a few hours before open, but it's an interesting idea!

Hello Thomas,

I posted some questions on https://www.quantopian.com/posts/before-trading-start-does-it-time-out, and on https://www.quantopian.com/posts/history-not-working-in-before-trading-start Grant Kot had asked when before_trading_start gets called prior to market open? So, depending on the answers, there could be a goodly chunk of time overnight to run code, right?

Also, above, you say "the algorithm would wait for the training to complete" but doesn't the standard 50-second time-out for handle_data also apply to a scheduled function? Or could bars be skipped while the scheduled function completes?

Grant

just wondering: i saw that the hmm functionality is deprecated in the current scikit version. will there be a workaround when you guys upgrade to 0.17 (and the hmm will be dropped from scikit) ?
thx

That's a good question. There's nothing planned right now but something like https://github.com/mattjj/pyhsmm would be really cool.

Following up on this, is there any chance pyhsmm or hmmlearn could be added?

I updated the code to use the new API, modified the training frequency, increased the number of states and EM iterations. I'm recording the HMM state variable and comparing against SPY benchmark. I'm wondering whether the state labels remain consistent for every run of the HMM algorithm. Also, it's not clear how to map HMM states to trends in S&P500. An alternative to using a fixed number of states is HDP-HMM that infers the number of states from the data. However, the problem remains of interpreting the resulting states.

59
Total Returns
--
Alpha
--
Beta
--
Sharpe
--
Sortino
--
Max Drawdown
--
Benchmark Returns
--
Volatility
--
 Returns 1 Month 3 Month 6 Month 12 Month
 Alpha 1 Month 3 Month 6 Month 12 Month
 Beta 1 Month 3 Month 6 Month 12 Month
 Sharpe 1 Month 3 Month 6 Month 12 Month
 Sortino 1 Month 3 Month 6 Month 12 Month
 Volatility 1 Month 3 Month 6 Month 12 Month
 Max Drawdown 1 Month 3 Month 6 Month 12 Month
from sklearn.hmm import GaussianHMM
import numpy as np

def initialize(context):

# some sids to look at
context.security = sid(8554)

context.means_prior = None
context.model = None
context.num_states = 16
context.history_range = 300

schedule_function(hmm_train,date_rules.week_end(), time_rules.market_close(minutes=60))
schedule_function(hmm_state,date_rules.every_day(), time_rules.market_open(minutes=60))

def hmm_train(context, data):
df_close = data.history(context.security, 'close', context.history_range, '1d')
df_open = data.history(context.security, 'open', context.history_range, '1d')
df_volume = data.history(context.security, 'volume', context.history_range, '1d')
# add the day's price range to the list for this sid
df_variation = (df_close - df_open)

# Create scikit-learn model using the means
# from the previous model as a prior
model = GaussianHMM(context.num_states,
covariance_type="diag",
n_iter=100,
means_prior=context.means_prior,
means_weight=0.5)

# Extract variation and volume
diff = df_variation.values
volume = df_volume.values
X = np.column_stack([diff, volume])

log.info('Retraining model')
# Estimate model
model.fit([X])

# Update model
context.model = model

# Remember mean for the prior
context.means_prior = model.means_

def hmm_state(context, data):
c = context

if context.model is None:
return

close = data.current(context.security, 'close')
opens = data.current(context.security, 'open')
volume = data.current(context.security, 'volume')

variation = close - opens
data_vec = [variation, volume]
state = context.model.predict([data_vec])

record(state=state)

def handle_data(context, data):
pass
There was a runtime error.