Back to Community
A naive attempt using Kinetic Component Analysis

A naive implementation of trading rules on Kinetic Component Analysis. Anyone care to improve trading rules (entry/exit)?

Clone Algorithm
65
Loading...
Backtest from to with initial capital
Total Returns
--
Alpha
--
Beta
--
Sharpe
--
Sortino
--
Max Drawdown
--
Benchmark Returns
--
Volatility
--
Returns 1 Month 3 Month 6 Month 12 Month
Alpha 1 Month 3 Month 6 Month 12 Month
Beta 1 Month 3 Month 6 Month 12 Month
Sharpe 1 Month 3 Month 6 Month 12 Month
Sortino 1 Month 3 Month 6 Month 12 Month
Volatility 1 Month 3 Month 6 Month 12 Month
Max Drawdown 1 Month 3 Month 6 Month 12 Month
from quantopian.pipeline import Pipeline
from quantopian.pipeline.data.builtin import USEquityPricing
from quantopian.algorithm import attach_pipeline, pipeline_output
from quantopian.pipeline.filters.morningstar import Q1500US
from quantopian.pipeline.classifiers.morningstar import Sector
from quantopian.pipeline.factors.morningstar import MarketCap

import cvxpy as cvx
import numpy as np
import pandas as pd
from pykalman import KalmanFilter

def fitKCA(t,z,q,fwd=0):
     '''
     Inputs:
     t: Iterable with time indices
     z: Iterable with measurements
     q: Scalar that multiplies the seed states covariance
     fwd: number of steps to forecast (optional, default=0) '''
     #1) Set up matrices A,H and a seed for Q
     h=1. / t.shape[0]
     A=np.array([[1,h,.5*h**2, 1./6.*h**3],
                 [0,1,h,h**2],
                 [0,0,1,h],
                 [0,0,0,1]])
    
     Q=q*np.eye(A.shape[0])
     #2) Apply the filter
     kf=KalmanFilter(transition_matrices=A,transition_covariance=Q)
     #3) EM estimates
     kf=kf.em(z)
     #4) Smooth
     x_mean,x_covar=kf.smooth(z)
     #5) Forecast
     for fwd_ in range(fwd):
         x_mean_,x_covar_=kf.filter_update(filtered_state_mean=x_mean[-1],
                                       filtered_state_covariance=x_covar[-1])
         x_mean=np.append(x_mean,x_mean_.reshape(1,-1),axis=0)
         x_covar_=np.expand_dims(x_covar_,axis=0)
         x_covar=np.append(x_covar,x_covar_,axis=0)
     #6) Std series
     x_std=(x_covar[:,0,0]**.5).reshape(-1,1)
     for i in range(1,x_covar.shape[1]):
         x_std_=x_covar[:,i,i]**.5
         x_std=np.append(x_std,x_std_.reshape(-1,1),axis=1)
     return x_mean,x_std,x_covar

def initialize(context):
    set_commission(commission.PerShare(cost=0.001, min_trade_cost=0))

    context.weights = None
    context.leverage = 1.0
    context.days = 60
    context.output = None
    context.sign = 0
    context.cumret = 0
    context.x_mean = 0
    context.SPY = sid(8554)
    schedule_function(opentrades, 
                      date_rules.every_day(), 
                      time_rules.market_open(minutes=90))
    
    schedule_function(record_vars, 
                      date_rules.every_day(), 
                      time_rules.market_close(minutes=1))


def opentrades(context, data):
    hist = data.history([context.SPY], 'price', 250, '1d').resample('W').last()
    dmret = np.log1p(hist.pct_change().dropna().values[:, 0])    
    dcumret = np.cumsum(dmret, axis=0)
    t = np.asarray(range(0, dcumret.shape[0]))
    d_mean, d_std, d_cov = fitKCA(t, dcumret, 0.01, 0)
    
    hist = data.history([context.SPY], 'price', 120, '1d').dropna(axis=1)
    mmret = np.log1p(hist.pct_change().dropna().values[:, 0])
    mcumret = np.cumsum(mmret, axis=0)
    context.x_mean = mcumret[-1]
    t = np.asarray(range(0, mmret.shape[0]))
    m_mean, m_std, m_cov = fitKCA(t, mcumret, 0.01, 0)
    
    hist = data.history([context.SPY], 'price', 60, '1d')
    smret = np.log1p(hist.pct_change().dropna().values[:, 0])
    scumret = np.cumsum(smret, axis=0)
    t = np.asarray(range(0, scumret.shape[0]))
    s_mean, s_std, s_cov = fitKCA(t, scumret, 0.01, 0)    
    
    shortvel = s_mean[-1, 1]
    midvel = m_mean[-1, 1]
    longvel = d_mean[-1, 1]
    shortacc = s_mean[-1, 2]
    midacc = m_mean[-1, 2]
    longacc = d_mean[-1, 2]
    context.x_mean = m_mean[-1, 0]
    sign = 0
    std = pd.rolling_std(scumret, window=15)[15:]
    
    if shortvel > 0 and shortacc > 0 and midvel > 0 and longvel > 0 and midacc > 0 and longacc > 0 and context.x_mean > 0:
        sign = np.median(std) / std[-1]
    elif shortvel < -0 and shortacc < 0 and midvel < 0 and longvel < 0 and midacc < 0 and longacc < 0 and context.x_mean < 0:
        sign = -np.median(std) / std[-1]
    elif shortvel > 0 and shortacc > 0 and context.SPY in context.portfolio.positions and context.portfolio.positions[context.SPY].amount > 0:
        return
    elif shortvel < 0 and shortacc < 0 and context.SPY in context.portfolio.positions and context.portfolio.positions[context.SPY].amount < 0:
        return

    sign = min(2.0, abs(sign)) * np.sign(sign)
    context.sign = sign
    
    for i, sid in enumerate(hist.columns):
        order_target_percent(sid, sign)
        
def record_vars(context, data):
    record(p=context.cumret, kalman=context.x_mean, sign=context.sign)
       
There was a runtime error.
6 responses

Hi Pravin -

What is Kinetic Component Analysis? And why might it work?

Also, since you are using daily data, could your code be written as a pipeline custom factor (so that it could be combined with other factors, in a single algo)?

Thanks,

Grant

Hi Grant,

Here is the paper. It could be written as a pipeline factor but first I want to get it working with a single asset. Mostly predicting futures.

Best regards,
Pravin

In the source code, he initialises the state transition matrix A to be equal to

A=np.array([[1,h,.5*h**2, 1./6.*h**3],  
                 [0,1,h,h**2],  
                 [0,0,1,h],  
                 [0,0,0,1]])  

Can someone give an explanation as to how is this derived?

In the paper, A is taken to be

A = np.array([[1,h,h**2],  
                 [0,1,h],  
                 [0,0,1]])  

Thanks!

Oh dear, I misread 0.5 to be 5. This is straightforward.

I tried to put this in pipeline. pipeline loads for a long time and then errors out on schedule function with
TimeoutException: Too much time spent in handle_data and/or scheduled functions. 50 second limit exceeded.

Clone Algorithm
22
Loading...
Backtest from to with initial capital
Total Returns
--
Alpha
--
Beta
--
Sharpe
--
Sortino
--
Max Drawdown
--
Benchmark Returns
--
Volatility
--
Returns 1 Month 3 Month 6 Month 12 Month
Alpha 1 Month 3 Month 6 Month 12 Month
Beta 1 Month 3 Month 6 Month 12 Month
Sharpe 1 Month 3 Month 6 Month 12 Month
Sortino 1 Month 3 Month 6 Month 12 Month
Volatility 1 Month 3 Month 6 Month 12 Month
Max Drawdown 1 Month 3 Month 6 Month 12 Month
from quantopian.pipeline import Pipeline
from quantopian.pipeline.data.builtin import USEquityPricing
from quantopian.algorithm import attach_pipeline, pipeline_output
from quantopian.pipeline.filters.morningstar import Q1500US, Q500US
from quantopian.pipeline.classifiers.morningstar import Sector
from quantopian.pipeline.factors.morningstar import MarketCap
from quantopian.pipeline.factors import CustomFactor, AverageDollarVolume

import numpy as np
import pandas as pd
from pykalman import KalmanFilter

class KCA(CustomFactor):
    inputs = [USEquityPricing.close]
    window_length = 100
    outputs = ['velocity', 'acceleration']

    def compute(self, today, assets, out, close):
        
        def fitKCA(t, z, q, fwd=0):
            
            h = 1. / t.shape[0]
            A = np.array([[1, h, .5 * h ** 2, 1. / 6. * h ** 3],
                          [0, 1, h, h ** 2],
                          [0, 0, 1, h],
                          [0, 0, 0, 1]])

            Q = q * np.eye(A.shape[0])

            kf = KalmanFilter(transition_matrices=A, transition_covariance=Q)

            kf = kf.em(z)

            x_mean, x_covar = kf.smooth(z)

            
            for fwd_ in range(fwd):
                x_mean_, x_covar_ = kf.filter_update(filtered_state_mean=x_mean[-1],
                                                     filtered_state_covariance=x_covar[-1])
                x_mean = np.append(x_mean, x_mean_.reshape(1, -1), axis=0)
                x_covar_ = np.expand_dims(x_covar_, axis=0)
                x_covar = np.append(x_covar, x_covar_, axis=0)


            x_std = (x_covar[:, 0, 0] ** .5).reshape(-1, 1)
            for i in range(1, x_covar.shape[1]):
                x_std_ = x_covar[:, i, i] ** .5
                x_std = np.append(x_std, x_std_.reshape(-1, 1), axis=1)
            return x_mean, x_std, x_covar
                

        close[np.isnan(close)] = 0
        close = pd.DataFrame(close)
        sret = np.log1p(np.array(close.pct_change().loc[1:, :].values))
        cumret = np.cumsum(sret, axis=0)
        cumret[np.isnan(cumret)] = 0
        cols = cumret.shape[0]
        t = np.asarray(range(0, cumret.shape[0]))
        velocity = np.zeros(np.shape(t)[0])
        acceleration = np.zeros(np.shape(t)[0])

        for column_index in range(0, cols):
            
            z = np.array(cumret[:, column_index])
            avg, _, _ = fitKCA(t, z, 0.1, fwd=0)
            
            velocity[column_index] = avg[-1, 1]
            acceleration[column_index] = avg[-1, 2]
            
        out.velocity[:] = velocity
        out.accleration[:] = acceleration

        
def initialize(context):

    context.weights = None
    context.output = None
    context.sign = 0
    context.cumret = 0

    attach_pipeline(make_pipeline(context), 'kinetic')

    schedule_function(opentrades, 
                      date_rules.every_day(), 
                      time_rules.market_open(minutes=90))
                    
    schedule_function(record_vars, 
                      date_rules.every_day(), 
                      time_rules.market_close(minutes=1))

def make_pipeline(context):
    
    pipe = Pipeline()
    acc, vel = KCA()
    
    pipe = Pipeline(
        columns={
            'acceleration': acc,
            'velocity': vel,
        },
        screen = Q500US()
    )
    return pipe
    
def opentrades(context, data):
    
    context.output = pipeline_output('kinetic')
    context.security_list = context.output.index
    
    print context.output
    
    
def record_vars(context, data):
    record(leverage=context.account.leverage)
There was a runtime error.

In general, you should call pipeline_output in before_trading_start, as opposed to a scheduled function or handle_data. The before_trading_start function has a 5 minute limit as opposed to scheduled functions and handle_data which have a 50 second limit. There has been some discussion on another thread about ways that might change at some point in the future, but for now the best solution is to call pipeline_output in before_trading_start, save the result in a context variable, and then reference the context variable in other parts of the algorithm.

Let me know if this helps.

Disclaimer

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.