Back to Community
multi-factor example

I welcome feedback (will post tear sheet next).

Clone Algorithm
101
Loading...
Backtest from to with initial capital
Total Returns
--
Alpha
--
Beta
--
Sharpe
--
Sortino
--
Max Drawdown
--
Benchmark Returns
--
Volatility
--
Returns 1 Month 3 Month 6 Month 12 Month
Alpha 1 Month 3 Month 6 Month 12 Month
Beta 1 Month 3 Month 6 Month 12 Month
Sharpe 1 Month 3 Month 6 Month 12 Month
Sortino 1 Month 3 Month 6 Month 12 Month
Volatility 1 Month 3 Month 6 Month 12 Month
Max Drawdown 1 Month 3 Month 6 Month 12 Month
# Backtest ID: 5a0f2f288b13eb440b93a10a
There was a runtime error.
18 responses

Here's the performance tear sheet and factor attribution analysis.

Loading notebook preview...
Notebook previews are currently unavailable.

Can someone read the tea leaves for me?

First off, there are a number of things I don't understand about how Q is approach "risk management" but I gather that it is a standard function in hedge fund construction and management, and this is their version of it. It seems that "risk" is anything that is not wanted in a particular fund in excess. The Q risk model somehow flows out of how the 1337 Street Fund is being constructed and managed, as a cobbled together fund of funds (licensed user algos). I guess the whole system is easier to manage, including compensating individual authors, if risk is managed at the algo level. Then licensed algos can be treated as a clean set of unexplained alphas, as input to the framework (and the more the 1337 Street Fund can be constructed of "new alpha" the more valuable it will be as a financial product). So, I guess the idea here is that authors need to deliver pure new alpha (as defined by the Q risk model). In the lingo of ML, authors will be paid for delivering new features (I have to think that ML will eventually play a role in the 1337 Street Fund construction, if it isn't already).

@ Karl - completely removing the Pipeline volatility factor reduces the SR from ~1 to ~0.8, over a longer backtest (back to 1/1/2010). Looking at the summary statistics, I can't tell if the algo is "good enough" or if I need to do more. It seems like the Specific stats are already pretty high, relative to the Common ones, but maybe there are other problems (the biggest, perhaps, is that I have no Strategic Intent explanation for the source of the specific returns, so the whole thing may be hopeless, until I'm a quant expert and know what I'm doing). I guess I could fiddle around trying to get the Common Returns curve to center on zero returns, but it might just be another form of over-fitting, I'm thinking, and not move the algo closer to being allocate-able.

Summary Statistics
Annualized Specific Return 4.83%
Annualized Common Return -0.59%
Annualized Total Return 4.22%
Specific Sharpe Ratio 2.25

@ Karl -

A brief note that there is a lot of commonality between this risk model/factor attribution jazz and the response surface methodology (dating back to 1951, by the way--nothing new under the sun). One thing I haven't quite sorted out is the potential role for so-called "interaction terms" in the context of Q alpha factor combination and the risk model (and risk mitigation techniques). If you and anyone has insight, I'd be interested. In the end, I think the interaction terms will need to be included (computationally, this is no big deal, if I'm thinking about it correctly).

Grant

@Grant The factor attribution tearsheet looks great. One thing I would try to improve is the 47.6% daily turnover rate. It's probably a little high for a 300 positions portfolio. Thanks for sharing the actual code!

@ Charles -

Thanks. No problem sharing the code--it is more of a template to improve and dink around with. I don't think there is anything magical in there.

Here's a tweaked version. Will post sheets next.

Clone Algorithm
101
Loading...
Backtest from to with initial capital
Total Returns
--
Alpha
--
Beta
--
Sharpe
--
Sortino
--
Max Drawdown
--
Benchmark Returns
--
Volatility
--
Returns 1 Month 3 Month 6 Month 12 Month
Alpha 1 Month 3 Month 6 Month 12 Month
Beta 1 Month 3 Month 6 Month 12 Month
Sharpe 1 Month 3 Month 6 Month 12 Month
Sortino 1 Month 3 Month 6 Month 12 Month
Volatility 1 Month 3 Month 6 Month 12 Month
Max Drawdown 1 Month 3 Month 6 Month 12 Month
# Backtest ID: 5a10aabd4ce931411f988252
There was a runtime error.

The sheets.

Loading notebook preview...
Notebook previews are currently unavailable.

Not sure what it all means, but my guess is that it is not suitable for the Q fund. Reading tea leaves to me, without a frame of reference.

That first version you posted, Grant, was pretty interesting. It has a persistent short-term mean reversal exposure, and also has a persistent underexposure to volatility (the portfolio tends to prefer low-volatility stocks). On one hand, the exposures aren't really that high, .2-.3 or so. On the other hand, does further analysis explain why those risk exposures are present? Is there a way to keep your overall returns while managing out that that risk exposure? (We are getting close to releasing a way for you to do more management in your algorithm code - it will soon be easier to answer the question).

The later version you posted had higher exposure to those style factors, more like .6-.7. That is a step backwards.

Another over-arching concern: do you have a way of keeping yourself from overfitting? I'd consider only running your tests over part of the history, and keep the last year or so hidden from yourself. Once you get close to a final product, then run it over a longer time frame, and see if you managed to avoid the overfitting trap. As it is, that first algo is interesting, but we won't know how interesting until many months have passed.

Disclaimer

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.

Here's another one for kicking the tires of the risk model. In this case I multiply the mean reversion and volatility factor together and end up with negligible risk factor exposure (I think...)

Will post sheets next.

Clone Algorithm
112
Loading...
Backtest from to with initial capital
Total Returns
--
Alpha
--
Beta
--
Sharpe
--
Sortino
--
Max Drawdown
--
Benchmark Returns
--
Volatility
--
Returns 1 Month 3 Month 6 Month 12 Month
Alpha 1 Month 3 Month 6 Month 12 Month
Beta 1 Month 3 Month 6 Month 12 Month
Sharpe 1 Month 3 Month 6 Month 12 Month
Sortino 1 Month 3 Month 6 Month 12 Month
Volatility 1 Month 3 Month 6 Month 12 Month
Max Drawdown 1 Month 3 Month 6 Month 12 Month
# Backtest ID: 5a1374c4b096ec4104654e0c
There was a runtime error.

Here are the sheets. Not sure what it all means, but lots of numbers and charts to ponder...

Loading notebook preview...
Notebook previews are currently unavailable.

Hi Dan,

Thanks for the feedback. Feel free to put a reminder on your calendar to check the backtest above (Backtest ID: 5a0f2f288b13eb440b93a10a) in six months and post the allocation suitability analysis here. I'd be curious how things play out. Since it is in the public domain, I'm guessing it won't be allocatable, but it would be an interesting exercise, nonetheless.

From first algo. Where are the 0.0033 weights decided? Bringing them down, will be fewer partial fills also. I typically look for rules that may reduce the magnitude of those at the bottom, PnL, second column here.

2017-11-16 13:00 _ti:1154 INFO  726 trading days     2 hr 33.3 min   2017-11-27 07:12 US/Pacific  
    Portfolio: 11265327              2015-01-02 to 2017-11-16                      Alpha:     0.05  
 Initial Cash: 10000000                     Buys:   715529 (2952944 partial)        Beta:    -0.06  
  Unused Cash:  9452437                    Sells:   613683 (2886456 partial)      Sharpe:     1.37  
   Max Margin:        0              Commissions:    69656                      Drawdown:     2.88  
     Max Risk:  5926339 (59%)         Shares Now:   211077                     Stability:    0.295  
  Cash Profit:  1273782               Shares Val:    -8455                       Sortino:     2.18  
 Total Profit:  1265327                 Cash Now: 11273782                  Shrt/Lng Now:     1.00  
     Q Return:   12.65% Profit/Init     Max Lvrg:     1.04  2015-10-05        Max Shorts: -5926339  
   PvR Return:   21.35% Profit/Risk    PvR %/day:     0.03                    Max  Longs:  6020397  
2017-11-16 13:00 ti:1326 INFO .  
2017-11-16 13:00 _ti:1147 INFO   Sort column 1             Positions: 302 of 591 traded  
         Profit   Max    Return  Return   Buy     Price    Buy|Sell     Max      Shares   Shares  
 Symbol     PnL  Risked      %    %/day   Hold   Init|Now   Orders    Shrt|Lng     Now       %  
    CVE   48654  86283     56.4   0.12    -0.5    20|10    171|169   -3651|3746      0         0  
   CNHI   38067  76956     49.5   0.15     0.5     8|13    123|128   -6038|6139   3011    0.0034  
    TXN   36133  73574     49.1   0.11     0.8    54|98    162|152    -720|777       0         0  
    VRX   35488  72400     49.0   0.16    -0.9    144|14   107|106   -1270|1132      0         0  
    NWL   32520  69751     46.6   0.12    -0.3    42|28    157|144   -1259|1364      0         0  
    UHS   31091  68168     45.6   0.13    -0.1    112|98   134|125    -338|359       0         0  
    STJ   31033  65442     47.4   0.14    -1.0     65|0     101|99    -672|709       0         0  
2017-11-16 13:00 _ti:1147 INFO _  
    APA   30863  73196     42.2   0.07    -0.3    62|40    218|215   -1070|967     922    0.0033  
    IMO   29643  67994     43.6   0.08    -0.3    43|31    233|202   -1338|1356   1208    0.0033  
    STX   28686  65797     43.6   0.12    -0.4    67|39    131|119   -1238|940       0         0  
    PXD   27995  71234     39.3   0.06     0.0   148|151   216|192    -331|336     250    0.0034  
    NOV   27799  64875     42.8   0.07    -0.5    63|32    208|197   -1334|1349      0         0  
                       ... 579 midrange rows omitted, x_high_low is 12 ...  
    FCX  -25188  70477    -35.7  -0.06    -0.4    23|14    208|209   -4964|3858   2763    0.0033  
   ATVI  -27336  60768    -45.0  -0.08     2.2    20|64    192|142   -1769|1786   -593   -0.0034  
    HPQ  -29768  71009    -41.9  -0.08    -0.5    40|22    209|170   -3826|3739      0         0  
   AMZN  -30596  52551    -58.2  -0.09     2.6   314|1137  148|109    -114|89      -33   -0.0033  
2017-11-16 13:00 _ti:1154 INFO _  
   ADSK  -32982  70401    -46.8   -0.1     1.1    60|127   216|165    -843|827    -301   -0.0034  
     FB  -33275  60374    -55.1  -0.08     1.3    79|180    126|90    -444|264    -211   -0.0034  
   INCY  -34622  78214    -44.3  -0.06     0.4    74|106   239|198    -641|585    -352   -0.0033  
    ABX  -36539  82460    -44.3  -0.08     0.3    10|14    183|162   -4486|3627  -2711   -0.0033  
   PYPL  -38863  63350    -61.3   -0.2     0.9    41|78     102|71   -1022|1052   -496   -0.0034  
    WDC  -45740  85111    -53.7  -0.09    -0.2    111|91   211|174    -919|919    -415   -0.0034  
   NFLX  -59231  89249    -66.4  -0.09    -0.4   348|196   190|137    -429|387    -191   -0.0033  
   NVDA  -78045  57419   -136.0  -0.23     9.8    20|212   209|140   -1705|1730   -176   -0.0033  

Info is a from a tool I wrote. If you're the hero with means who will make it open source to the community, ping.

Blue Seahawk -

Thanks. The .0033 comes from:

MAX_LONG_POSITION_SIZE = 2*1.0/NUM_TOTAL_POSITIONS  
MAX_SHORT_POSITION_SIZE = MAX_LONG_POSITION_SIZE  

NUM_TOTAL_POSITIONS = 600

You can clone the algo and play around with it. Either adjust NUM_TOTAL_POSITIONS or the multiplier (presently set to 2) or both.

I didn't have time to read past a few posts up, but just wanted to say that multiplying factors falls into the category of 'non-linear' combinations and is super interesting. Obviously you want to have some economic rationale when you do this for real (I get this is an example to kick the tires), but wanted to note that using non-linear combinations of factors is a real thing. What people will do sometimes is look at 'corners' in factor matrices. For instance if you have factors A and B, you can look at stocks where A is high and B is low, and the 3 other corners. There can be interesting anomalies hidden there from what pro quants have told me.

Disclaimer

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.

Thanks Delaney -

Well, it is just the idea that if one has multiple factors, a weighted sum of factors is just the simplest case of expressing the alpha as a multivariate polynomial of factors. Including the lowest-order interaction terms, the form is:

alpha = b0 + b1*alpha1 + b2*alpha2 + b12*alpha1*alpha2  

This can be generalized to any number of factors.

Regarding the economic rationale jazz, it probably makes sense that the individual factors have some basis, but requiring a story for their interactions may be a stretch.

One thought is to determine the statistical significance of the b-coefficients on a rolling basis (akin to how one would approach the problem in design of experiments/response surface methodology).

About the numbers above: I had added my code with a paste to the cloned (first) algo, ran it, and at the end of the 3 hour backtest I copied the last output and pasted it above. Can be set to produce that output at any interval or condition for watching progress or examining downturns etc. The second column, PnL is all-time, not just the current position. A few unauthorized tea leaf readings: If you take a look at the last line, NVDA, you'll see your algorithm has a position that is currently short and an outlier in Return % at -136, and lowest in PnL. It's price went from 20 to 212 since first seen but they were both long and short.

Generally eleven of the twelve worst losers shown are still being held, that's unusual. Changes to factors can have big effects on profiles like that. I've also had some benefit from gathering current PnL's and dropping the lowest periodically, can use idxmin() on a Series as one way. Can also add a stock to a list fed to Optimze CannotHold() as another. Could even try a flip of long, short. The 12 highs and 12 lows shown could be changed, doubling it or something to find out if the number of losing stocks still being held might be even more dramatic than just those 11. Partials: Approaching 3 million partial buys and partial sells for a backtest of this length looks high to me. Couple of things that I find beneficial, a mask for top minimum volumes for some window length and/or canceling orders after a certain amount of time or after a certain number of partial fills (can treat positive and negative PnL stocks differently).

If this wraps you could copy to an editor. It's fine in the backtester.
A few partial fills:

Datetime           Trading Minute     Action    Filled|Prv|Order  Sym   Prc   Shrs_Now  Stop|Limit      PnL  Lvrg   oId   Cash  
2016-07-05 07:00 _t:620 INFO   30  Buy                      1560 NVDA  46.61    -774                    -22  0.99  d042 10946288  
2016-07-05 07:01 _t:620 INFO   31    Bot              689|0|1560 NVDA  46.59     -85                     -1  1.01  d042 10974839  
2016-07-05 07:02 _t:620 INFO   32    Bot            401|689|1560 NVDA  46.63     316                     -1  1.01  d042 10897848  
2016-07-05 07:03 _t:620 INFO   33    Bot       all 470|1090|1560 NVDA  46.61     786                     -9  1.02  d042 10899336  
2016-07-06 07:00 _t:620 INFO   30  Sell                    -1560 NVDA  47.28     786                    518  1.00  1d13 10960303  
2016-07-06 07:01 _t:620 INFO   31    Sold          -1180|0|-1560 NVDA  47.27    -394                     -2  0.97  1d13 11181913  
2016-07-06 07:02 _t:620 INFO   32    Sold   all -380|-1180|-1560 NVDA  47.47    -774                    -81  0.97  1d13 10959183  
2016-07-07 07:00 _t:620 INFO   30  Buy                        12 NVDA  48.08    -774                   -554  1.02  d77a 10962642  
2016-07-07 07:01 _t:620 INFO   31    Bot                      12 NVDA  48.06    -762                   -530  0.99  d77a 10857137  
2016-07-08 07:00 _t:620 INFO   30  Buy                        28 NVDA  49.94    -762                  -1963  1.01  48ed 10984581  
2016-07-08 07:01 _t:620 INFO   31    Bot                      28 NVDA  49.90    -734                  -1861  1.00  48ed 11146479  
2016-07-11 07:00 _t:620 INFO   30  Buy                        31 NVDA  52.06    -734                  -3447  1.01  3883 10998615  
2016-07-11 07:01 _t:620 INFO   31    Bot                      31 NVDA  52.09    -703                  -3326  0.99  3883 10925876  
2016-07-13 07:00 _t:620 INFO   30  Buy                        14 NVDA  52.99    -703                  -3955  1.00  8711 10949374  
2016-07-13 07:01 _t:620 INFO   31    Bot                      14 NVDA  52.97    -689                  -3863  1.00  8711 10791296  
2016-07-19 07:00 _t:620 INFO   30  Buy                       689 NVDA  53.09    -689                  -3942  1.00  3b25 10953302  
2016-07-19 07:01 _t:620 INFO   31    Bot               238|0|689 NVDA  53.09    -451                  -2585  0.97  3b25 10820012  
2016-07-19 07:02 _t:620 INFO   32    Bot             325|238|689 NVDA  53.10    -126                   -723  0.97  3b25 10920719  

With NVDA, it was not affected by partial fills as often as some others. More often was like this, complete buying and selling at lower volumes, both long and short.

2017-09-14 07:00 _t:620 INFO   30  Buy                         4 NVDA 169.80    -225                 -11908  1.00  5762 11273249  
2017-09-14 07:01 _t:620 INFO   31    Bot                       4 NVDA 169.67    -221                 -11668  1.00  5762 11204744  
2017-09-15 07:00 _t:620 INFO   30  Buy                         7 NVDA 175.90    -221                 -13045  1.00  439e 11244305  
2017-09-15 07:01 _t:620 INFO   31    Bot                       7 NVDA 176.22    -214                 -12700  1.00  439e 11308007  
2017-09-18 07:00 _t:620 INFO   30  Buy                        12 NVDA 185.60    -214                 -14707  1.01  f16d 11261151  
2017-09-18 07:01 _t:620 INFO   31    Bot                      12 NVDA 185.53    -202                 -13868  1.01  f16d 11031559  
2017-09-21 07:00 _t:620 INFO   30  Sell                       -5 NVDA 181.15    -202                 -12984  1.00  7821 11247529  
2017-09-21 07:01 _t:620 INFO   31    Sold                     -5 NVDA 180.79    -207                 -12911  0.98  7821 11237903  
2017-09-25 07:00 _t:620 INFO   30  Sell                       -8 NVDA 174.01    -207                 -11507  1.00  d1a6 11240518  
2017-09-25 07:01 _t:620 INFO   31    Sold                     -8 NVDA 174.34    -215                 -11575  1.03  d1a6 11352925  
2017-09-26 07:00 _t:620 INFO   30  Buy                         4 NVDA 176.93    -215                 -12133  1.00  e677 11207084  
2017-09-26 07:01 _t:620 INFO   31    Bot                       4 NVDA 176.86    -211                 -11892  0.98  e677 11225226  
2017-09-27 07:00 _t:620 INFO   30  Sell                       -4 NVDA 173.60    -211                 -11204  1.00  83d4 11207135  
2017-09-27 07:01 _t:620 INFO   31    Sold                     -4 NVDA 173.87    -215                 -11261  0.98  83d4 10988039  
2017-09-29 07:00 _t:620 INFO   30  Buy                         6 NVDA 178.37    -215                 -12229  1.00  1523 11188759  
2017-09-29 07:01 _t:620 INFO   31    Bot                       6 NVDA 178.75    -209                 -11967  1.00  1523 11068840  
2017-10-09 07:00 _t:620 INFO   30  Buy                         7 NVDA 184.68    -209                 -13205  1.00  145a 11172453  
2017-10-09 07:01 _t:620 INFO   31    Bot                       7 NVDA 185.07    -202                 -12843  1.00  145a 10993256  
2017-10-10 07:00 _t:620 INFO   30  Buy                         6 NVDA 190.44    -202                 -13928  1.00  928c 11176835  
2017-10-10 07:01 _t:620 INFO   31    Bot                       6 NVDA 190.05    -196                 -13437  1.01  928c 11037683  
2017-10-16 07:00 _t:620 INFO   30  Buy                         5 NVDA 195.18    -196                 -14443  1.00  d262 11183832  
2017-10-16 07:01 _t:620 INFO   31    Bot                       5 NVDA 195.34    -191                 -14105  1.00  d262 11378467  
2017-10-30 07:00 _t:620 INFO   30  Buy                         8 NVDA 205.18    -191                 -15984  1.00  08e8 11246975  
2017-10-30 07:01 _t:620 INFO   31    Bot                       8 NVDA 205.50    -183                 -15373  1.00  08e8 11339853