Back to Community
Is this worth deploying capital? (2.5 sharpe)

1 million $ - zero leverage - 150 names. Would like community's feedback if this is worth trading some money?

Loading notebook preview...
Notebook previews are currently unavailable.
26 responses

That's nice. What's the theory or paper behind it?

I'm not really sure anybody can tell you whether you should put your money up or not, that is ultimately up to you. If you are happy with your investment thesis and satisfied with your implementation of it, then you should certainly entertain the idea. After all, deploying capital is the ultimate goal.

If you have not already started paper trading through IB it's a good idea to get that going to see if any bugs present themselves. Something else to consider is the amount of money you would actually deploy vs. what you have been using to test the strategy. If you deploy significantly less money the fees may eat into your profits and your results will differ from your expectations.

On a positive note it looks like the strategy is not very volatile so you probably don't need to worry about a total blow up and your heartache will be minimal if live results don't stack up. The low vol also means It will be easier to tell if the strategy isn't working as intended and you'll be able stop the bleeding in a timely manner.

Unfortunately the only way to know is to live trade the thing, paper trading and backtesting can only tell you so much. That is also where anything that can go wrong will go wrong, you will have to deal with the non/partial fills and other things that only show up in live trading.

Whatever you decide I wish you the best of luck (or skill).

Thanks David. It takes a lot of courage to go from backtesting to live trading. Even a 10K drawdown hits hard. I will paper trade it for a while to see how it goes.

@Minh - using dynamic factor models.

Minh Ngo asks a question I would (probably incorrectly!) ask:

What's the theory behind it?

David Edwards makes some very valid points but again talks of:

happy with your investment thesis

This is where I get totally left behind by both Quantopian an Numerai management. They only look at numbers and statistics. They don't bother with any underlying philosophy. If the numbers add up, they don't care about thetheory...as long as the system makes money in the low vol way they want to operate.

Neither Quantopian or Numerai give a damn about the thesis or theory. All they care about is that it works. Q makes its mind up by paper trading the strategy for a few months, makes sure its alpha and beta is within requirements and that the system trades both long and short.

Numerai takes the output of the predictions you have made on classifying data using machine learning, puts the top predictions on test data (most accurate) to the test in (paper? real?) trading and then uses the top predicting algos until they fail.

The approach in a way is identical. Philosophically at least. Q is blind to the system but likes the results out of sample so employs the algo. N hasn't a clue what ML algo is giving predictions on the secret data it has supplied but likes those predictions and uses them until they fail. Although the blindness is greater for Numerai participants - they have absolutely no clue what the test/ training data represents so they can't actually use their own algos except to lease them to N.

At least with Q, the punter knows the algos AND the data and can use them for himself if he so chooses. Q is therefore acting more blind that N? Or do I mean the N algo writer is more blind than the Q algo writer? Or both.

Anyway,

Is this worth deploying capital?

Ask Q! I would be interested in their thoughts. After all their entire hedge fund model is based on the replies they give.

I am fascinated. Because Amateur Quant is asking the questions both Q and N must ask themselves. And because I'm not sure I actually give a damn about the theory either. Really. Why? Because it is only theory. Because it is un-testable anyway. No one agrees whether markets are random or follow trends or patters. No one agrees on the EMH.

All this stuff (my own stuff included) works until it doesn't. So yes, I rather like the Q and the N approaches.

am i reading leverage correctly? gross leverage appears to creep up over time

Thanks James. You caught a bug in the algorithm. Let me fix the leverage.

Hi Anthony,

I guess I will only find out by entering this into the contest and see what happens. The theory is simple. Use dynamic factor models to predict returns and compute fair price of a stock. Do mean reversion if current stock price is above/under the fair price.

Pravin - That's a very interesting tear sheet. When did you finalize the algo? I'd be very interested in seeing how much of that performance is in-sample versus out-of-sample. It's had a very hard last 16 months. Presumably any out-of-sample data includes that bad period, and before I'd invest my money, I'd want to understand why that was true.

Anthony - I'd like to correct some of your thoughts about how Quantopian looks at algorithms for allocations.

You said that "Neither Quantopian or Numerai give a damn about the thesis or theory. All they care about is that it works." For Quantopian, this is quite incorrect. We care mightily about the theory behind your algorithm. One of the places we talk what we want is in on the allocation page, in the "strategic intent" section:

We are looking for algorithms that are driven by underlying economic reasoning. Examples of economic reasoning could be a supply and demand imbalance, a mixture of technical and fundamental drivers of stock price or a structural mispricing between an index ETF and its constituent stocks.  

We use the numeric/statistic criteria to sort through algorithms and find interesting ones. Once we've gotten to the interesting ones, we start talking to the author. One of the key aspects of that conversation is to understand the investment thesis that the algorithm is built on. That thesis is vital to our allocation decision.

I recognize that this thesis-driven part of the process is less visible to the Quantopian community. The Quantopian Open and Pyfolio's tearsheets expose the numeric/statistic aspects of our work. The conversations about the investment thesis is done more privately. However, the conversation around theses does happen publicly here every day here on the forums, and in places like the CIO blog (I've read a draft of his next post, and it's fascinating!).

I hope this clarifies your understanding of how we make allocation selections.

Disclaimer

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.

Thanks Dan. I developed this algo last night and haven't paper traded it yet. I have resolved the draw downs in last 16 months and have entered it into contest. The algo now has a 2 year back test sharpe of 2.66 from 2014. It is still working progress and I am working on exit strategy to book profits early in cycle to see if it further improves performance.

Once we've gotten to the interesting ones, we start talking to the author. One of the key aspects of that conversation is to understand the investment thesis that the algorithm is built on.

Ah! My apologies. You don't ask for the code but you do ask for the theory.

Perhaps this is the way Q is differentiated from Numerai.

Either way this was no criticism. And as I have already said, to be honest I am not sure the theory counts for much if anything. But that is perhaps my own jaundiced view. If markets truly are a chaotic system and unpredictable then perhaps no amount of maths or even economic theory is going to help much.

As you can tell, I am truly interested in these matters and not trying to be flippant.

We don't have any answers but it is always correct to ask the question.

3.56 Sharpe from 2004 till date. Simple algo based on covariances and regression.

Loading notebook preview...
Notebook previews are currently unavailable.

Dan
In which case, a follow up question if I may. We have no idea what Numerai data represents but we use it to make predictions with machine learning. Which will usually contain a small random element (in setting the initial weights for instance).

ML attempts for find patterns or correlations in the data. Hopefully the latter are causal - if one of the inputs to the ML is GDP or corporate earnings growth for instance. Or perhaps, as mentioned in another post here on Q, the ML algo uses JUST price. In which case it is simply searching for patterns - trends perhaps, or mean reversion.

There have been a number of oblique references to ML by Thomas Wicki and others and even some systems or at least notebooks posted.

What is Q's attitude to ML? Do you still look for an underlying REASON? (Earnings growth or debt levels as an input for instance as well as price) Or are you content to go with an algo that JUST uses price (hence you don't really know what it has found or why).

Thanks David. It takes a lot of courage to go from backtesting to live trading. Even a 10K drawdown hits hard. I will paper trade it for a while to see how it goes.

I would echo this, but I think that this is really the only way you can be honest with yourself. It's easy to cut corners when backtesting, iterating too much, revising parameters based on your a priori knowledge of the markets (which are tainted by historical data), including or excluding specific assets with no rigorous reason.

But, when you put some fraction of your life's savings on the line, when you push the live trading button, you know whether or not you've been intellectually honest about your strategy development process, and whether or not you've been cheating. If you have done the best you can to produce a strategy that has every chance to succeed in the future, then courage is all you need. Luck is bonus. If you get lucky, that's great. If you get unlucky, that sucks, but you persevere.

If you've cheated, and you secretly know that your backtests were all tainted by twiddling, then you NEED luck. If you end up getting lucky, then you're doubly unlucky because you'll get whacked some time in the future when you're not expecting it. The best case is that you get unlucky right away, and shut it down without losing TOO much money.

when you push the live trading button, you know whether or not you've been intellectually honest about your strategy development process, and whether or not you've been cheating.  

This is the absolute test. That is why most funds require a live track record and don't consider back tests or paper trading results.

On another note, where I am from, I get 9% returns (post tax) on government bonds. I don't see any reason to put personal savings in stocks and strategies that fetch around 10-12% returns for the extra risk :).

Nice, what are the net returns of those bonds in US dollars?

They are not available to everyone. Only Indian citizens but residing outside India can avail them. The idea is that there is more capital inflow into country from outside.

I am just curious how much of the return of these bonds is offset by the depreciation of the rupee.

USDINR went from 50 to 67 in past 5 years. That is a return of 34%.
FD for 5 years returns 54%.
So net is 20% in 5 years.

The theory is simple. Use dynamic factor models to predict returns and compute fair price of a stock. Do mean reversion if current stock price is above/under the fair price.

Any more details to share? Is there a "sweet spot" time scale for mean reversion? Also, I'm curious if you are using pipeline, and in what fashion? Minute bar data? Any smoothing? What are the strengths and weaknesses of the Q platform in researching and executing the strategy?

Regarding Quantopian's interest in the "strategic intent" of candidate algos, there are a number of angles on this topic. Firstly, from https://www.quantopian.com/fund, we have:

Strategic Intent
We are looking for algorithms that are driven by underlying economic reasoning. Examples of economic reasoning could be a supply and demand imbalance, a mixture of technical and fundamental drivers of stock price or a structural mispricing between an index ETF and its constituent stocks.

You will always own your intellectual property.

With rare exceptions described in our Terms of Use, we don't look at your algorithm code.

So, I think Q is saying, if you want the best shot at getting funded, you should be prepared to reveal details about your algo (and although it is not mentioned explicitly, share the code, if you wish). Let's see...I'm sitting across the table from a guy with a briefcase with $10M in it. Should I tell him what my algo does or not? Hmm...maybe I should just give him the code? Of the 10 or so algos that have been funded to-date, it would be interesting to know the kind of information that was shared by the algo owners (and if anyone got funded without sharing anything). Presumably the licensing does not require sharing of any information, but it does enter into the deal. I don't think this is clear to all users, as illustrated by Anthony's comments above. Although Q could deal in "black box" algos, my read is that they'd prefer not to.

Attempting to divine the "strategic intent" of an algo by talking with its author is a dangerously qualitative assessment, and potentially falls prey to Q bias. It is hard to articulate, but I think there is a risk inherent in not treating algos as "black box" since there is the potential for individuals in power within Q/Point72 to make bad decisions, based on words not numbers. I wonder if Q understood this risk and considered if they should be asking for "strategic intent" from authors in the first place? Why did they decide not to go the full "black box" route?

We use the numeric/statistic criteria to sort through algorithms and find interesting ones. Once we've gotten to the interesting ones, we start talking to the author. One of the key aspects of that conversation is to understand the investment thesis that the algorithm is built on. That thesis is vital to our allocation decision.

This process is not fully automatable and scalable. With a crowd of 90,000 users, it won't work. My concern here is that Q hasn't put enough thought and effort into how a crowd-sourced fund could work (or maybe already proved it?), and have mapped their business onto the established one of Point72 (e.g. see https://www.point72.com/careers/). This is a logical business choice, but not necessarily an innovative, blue-sky approach. The risk is that the new thing with potentially a new reward is never tried. Or maybe the automation and scalability are in the works? Is it possible to fund a decent fraction of 90,000 algo authors?

Any more details to share? Is there a "sweet spot" time scale for mean reversion? Also, I'm curious if you are using pipeline, and in what fashion? Minute bar data? Any smoothing? What are the strengths and weaknesses of the Q platform in researching and executing the strategy?  

It reverts in 2-3 days.
I am not using pipeline yet but intend to use it to filter tradeable 500 stocks.
Daily data.
I use some techniques not smoothing.
I use the notebook to research and algorithm to backtest. Quantopian's platform is immensely useful and saves tons of time and effort. However, I don't understand their "daily positions and gains" in algorithm backtest details. I wish it were simpler. Also there should be a better way to analyze trade and gains so that we can identify reasons for drawdown. At the moment, I find it very difficult to narrow down the reasons for a drawdown. I want to know everything about the algorithm. Why it works when it does and why it fails but that analysis is difficult to do on Quantopian because they don't allow you to download any transaction or position data. I would like to map my signals with the transactions and bets and see when signals have a positive alpha.

@Anthony: As you know, there's different schools of thought. Coming up with a good hypothesis for market inefficiencies that is a result of market understanding, evaluating that hypothesis, deploying if it works, discarding if it doesn't; intuitively that has a lot of appeal, it's basically the scientific approach. However, even in science things are rarely that principled. Humans are easily fooled by randomness and see patterns where there are none and then very good at coming up with hypotheses about the causal relationships behind these patterns. The smarter someone is, the easier it is for them to come up with these hypotheses.

Based on that you could argue that I shouldn't trust my monkey-brain which might not be smart enough in the first place to understand how markets with millions of participants (including governments with interventions like QE) work. I could then just look at what the data says and try to validate the patterns that emerge in a statistically rigorous way.

A middle-ground might be provided by the factor-based approach where you have many individual factors that all track some economically rational thing (quality, value, momentum etc). Each factor might only carry weak predictive power so you have to combine them in potentially non-linear ways where the weighting could also change over time. ML is a tool to figure out that weighting in an automated fashion that only looks at the data. Doesn't this have the risk of overfitting even if you do proper cross-validation? Most likely, but so might the hypotheses you form by looking at patterns in the market.

Which one is the right philosophy? I don't know. Can they all have different up and downsides? That seems plausible to me. The only true test is OOS.
Note that those are just my current, personal views on the matter. Happy to hear yours.

Disclaimer

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.

Thomas

I share ALL your sentiments. Including the admission that I simply do not know the answers. From hereon, I intend to try to defeat my bias' and to take a multiple approach in much the way you suggest.

The only real belief I have left in investing is the necessity for diversification. Of every conceivable kind.

Hi Thomas,

Just curious if you've started to play around with ML and the 10 or more algos now funded by Q? Presumably the terms of the licensing allow Q to run backtests, etc., mixing and matching the strategies. And you should be getting out-of-sample data now, too. Does ML actually work in this case? How do you avoid over-fitting?

Also, I saw that this weak, transitory multi-alpha concept is part of the workflow presented on https://www.quantopian.com/posts/alphalens-a-new-tool-for-analyzing-alpha-factors. Is Q working on tools that would allow individual users to combine multiple algos with ML, or is the idea that all of the mini-alphas would need to be combined into one algo? Or something else? Maybe combining alphas would be what you'd do at the fund level? Just wondering how this might play out (perhaps a topic for a separate thread).

Grant

Hi Grant,

I don't think of this workflow in regards to individual algos, but rather individual factors/signals/alphas that can be combined with ML to one mega-alpha. But you're right, you could also imagine each algorithm being a factor and then combining them in a clever way across strategies.

Thomas

Well, maybe the term "alpha" is being used more generally. In my mind, one doesn't end up with an alpha & beta until an algo is written and backtested. So, one would write a bunch of algos and then combine them. In the workflow, I see now that you are talking about "Alpha Factor Combination" which I suppose is more computationally tractable and amenable to ML which requires lots of iterations, I gather. But it still seems like you need something that approximates a backtest to estimate alpha & beta point-in-time, so you kinda end up in the same place, needing to run something like an algo. Maybe effectively it ends up being a vectorized backtesting approach? Seems like it would need to be, or you wouldn't be able to apply the ML in a cost-effective manner, across 80,000+ users?

The alpha combination step would need to be done dynamically within an algo, I would think? In other words, I'd find my N factors that have transient predictive power. Then, within an algo, periodically, I would sort out how to combine them optimally. But for the ML to work, wouldn't one need to call an objective function? Would alphalens be called? Why not just run repeated backtests over the factors to do the optimization? Or is the alpha combination step expected to be static (i.e. I'd come up with some way of combining the factors, which potentially could be dynamic versus fixed weights, but it would be a computationally "light" approach).

Just a few thoughts...

Admittedly, I haven't quite digested the alphalens approach. Gotta have another look, at some point, to see if I can understand what it does. I see there is a returns analysis section, so maybe it is kinda like a back-of-the-envelope backtest?