Grant makes some really good points. One similar one I was going to make is that in general you want to combine multiple weaker models into one stronger model. So whereas each model may fail transaction costs on its own, after combining them the resulting model will be better than the sum of the parts and possibly do fine after transaction costs and such.
In response to backtesting, generally backtesting should be the final test done in a series of many rigorous statistical tests. Namely, some consider it a way to develop algorithms, but for concerns that Lucas raises (mainly overfitting) one generally wants to spend most of one's time in research and then backtest only when you've convinced yourself that one's model (often combined alpha factor) works. The purposes of the backtest are mainly as follows:
- See if your model survives real world liquidity/turnover constraints.
- See if your model survives real world market impact (slippage) constraints.
- See how much these constraints eat into your returns.
- See how much transaction costs eat into your returns.
- Estimate capacity range.
Basically I'm not sure if I can think of a specific case where I would try to run a backtest with 0 commissions, but certainly I would probably do a parameter sweep across a range of assumptions for commissions, slippage, liquidity, and capital to get a sense of how robust the model is to different possible market and execution conditions as well as see which parameters cause the biggest drop-off in returns. For instance, you might find your high turnover model is generally okay under different liquidity assumptions, as it trades very liquid stocks, but transaction costs will quickly change the estimated returns. Or you might find that a model which is okay under one set of assumed parameters quickly breaks under a small tweak to one of the conditions, indicating fragility.
Lastly you want to check that your model will work under the execution conditions of the brokerage in which you plan to execute. Every brokerage has different parameters as discussed above, so it's a consideration when developing and deploying models.
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.