Back to Community
Paper Trading - Looking For Feedback

Hey everyone,

We want to hear from you about a change we are considering making to the Q platform. In short, we are considering removing paper trading, because it doesn't align with other features on the platform like it used to. More specifically, we want to provide data and features to help the whole community research and develop more contest and fund algorithms. Paper trading doesn't allow you to use datasets with a holdout (rendering it less useful), it doesn't align with our investment team's new approach to running the Q fund, and it is a costly feature to maintain (focus, money, and engineering time). We would especially like to reallocate that effort to helping you find and develop better algorithms.


History of Paper Trading

A couple of years ago, we removed live trading from Quantopian. It was an extremely difficult decision that impacted many of our community members. Our reason for removing live trading was that we wanted to focus on our investment business, and we noticed that algos for live trading split that focus both in our community and internally. Taking that one step further, we launched a new contest (the current one!) to align better with our investing focus. With two years of out-of-sample data on that decision, we feel like we made the right choice. As painful as it was at the time, the Q community has stepped up to the challenge of the new contest, used new datasets as they get added to the platform, and collaborated on ideas and best practices for developing a contest or fund algo (example 1, example 2, etc.).

At the time of removing live trading, we decided to keep paper trading. There were a couple of reasons to keep it, including easing the transition for folks who were using Q to do live trading as well as the fact that we depended on paper trading to test algorithms before running them in the Q fund. Today, we don't have the same live trading community that we used to and we no longer depend on paper trading to test fund algos]. Additionally, the FactSet datasets that we've been adding to the platform have the most recent year or two of data held out, which means they can't be used in paper trading. At this point, we feel that paper trading is not a feature that supports the research and development of contest & fund algorithms, so we are looking into removing it by the end of the year**.

** Note that we haven't committed to a specific date yet.


That said, before making a decision one way or the other, we'd like to learn more from you about how you use paper trading. We're curious about how you use it generally and also specifically if you think losing the feature would have an impact on your research and development process for contest & fund algorithms.

Let us know!

Disclaimer

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.

43 responses

Jamie,

Paper trading is the best out of sample test of the strategy you developing.
I 've been using it for over four years.
I only keep there those who meet my expectations, the best of the best algo.
Now there are about 20, the oldest have run for more than 3 years.
This is very important to me Quantopian tool.
Lately I prefer to watch the best of the best end of day the in reaserch notebooks.
Removing paper trading wil be a big hole for my ability to analyze intraday breath indicators
as ADV, DECL, UNCH, UPVOL, DNVOL, XVOL, TRIN, BRIN...and their inpact on next day market movement.
There is more infirmation then in 95% of fundamentals.

Save it please.

Jamie,

Instead of removing paper trading, you can simplify it and clear the load from inactive users:
If you show statistics of the last minute on the Dashboard, you can easily delete the Statistics sheet.
You can limit the visible Orders and Fills to the history of 1 day.
You can limit the visibility of journals to 1 month.
But most of the effect will be from this:
Email 200000 Paper users, to confirm their paper accounts within a month, otherwise
they will be closed.
I bet you will have no more than a thousand answers - 95% load reduction.

and the wolves are fed and the sheep are safe

Paper trading is the last thing that keeps me active on Q as I use it for live trading. Although I won algo contest awards twice, the general restrictions around algo design do not match my style and I do not currently participate, hoping they will become more liberal over time. In the meantime, I enjoy working on new ideas and checking the forums.

Another issue is that these kind of breaking changes make Q look bad, similar to Google, where products and features suddenly disappear. I was on a sales call for Q Enterprise and this is what I literally asked the poor sales person (if I buy Q enterprise will I get all the features that I have now guranteed) and they could not answer. This is one of the reasons I didn't go forward with that.

In another words, Q's "sharpe ratio" is in question. While 'returns' are great (features), the volatility is also large (changes). It would be better that it was more like Amazon where I know I can trust the platform long-term.

Find a way to save it.

ps. I do see a progress in Q's attitude that I want to commend, with live trading the feature was just removed with 30 day notice, at least this time you are asking the community for opinion.

keep it, thanks.

current contest has many restrictions, i know you have many considerations for that design, but indeed it is not friendly.

paper trading is more useful than contest algo.

I don't see it as a zero-sum game. The more time I spend on the Quantopian platform, the more ideas I produce, the better I understand markets and trading, the better my submissions will be. When Quantopian shut down live trading, I probably didn't use the platform for half a year or more, as my focus got diverted elsewhere (including getting immersed in setting up my own trading platform, as well as simply being discouraged that my algo trading hobby was being such a pain and focusing on other hobbies instead). Quantopian removing that feature didn't lead to my focus no longer being split. Today I am still algorithmically trading my own account -- but now that means much of my focus is diverted to other toolsets/platforms. From Quantopian's perspective -- my contest and fund submissions have been set back six months, at least, and have taken more of a backseat.

The paper trade is another one of these features that, while perhaps no longer directly contributing towards Quantopian's ends, makes the platform more sticky. It is a feature that I do use, draws me to the platform, and while I'm here I also try out some contest ideas, run some backtests, chat on the forum, and check in on my contest entries. The paper trade gives us something to keep track of in real time, which is nice, since otherwise everything else happens on the scale of days, weeks, half-years. Humans aren't built to be engaged with things that happen that slowly. With the contest not updating more than once a day (and sometimes a couple weeks elapsing between updates), it's nice to have the option to check in and see how our strategies are performing. I still use Quantopian to backtest ideas for my own trading, so I also have been comparing my live trading results side-by-side with the same strategy paper trading on Quantopian so that I can better understand what kinds of assumptions (or mistakes) are baked into the fill model and minute-resolution model. It's also actually very valuable to me to validate that everything is running as it should be, data is as it sound be, trades are executing as they should be, etc. I also use it to debug. While it's possible to dig through the EOD logs and do the same, it's just not the same. For example, I can thank the fact that I was watching the paper trade in real-time for realizing that one of my self-serve datasets was stale. In hindsight there were other ways I could have discovered that, but I don't necessarily have that kind of foresight. The paper trade offers a unique lens into what an algorithm is doing.

So while most of what I use paper trades for is not directly for the contest or fund, it is keeping me engaged in algorithmic trading and alpha and understanding markets and all the things that could lead to a killer submission.

With the paper trade gone, more of my time and energy will once again be diverted elsewhere.

I am on this website to develop algorithms because it is my hobby and I enjoy doing it. All of my 200+ posts, community interaction and contest entries have come about as a result of the features of the platform. I'm not here purely to develop (for free!) for a faceless company with millions of dollars backing them. In return for us doing the work of paid quants for you, you guys provide us with the tools to do it (democratising wall street as it was)

The process of development relies on paper trading to check out of sample returns and to check if everything is working as designed. To be completely honest, if the paper trading had been removed with live trading I would not be on this website anymore and would have long moved over to a competitor.

If costs are an issue, Vladimir put forward a ton of great suggestions. Potentially you could also limit the number of live algo's someone can have to 5-10 per person. As it stands however, removing paper trading is just going to lead to a lot of the active 'quants' leaving to QuantConnect or similar.

If the paper trading functionality is removed then next will be the order type functions in backtesting, so only MaximizeAlpha() can be used. Get ready for removal of limit orders, order_target, percent, etc. Then get ready for removal of stock data outside of QTradableStocksUS.

This platform now is a focus on the enterprise's needs over the individual's needs. That wasn't always the case. It's free so who is anyone to complain. The contest is an extremely specific product to leverage the heck out of low vol strategies as an optimization problem, limited to ~2000 stocks, to which we can't access the source code. A new or current quant/researcher striving to make a living trading would not be putting forward their best interests focusing on the contest or this universe of stocks.

Why not re-establish live trading as a paid service along with data subscriptions then, even as a separate entity? If you offer it to enterprise you surely can offer it to individuals. If you enable the individual, you enable a community. It appears you're going the other way. And for those who say "its a hedge fund" - I don't know of another hedge fund that licenses out their platform as 'enterprise.'

I'd be ok with paying Quantopian 10% of profits for live/real via Robinhood. If a lot of us did well in that where constraints wouldn't be in play, it's always possible Q might notice some approaches that could possibly be considered for adopting into the contest.

Active users keep using their time(that's our money) to contribute to the platform and hand over good ideas/algorithms, but you guys keep removing some good functions that benefits the users. I dont see it is fair enough. Quantopian is actually better than others like Quantconnect, mainly the stock screening system. I have already put some non-pipeline algos to Quantconnect. I could see one day they or other platforms improve it and the remaining users here will leave.

I also wanted to add -- I've been using the paper trade to keep at the fore of my attention potential contest/fund entries as they collect OOS performance stats. I can keep track of how they unfold. And I won't forget them. Sure, I could mark it on a calendar six months from now to check in and run a backtest to check the OOS, But six months is a long time and I could miss it and the idea will be forever forgotten.

Live trading would be a good feature to add. Q can make good money if it adds in paid commercial and retail trading

You will lose almost all of your community if you remove paper trading. It’s what makes Q unique and interesting to people here.

If you need to get the operational costs off the P&L, get creative about spinning off what you have. I don’t know what the financials are, but there may be an ad funded way to make it worth the effort.

Best, LP

I do not use paper trading, so my opinion might not have value.

However, if some use it, and the cost is not that prohibitive, then for sure, Quantopian should keep it alive.

Quantopian looses members every day. Sure, new ones come in all the time. But if the experienced contributors to the community go because they lost a service they liked, it is the whole community that is losing on that deal. Just like when live trading was put down.

What Quantopian needs to do is keep its community alive and vibrant. I find it incredibly generous of all forum participants to post advice, points of view, backtests, tearsheets, algos, but maybe most of all their vision of what they do or intend to do. I read almost every posts, and cannot wait to read what my favorite authors will say.

So, keeping anything that supports their research, their motivation and their interest should be welcomed. It is not just a few of the community that benefits from this, it is all of us. And, in the long run, it is Quantopian that will benefit the most.

I use it to run models that i have in contest and also to test models that don't meet contest but are of interest to me. What's interesting is that papertrade has revealed issues with the platform that weren't apparent in backtest (Dan Whitnable can attest to this). Papertrade got me interested in this platform. For the record, I'll be very sad if you end it.

@Jamie

There are a few common themes in the responses here. It would be good for Q to address this.

If paper trading were taken away, I don't think I would come back much - except to view other's posts.

I figured that this day was coming and I have tried to port over as much of my library/algorithms to other trading platforms (IB/Tradestation). I still use some live paper trading algos for live trading that I check on each day.

I can even see the reduction in activity since live trading was cut. I believe this would probably be the death of the majority of the remaining users - though I'm sure some will stay and continue to research. But again we keep getting the hint that Quantopian is not here for the small investors like me.

@Jamie,
Truth be told, I forgot about the paper trading feature, as I was sure it would go away soon after the live trading went away.
Well...two years later and here we are...paper trading is still alive!

I just started running one of my algos under paper trading, and was pleasantly surprised how well that works!

I can see using paper trading for:
1. Factor validation, including those factors that aren't currently contest conformant or winners, yet are viable Q allocation factors. This matches up with the current mini-contest being run by @Wiecki.
2. Can also use it as a validation of live-running strategies that are much simpler than the current contest.(e.g. long-only, less-than 100 assets, or smaller co-integrated strategies).

Bottom line (my co-integrated pair of cents): Keep it if you want to expand your search-for-alpha to other than the current contest massive long-short strategies, and want to peripherally support your user's real-life trading to entice them to do the contest-like research.

alan

"Leveling Wall Street's Playing Field" indeed.

Respectfully consider this... I've been inspired to write contest algos while discovering my own less strict small scale alpha (that I'm able to trade) as well as vis versa. At the end of the day, we all want to pull money out of the market - stopping papertrade makes that opportunity seemingly less possible. I've actually thought it over long and hard (I was in mourning for 24hrs) and if the only reason to come to the site is to develop strict contest algos, I'll sadly spend less time here. I've found interesting alpha out there that doesn't wholly meet Q criteria but I do always aspire to fit to the Q constraints. Yin Yang. I for one, will be unfulfilled, if I never get to win the contest:(

Keep the paper trading please!

Was so excited to run my first live algorithms with Quantopian and Robinhood but, now that's gone! Paper trading was the precursor to my live trading experience and I still use them to live trade when I can get them to run. Also believe some of the best algorithms on Quantopian are tied the the paper trade algorithms that never make it to the contest. Keep paper trading alive. Thanks.

Hey everyone,

First off, thank you all for taking the time to write out your thoughts in this thread. We really appreciate hearing from you about why paper trading is important.

So far, I've noticed a few themes in responses in this thread:
- Algo doesn't meet the contest criteria. The contest is designed to support algorithms of a very specific nature, which can be prohibitive when you want to work on an idea that doesn't meeting the criteria. As such, the contest isn't always relevant to your interests in quantitative finance and Quantopian.
- Out-of-sample testing. You use paper trading to conduct out-of-sample testing on new ideas, which is a critical part of the development process.
- Desirable feature. Paper trading is a desirable feature that draws in new community members and generally makes the Q platform more appealing.
- Feature churn. Understandably, it is frustrating when features that you use are removed from the platform. Furthermore, the fact that features can be removed makes you feel uncertain about whether other features that you use will be around going forward.

I'd like to keep the thread open a little longer before responding to these comments. I'm hoping to follow up some time later this week or early next week. Also, if you feel like I missed any themes in the responses so far, please let me know.

Paper trading is the one of the main reason I come to Q. It is really useful for overall learning and it helps a lot to test out the algos that doesn't meet contest criteria.

i move my ideas to the contest, use maximizeAlpha method, the paper trading annual return was 20% to 30%, but the contest results just keep going down to -50%..

really don't understand what is " optimized " by the required ordering method.

Even if we cannot use paper trading with datasets which have a holdout but we are not required to use those.
Losing live trading was bad enough, losing paper trading for no good reason will make you lose your userbase.

I mentioned earlier, that I (along with Dan Whitnable) have noticed algo bug issues that bubble up in Papertrade that do not in backtest - well It is a current issue (again) for me. I run code in same timeframe through backtest and through papertrade, where only in papertrade errors bubble up. Furthermore It doesn't appear that Q's 'send us feedback' linkage works anymore (in my current case the copied link address: https://www.quantopian.com/live_algorithms/5d750f85aaf21d0042d853c5#feedback-modal). Can someone help me drill into this bug (my code, not platform)?

I was pretty miffed when live trading was yanked out from under us and 'level the playing field' was no longer honored. If paper trading goes too, so will I. If you offered paid live trading I would stay and become a disciple and, IMO, your community would flourish.

Please keep paper trade and implement live trading. It will benefit the community for the long run. It's valid to shut down idle accounts, but these are important features for quants.

I am new to Q. I signed up because there was a paper trading feature to backtest algos. If this feature is going away there is no need for me to stay. As a beginner statistics guy (i am good with coding and DBs) I’m looking more for a community to explore with as I don’t think my skill set is yet there to win contests. Yet.

Jamie,

Paper trading is extremely important to keep track of the algorithms daily response to the market and getting pricing and factor data computed for the current day.

If Quantopian removes the paper trading feature the entire platform becomes less valuable to many people (including me) who work on it and contribute to the community and damages the entire Q value proposition. I use paper trading all the time to see real-time positions and weight computations based on today's market data. It's extremely valuable and one of the main reasons I continue to work on this platform.

One of my biggest fears about investing more time and effort into the Quantopian platform is the fact that features suddenly disappear based on the singular focus on the contest input. Perhaps you guys can create a premium user profile and charge a fee for using the system with these features. A Freemium type business model.

This will create another incentive to move off the platform. Please keep it running and fund a way to fund the effort. You might be surprised as to how you can even raise revenue.

Thanks

Kamran

If you get rid of paper trading I personally know two funds and a society of quants who will leave the platform

Paper trading is a very educational tool and I would definately prefer it stay on the platform, because a few of the experimental algorithms I like to develop might not be suitable for the contest, but could do okay still.

Honestly, I think you should expand the functionality of this tool instead. I assume many would like to try their own algorithms on real money, which is definately a learning experience, so maybe you should consider to turn it around and actually expand the feature of this tool - so it will be possible to live trade real funds as well. I am sure there is a good business model to this concept.

Hi,

I am not sure if you've already turned off the paper trading. I've set two algos for paper trading this week. But noen of them works.

Good summary, Jamie and thanks for reaching out for feedback.

I agree with these, with comments:

  • Algo doesn't meet the contest criteria.
    For me it's less about meeting the contest criteria than getting to test those parts that are contrary to the risk model or order methods sans Q optimiser.

  • Out-of-sample testing.
    I test various methods unrelated to or in parts for OOS outside the contests but purely for experimental purpose - to see how it goes for real walking forward.

Cheers

I understand if you remove paper trading it could help free up resources, as you all have to take care of the employees too. But I agree with others, doesn't mean it should go to the attic never to be seen again.
Take any potential savings if any, and get some more recent Factset data with some of the holdouts. If you cancel live trading altogether, get more recent FactSet data so we can backtest in Research and feel as if the algo is more relevant. Or offer it as a paid service, and while it may not be a big money maker, it'll still be something to keep people on the platform

There's little reason or value for anyone to use the platform without live trading and "simulated" live trading. There's already very little value add to the platform and no real incentive for people to build any complex systems on here now.

And let's be honest - the contest and "funding" isn't that big a deal where you can self fund your own algo and keep 100% of the profit and not give away your IP to Quantopian for free

Please keep the paper trading. It is my main use case for Qunatopian. Thanks.

As a side note "$270k paid in royalties since August 2018" --> this is less than ONE decent quant makes in base salary before bonuses in NYC - having worked at several hedge funds as a PM. Now Quantopian has a limitless supply of quants working for free now and paying pennies out... it's nice when you can crowdsource your IP for free.

i think that there are many factors that push people to use the Q framework to develop strategyes with one own money using the paper trading feature, and one of them is that the fund is a black box and nobody knows if they are contributing to something that actually works.

Beside that many people have no time to dive in the complexity of developing complex strategies like these long-short ones, and just use the Q tools for theyr own interest, maybe just ginving some contribution to the communitiy by helping other people or sharing some useful algos, i am one of them and maybe i would accept to pay some annual fees to use datasets and frameworks to manage my strategies.

I think Q (Quantopian) business model will be even less sustainable without paper trading. Why? Because Q business model consider that the best Quants in the world will be willing to learn Q platform and enter Content to license their algorithms for very small fee comparing what they can earn. As mentioned above, best quant's yearly salary can by as all royalties paid per year at Q.

I believe the best Quants are attracted a) by possibility of live trading, b) by possibility of improving their trading skill, 3) by further get additional cash by entering contents to license their algo.

QC(QuantConnect) proved it is possible to get valuable cashflow from live trading platform and currently offer live trading servers additionally to contents. QC attract currently best Quants.

I'm looking currently on your business model with fresh eyes. I just started to learn Q and QC platforms. What I see is that QC is expanding, where Q is cutting off services. I think by this strategy more quants will move from Q to QC.

Focus is very good tool, but I think if Q will not change direction and will focusing in putting all eggs to one basket (contest), there is high risk QC will soon attract the rest of good Quants at Q business model will not be sustainable without Quants.

Please ask at Q question, how QC is able to provid live trading despite running on much smaller budget? Ask best Quants in the community, what features do you want Q to implement that you will stay with us?

Below is the explanation from Dan Dunn from Aug 23, 2017, why Q closed live trading. Cost of servers are decreeing and Quant community is increasing every day, maybe if you make calculation again you will find it is possible to profit from live trading in 2020-2030 additionally more quants will license algos with Q.

On Aug 23, 2017 Dan Dunn wrote about live trading closing:

Thanks for bringing this question up.

We gave a lot of consideration to this idea. Unfortunately, the economics of it still don't add up. Even if every person who is using the brokerage trading today gave us $20/month, it still wouldn't cover the costs of the service. The market isn't big enough. If you look at our forum posts and press releases from around 2013, back then we thought we could turn this into a full-fledged business. We tried very hard to do that. It was not easy. There are server costs every day, engineering costs on an ongoing basis, and support and operation considerations.

The monetary cost is only a part of the decision, though. Earlier, Fawce talked about focus. The benefits of focus are hard to quantify. We are teaching thousands of people around the globe each month about quantitative trading. These people are new to the industry. We teach them statistical and coding concepts, and we give them the data and tools to research their own ideas. We're helping thousands of people learn, and some of them will even get rewarded. We want to do that more, and we want to do it better. The focus we get from phasing out brokerage integrations will help us do more.

We are very sorry that we can't continue to provide this feature in this way.

Are there currently any limits in place for users? Since everyone wants it to stay but it is likely very costly to run, I feel like trying out a hard limit of 1-3 live algorithms per user, with an inactivity timeout via email (as suggested above) would as least be a step towards easing the cost to keep live. There are no Quantopian alternatives with the same quality tooling imo, and I hate to think that it doesn't work as a "fully-fledged business," paper-trading alone would definitely be a subscription-worthy service to me.

As a beginner Quant, Quantopian provides the best environment to get your feet wet. After a long time struggling with choosing the right software and approach I am very kind to finally start from here. I think your advantage in the industry is to provide tools for interested people that can directly test, paper trade or even license the written Algo. If the paper trading feature will be shut down, I will have to search for another place to work with quantitative trading....
And I really want to work hard in your environment at Quantopian.