Back to Community
End of Paper Trading

After much deliberation, we’ve decided to remove the paper trading feature from Quantopian. The service will be shut down after market close on December 31. However, you will continue to have access to the past results of any paper trading algorithms that you have in your account. We know that many of you use paper trading and expressed a desire for us to keep it around, so it was a difficult decision to make. Below, we will share what drove us to make this decision.

In another thread, several of you shared why paper trading is important to you. Additionally, we reached out to several paper traders privately, and learned about a few more use cases. There were a variety of responses, but a few themes arose:

  • Some community members aren’t interested in an allocation or participating in the contest. For folks in this group, paper trading provides value that is otherwise only available by submitting to the contest.
  • Other than the contest, paper trading is regarded as the best tool on Q for out-of-sample testing.
  • Other than the contest, paper trading is the best way to track the day-to-day results of an algorithm.

These are all valid points. The platform is currently optimized for researching and developing contest algorithms, with an eye to getting an allocation. That said, our decision to remove paper trading is not about further optimizing the platform for researching contest algorithms. Instead, it’s about moving the platform toward a more consistent experience.

In its current form, paper trading runs on different infrastructure than any of our other systems, it uses different technology that we no longer use anywhere else, and it doesn’t work with datasets that have holdouts. The fact that paper trading has its own infrastructure means our engineering team has to spend extra time maintaining the service, and the fact that datasets with holdouts don’t work in paper trading means that paper traders don’t benefit from new datasets. Cutting the maintenance cost of paper trading will allow us to make improvements to the platform at a faster rate.

Looking ahead to 2020, we are starting to think about how to leverage existing features, like the backtester and pipeline, to support more use cases. The contest and fund both operate on top of our backtesting infrastructure. We would like to see if there are ways to build on top of these features to support a wider variety of use cases and potentially offer an alternative for testing strategies out-of-sample. In the meantime, we suggest running regular backtests on an algorithm that you want to test out-of-sample. The backtester is designed to avoid lookahead bias, so if you keep the code the same, it serves as an effective test.

Additionally, we are looking to create more opportunities for community members to earn money. We recognize that the constraints on fund algorithms are very strict, and we are looking for ways to make it possible for authors of other types of strategies (e.g. long-only, sector specific, global markets etc.) to get rewarded. We hope to have more information to share on this topic soon.

We know this is not what some of you wanted to hear. This was not an easy decision, and we understand the frustration of losing a feature that you use. While we don’t have any news to share now, we plan to look at other ideas for improving out-of-sample testing on algorithms that don’t meet the contest criteria.

Best,
Jamie

Disclaimer

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.

54 responses

Goodbye quantopian

beginning of the end of Quantopian. Moving to other solution

Is it possible we'll see the return of paper trading again in the future? The infrastructure and code for running the contest must be similar to paper trading I would think.

As it stands, we currently cannot utilize the backtester to access a day's EOD data until 2 mornings later. Would this be a possible extension to consider now that you are looking into giving the remaining tools more leverage? As far as i'm aware Quantopian's system-wide pipeline data is updated around midnight, so realistically the data is there, we are just bottle-necked by the backtester's inability to schedule a pipeline at the end of the day. This would really help track strategies that rebalance daily.

Thanks for your thorough and honest answer @Jamie.

potentially offer an alternative for testing strategies out-of-sample.

This gets my vote. Offering back testing on international markets might provide another way to test robustness of US models on unseen data?

sad. is there a QuantCon next year?

It's doomed after living trading closed, right?

It is the right of any private business to do anything it wants. But when that business also wants to create and nurture a sense of community actions like these are devastating to its credibility.

Why was there a charade with asking the community about it? There was not a single response in support of ending paper trading and yet you did it any way. And it wasn't just 'several' people, it had over 50 replies. What were you expecting to hear that would change your mind?

I feel like my point was not acknowledged. I don't think it's accurate that all your users are siloed as either researching for personal use vs users researching for the fund. Myself and others I collaborate with are more opportunistic than that. I think this push to silo these two styles of user is counter-productive, and cuts off a valuable source of ideas from the platform. A richer toolset means more of my alpha research is done on-platform and is consequently compatible with the fund and analysis tools (such as alpha decay, tearsheets, etc.). When I research a source of alpha off-platform using my own testing framework, my own datasets, my own suitability criteria, etc., it no longer fits the Quantopian-fund paradigm, it's no longer compatible with the Quantopian toolset and datasets. It adds significant friction to bringing ideas to Quantopian for fund-suitability analysis. As limited as your resources are, I promise you, mine are much more limited. With this change, more and more of my limited resources are being directed off-platform. The more I have to look off platform for tools I use, the less I have time for Quantopian.

For me, paper trade provides an important lens into the cohesion between Quantopian backtests and off-platform live trading. Without the paper trading feature, it suddenly makes little sense to even backtest on Quantopian, since it'll be redundant. I'll need to backtest off-platform anyways in order to ensure cohesion with my live trading setup. And while I'm at it, I might as well take all my idea-generation off-platform as well. It quickly becomes a domino effect. Idea generation in of itself is neutral as far as whether they're fund-worthy or have personal-trading potential. But if we have to choose, off-platform is starting to look like it offers better prospects. And the cohesion issue goes both ways -- a strategy designed off-platform without the artificial constraints of the zipline paradigm and data-delay may not translate to being brought on-platform.

As researching on the Quantopian platform becomes more and more restrictive and feature-poor, you decrease the value proposition to your users. It largely boils down to the fruits of our labors that Quantopian passes on. Here is the question Quantopian should be asking itself -- Why should we (the "crowd") make Quantopian our primary research tool? With the direction Quantopian has been going, if we find valuable alpha on the platform and it's not in the sliver of strategies that receives an allocation, we'll have wasted our precious time. That same precious time could have been spent discovering valuable alpha off-platform, in which case it could still have a life, traded or licensed/sold.

Glad I was able to port over mostly all of my systems to my own server... so long Quantopian, it was a fun ride!

I agree with @Viridian on all counts.

I also totally agree with @Viridian Hawk
Well said Mr V Hawk.

@Viridian Hawk:

Thanks for the detailed response. I really appreciate the thought that you put into your comments, in both this thread and the other. I actually agree with what you said in this post, and I'd like to share some more thoughts and ask a couple of question to get a better understanding of your opinion.

Today, the backtester, the contest, and the Q fund all run on top of the same infrastructure. As mentioned in the original post, the current implementation of paper trading runs on different infrastructure. The fact that paper trading runs on different infrastructure means the maintenance costs are higher for our engineering teams, but it also means that it doesn't lend itself to being a lens into the cohesion between Q backtests and Q fund trading (I liked your wording there). Importantly, I think the tool is actually contributing to the silos that you currently perceive around the personal and research use cases. From a feature perspective, the biggest differences between running a paper trading algorithm and running a T+1 backtest is access to intraday pricing data, and T+1 fundamentals data (if you run a backtest through yesterday, the most recent fundamentals data you can get is from two days ago). Given how our fund infrastructure works, neither of these features are compatible with the Q fund, so any algorithm that uses them won't be able to receive a fund allocation.

Are there personal strategies that may depend on intraday pricing data or T+1 fundamentals data? Absolutely. Are there also strategies that don't depend on these features? Yes. Right now, the latter group fits the description you shared - they can be tested OOS via backtests and they can be considered for an allocation, regardless of whether or not they meet the contest criteria. The first group does not.

One thing I'd like to dig into a bit more, if possible, is this statement:

Without the paper trading feature, it suddenly makes little sense to even backtest on Quantopian, since it'll be redundant.

Can I ask you to elaborate on it a bit more? In particular, I'm curious as to why you consider the backtester to be redundant without paper trading.

I acknowledge that regularly running backtests to keep track of the OOS of an algorithm is clunkier than clicking "paper trade" and having it update on a daily basis. My comment in the original post, about looking for ways to build on top of backtesting infrastructure to support a wider variety of use cases and potentially offer an alternative for testing strategies out-of-sample, was meant to suggest that we'd like to improve that workflow. We don't have any specific solutions in mind yet, but it's something that we're thinking about.

I hear you - you are incentivized to use Q by the prospect of saving R&D time on your personal strategies, not by the prospect of receiving an allocation (for you, it sounds like it's the probability that's unappealing, and for others, it might be that they aren't interested in developing a strategy that conforms to the constraints of the Q fund). It is in Quantopian's benefit to have more community members so that there is a larger selection of algorithms to evaluate, so we should look for more ways to incentivize people to choose Quantopian as their primary research tool. I think you're right to frame this as a problem of incentives. In addition to my earlier question about Q backtesting being redundant, can I ask you to elaborate on this comment?

With the direction Quantopian has been going, if we find valuable alpha on the platform and it's not in the sliver of strategies that receives an allocation, we'll have wasted our precious time.

What end result would you want to see that would make you feel like your time spent working on Q would have been well spent, even if you don't receive an allocation?

@Ben Hawkins: Great question. You can actually already run pipeline through EOD yesterday (as opposed to the T+2 limit in the backtester), so long as it doesn't use any datasets with holdouts. You're right about the fact that our data updates overnight. We would like to make it possible for daily strategies to be able to operate on T+1 data as opposed to having to wait until the next day. It's something that we're thinking about, but we don't have any active projects to address the problem yet. In the meantime, have you considered running a pipeline + alphalens in research to see how a factor performed the previous day?

@Robert Rose: I think it depends on your definition of paper trading. At this point, we don't expect to create a new feature that depends on current day intraday pricing data. However, we would like to have better support for OOS testing on daily data. Similar to the above response, I don't have any more specifics or timelines on this topic, but it's something that we're thinking about.

@Jamie McCorriston You're right, I actually realized (duh!) right after posting that the research environment is capable of loading pipeline data without the restriction of needing to simulate a full day, thus making it a suitable replacement for tracking daily performance. Thanks!

Correct me if I'm wrong, but isn't the only real use-case of paper-trading access to 15-minute delayed intraday data? Since daily strategies generate alpha from yesterday's data and we have access to all non-holdout fundamental data before trading starts in the research environment, the only other reason I see for relying on paper-trading is intraday signals/performance. If trades are made once in the morning, you can track its realtime performance using 3rd party tools. And if trades are being made intraday, then paper-trading was already a sub-optimal solution since it is both delayed and you have to literally check it constantly for potential intraday signals. So at this point i'm not seeing what people are missing out on?

I find it hard to believe that Hedge funds won’t donate to you…understandably you don’t want to monetize your great tool, and rather, want to horde as much alpha as your tool can generate. Really not trying to guilt you, but I felt like your site (paper trade being integral) secured Q as synonymous with quantitative algorithmic investing/trading. I really hope that doesn’t change as I am truly grateful for the wealth of material you have shared. Generosity is associated with legacy and I'm sure your many contributions to the quant community will grow as you do.
~anyone want a scraper:(

Thanks Jamie for addressing my concerns. I also totally understand why you are killing the paper trade feature. The infrastructure required to pipe in a real-time price stream as opposed to simply getting a dump EOD -- just this alone presumably requires quite a bit of computing and manpower resources to maintain.

Can I ask you to elaborate on it a bit more? In particular, I'm curious as to why you consider the backtester to be redundant without paper trading.

I realize this is probably very specific to my situation, and everybody will have their own unique relationship to the platform. Since brokerage trading was disabled, I've been running my own trading system locally on my computer. What I've found is that small differences between my system and Quantopian can actually lead to massively different outcomes. So I've been using paper trading as a more-or-less real-time sanity check to help me debug where either my system is going wrong or Quantopian model is making unrealistic assumptions. Sure, without the paper trade feature I could wait until the next day to check the data, but then it'll be too late to intervene. It's also harder to reconstruct events from log timestamps than it is to watch as it happens.

So the big issue I'm struggling with at the moment in my live trading is debugging what causes a strategy to work on Quantopian but fail in my own system. The paper trade gives me a lens to compare the two. The other route is to build my own local backtesting environment using the same code base and data sources as my live trading system uses. This solves many of my problems, but comes at the cost of man-hours. But this is what I meant by redundant -- if I've built my own backtesting tools, gathered my own data sources, etc., I'll have that to use instead of Quantopian.

What end result would you want to see that would make you feel like your time spent working on Q would have been well spent, even if you don't receive an allocation?

That's the question: What can we do with the fruits of our labor, the countless hours we've spent developing a strategy on the platform if Quantopian passes on the strategy? The immediate answer that pops into everybody's head is, well if a strategy doesn't have hedge-fund level capacity, for instance, perhaps I can trade it myself. If a strategy isn't risk-neutral, but fits my own risk profile, perhaps I can trade it myself. Etc. etc. This is much easier said than done, it turns out, and with the demise of live brokerage trading, and now paper trading, this option drifts further and further out of reach for most of us.

Here's a recurring idea that I think is very popular with everyone: a strategies marketplace that connects algorithm authors directly with individual stock market investors or 3rd-party hedge funds. Quantopian could maintain a right of first refusal (aka "first dibs") for the Q fund and then collect a smaller fee on the strategies licensed to 3rd parties. The Quantopian platform already has the tools for analyzing and comparing strategies on different investability metrics -- sober, realistic assessments. Quantopian could do it right.

Basically it boils down to, if we have developed something of value (a strategy that is consistently predictive/profitable) on the Quantopian platform, and Quantopian passes on it for the fund, is there some way we can still monetize it? Personal trading and a strategies marketplace are the only two ideas I can think of, but maybe there are more.

Perhaps Quantopian could launch a Fund II that is less restrictive and more inclusive. Smaller AUM, no/minimal leverage, and larger risks, but much greater diversity of strategies. Could allow long-only strategies, strategies hedged by bonds, etc. Lots of great alpha that doesn't fit the Q-Fund risk criteria. Seems like it would require minimal additional infrastructure, since it's already in place for the existing Q Fund. Even if it wasn't a big money maker, it would likely encourage engagement from the community. Presumably you have a giant dataset of OOS strategy performance and could simulate how a fund of say the 500 best non-Q Fund strategies would have performed over the past year. If I've learned one thing about the stock market over the past few years, it's the benefits of diversification. If I want a high sharpe ratio, I hold as many stocks as my signal is predictive for and combine as many orthogonal alpha signals as I can. If I want to place on the contest leaderboard, I submit as many orthogonal entries as I am allowed. If Quantopian wants to have an industry-leading fund, they would be wise to offer a diversity of them.

@Jamie & Ben Hawkins thanks for pointing out the possibility of using the research environment during pre-market. I hadn't realized that that was possible. That will be useful for the "sanity check" aspect of what I've been using the paper trade for.

Deleting account...

The 'level the playing field' was what brought me in. When live trading ended that flat out ended. I've had fun with paper trading and don't have the time or intellect to enter the contest. I will be exiting and trying to figure out how to port my algos to Alpaca and get back to that level playing bit you removed long ago. Could use a consultant in CT who knows more than I how to do that without taking years like me.

Wow, this is really terrible news. I'm disappointed that I missed the earlier thread asking for feedback. Was there an email announcement? From what I can tell, the countless hours I've put into learning Python, learning the Quantopian API, and developing my algorithms is all going down the drain in ~2 weeks. Questions and comments:

  1. @Jamie - can we please get more time (end of Jan or Feb) to react to this? I use Quantopian paper trading to do manual live trading with a high volatility algorithm, so interruptions in trading are pretty impactful.
  2. @Jamie - I saw at least one other person suggest making paper trading a premium Quantopian service that users pay for. Was this considered as an option for offsetting the engineering costs. I would gladly pay a service fee for this.
  3. @Jamie - I use paper trading in conjunction with M1 Finance. My algorithm rebalances weekly. I logs the equities I should be invested in 15 minutes after market open one day a week, and then executes the order at the same time M1 does (7am PST). I just read the logs from the live paper trading and manually enter the orders in M1. Is there any way I will still be able to use Quantopian to do this after paper trading shuts down? From what I can tell, the answer is no due to backtesting using delayed data (1-2 days).
  4. @Community - sorry to ask this here Quantopian, but... what alternatives are Quantopian community members considering? QuantConnect does paper trading, but last time I tried it it was about 10x-20x slower than Quantopian for backtests, which seems like it makes it not a practical solution.

@Viridian Hawk: Thank you for the suggestions. It means a lot to us that you are looking for ways we can help you stay incentivized to keep using the platform.

Your suggestions of personal trading and a 'strategies marketplace' make sense to me. For the latter, I think the main issue is that there are a lot of good strategies written on Q that just don't fit our fund model. Your question about other opportunities to reward the authors of these strategies is a good one and it's something that we are actively looking into. In the original post of this thread, I shared this:

Additionally, we are looking to create more opportunities for community members to earn money. We recognize that the constraints on fund algorithms are very strict, and we are looking for ways to make it possible for authors of other types of strategies (e.g. long-only, sector specific, global markets etc.) to get rewarded. We hope to have more information to share on this topic soon.

I don't have any more specific info to share yet, but we are hoping to have something to share soon.

Regarding pipelines in research, I'm glad that you expect these to be useful for your sanity check. My advice would be to run the pipeline after 8:45 ET, as that's when pipeline locks in it's EOD knowledge from the previous day. Any new data that comes in after 8:45ET won't be available until the next day's pipeline.

@Ryan: The earlier discussions about potentially removing paper trading were only in the forums. In retrospect, we should have emailed more folks earlier. I'm sorry that the change is causing you to feel like your time spent learning Python, the Q API, and developing algos is going down the drain. That said, I'm wondering if there's an alternative approach you can use to keep entering your trades manually, without losing your work.

I am unfamiliar with M1Finance, so my suggestion may not make sense (I apologize in advance if that's the case), but would be be able to run a pipeline in Research once per week to get the buys/sells being considered by your algorithm, once a week? In Research, you won't have access to today's intraday pricing data at 7am PST, but you will be able to run a pipeline for today, getting EOD data up through the end of the previous trading day. As long as your weights are constructed using daily data (which I assume is the case, given your description of the algorithm), you should be able to use the pipeline output to inform your manual trades in M1 Finance. In fact, you will have access to that pipeline data by 5:45am PT, so you could even have your trades lined up for market open. Does that sound like it might help your case?

Regarding making paper trading a premium feature - we considered it. Unfortunately, it's not a feasible solution at this time.

Again, Quantopian not listening to its consumer base and missing on opportunities to make money. It's a shame that the Q team when given the option to implement a paid model for broker integration and paper trading capabilities, they choose the option to reduce the functionality and just miss out on revenue with no plans on bringing features back. This will prompt even more of the community to leave.

Just signed up here. If I code something here do I have to find a way to port that data into a diff platform in order to back test it?

Too bad, understand why the decision was made though. Thanks for the transparent, thoughtful message here.

I think it is fine that Q discontinued live paper. In my opinion, live paper gets you only so far beyond out-of-sample testing. The true test is live cash trading, and that means porting your code to another platform. If your algo is very dependent on execution, such as intra day trading, then you need to go beyond Q and test live with IB using more advanced ordering.

I have my own backtester and live trader, but I still find Q very effective for prototyping as it give me quick access to clean data sources and am efficient pipeline mechanism.

Also, I think it is great that they brought in 3rd licensors that could pick up algos that did not fit Q criterias.

Thanks again Q.

+1 Luc

Hello,
I have a quantopian algorithm that I wish live paper trade. Could someone explain what I can use to do this once Quantopian cancel its live paper trading service.

Thanks,
Charles

Bye-bye quantopian... Paper trading is essential part of algo research cycle.

@Charles Stiegemeier most brokerages will allow you to set up a paper trade account. Alpaca and IB are examples. There are versions of zipline (the engine that powers Quantopian's backtester) that are compatible for both of those brokerages, but I think you will typically need to make some changes to your Quantopian code in order to make it compatible.

@Jamie McCorriston: Since paper trading will be removed, any possibility that we can receive the source code for the paper trading dashboard frontend/backend for the people who would like to run zipline with a nice web interface on there own personal systems?

Where do we find market data that, like Quantopian, excludes premarket and after hours data?

@Emile Farkouh I haven't used https://www.alphavantage.co/ but it may have that. I have also heard of people using IEX. I use https://polygon.io/ which I get for free access to as part of having a funded Alpaca.markets account. It definitely has that.

@Viridian Hawk Thanks very much for the suggestions but alphavantage and IEX don't have those options, and at $200/month, Polygon is pricey and I'm still not guaranteeing I'll have the kind of data Quantopian is using (see below).

I tested the SPY ETF.

Interestingly, I was testing OHLCV data, and found that the data gets adjusted for dividends a day earlier than it does on Yahoo and Interactive Brokers data.

The most recent ex-dividend date was the 20th of December, 2019. Quantopian's data reflected this on the data for the 19th where it used Yahoo's Adjusted Close as the close price.

Looking at the Close, I saw that on the 17th and the 18th of December, Quantopian was using the same Close price as Yahoo (off by a couple cents), but on the day before the ex-dividend, the 19th, it starts using the Adjusted Close instead.

Another thing to note. If I'm running handle_data on data for the day itself (t), the data isn't adjusted. It's only adjusted on t+1. So if I run:

def handle_data(context, data):  
    spy_etf = sid(8554)  
    spy_c = data.history(spy_etf, "close", 30, "1d")  
    log.info("close " + str(spy_c[-1]))  

The 19th shows the "correct" (Yahoo) close. But if I run:

def handle_data(context, data):  
    spy_etf = sid(8554)  
    spy_c = data.history(spy_etf, "close", 30, "1d")[:-1]  
    log.info("close " + str(spy_c[-1]))  

then, on the 20th, it shows the Adjusted Close for the 19th.

I'm not sure how I can find data with these exact specs.

@Jamie McCorriston - I've never been able to easily copy paste code from /activity/code tabs in the full backtester. there has always been some formatting issue that prevents what should be a simple copy from backtest code, to paste into IDE. My workaround has been to spin up a papertrade instance and copy from code in papertrade to paste into IDE. Without getting into a discussion of if I'm overfitting or not lol, have you considered this work use case? Are you aware of the copy/paste from Historic Backtest issue I'm referencing? How can I efficiently copy paste code from Historic backtest to IDE? thanks.

I second stephan that this bug is super annoying!!! All double-newlines gets converted to some sort of illegal whitespace character. My workaround is to copy the code into a text editor on my computer and do a "Search and Replace All" to convert all the illegal whitespace characters into four spaces. Would be so happy to not have to do that each time I want to revert to a previous backtest code.

@Viridian & stephan: I've been annoyed by the inability to copy paste as well. However, I've found if you go to the "old" backtest page and use the "view code" dropdown, you can copy/paste from that popup with no further adjustment.

Hi everyone, we've removed the paper trading buttons from the new and old full backtest pages, which means that live algorithms can no longer be deployed. As a reminder, existing live algorithms will continue paper trading through close of market on December 31, 2019.

Disclaimer

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.

Remember when their company moto was "level wall streets playing field" ....

Pepperidge Farms remembers.

@Jamie - I don’t think your suggestion above will work for me, but I might be misunderstanding some details of how Quantopian works. Currently, I use schedule_function to run my rebalancing function at market open (calculating securities and their associated weights), and another schedule_function to execute my trades 30 minutes after market open to simulate how M1 Finance works (they do bulk trades for all customers at 2 fixed times per day). When I run this “live” with paper trading, I log out the securities and their weights, and this gives me a 30 minute window of real time to read the log output of my live algo and get my trade orders in. If I change my algorithm to use schedule_function at market close of the previous day, the results are very different, and this is why I think Jamie’s suggestion won’t work. Is it possible to achieve in research the same thing I described above (e.g. execute my rebalancing function at market open with market open data to determine which trades I should execute shortly after market open)?

bye..

Hey folks,

Wondering if some of you would consider an open way to buy/sell Quantopian algos. I wrote more about the idea here:

https://www.quantopian.com/posts/buy-slash-sell-quantopian-algorithms-in-an-open-way

I am really surprised to know that Quantopian does not support even paper trading anymore today.
If you are considering alternative solutions to run your Quantopian strategies on your own machine or a server in cloud, please try www.IBridgePy.com
Migration instruction is here
http://www.ibridgepy.com/tutorials/#Migrating_from_Quantopian_to_IBridgePy Also, IBridgePy starts to support backtest using IB's historical data and data from other data providers.
You may learn IBridgePy's backtest features on YouTube
https://www.youtube.com/watch?v=xWMzTgGWv48&t=624s

DISCLAIMER: This is Dr. Hui Liu, the creator of IBridgePy

@Hui Liu

I think the problem for most folks converting to most such solutions:
1. Do not support pipeline (which majority of algos use)
2. Do not have clean data out of the box (and even if you could find a source, it is likely to be cleaned in a different way than Quantopian). By this I mean both the pricing and fundamental data.

@Hui Liu I agree with Vladimir - but I saw you are looking to support pipeline in the future, so I'm sure many would be interested once that becomes available.

In the mean time, my work around is to select stocks at the close two days before trading at the open (haven't tried this live yet, but I think it should work given the data delay). Attached is an algorithm that does just that. It chooses trades on the close two days before it trades - you can change the days on lines 57 and 62:

schedule_function(  
        select_stocks_and_set_weights,  
        date_rules.month_end(days_offset = 11),  
        time_rules.market_close(minutes = 30)  
    )  
    schedule_function(  
        trade,  
        date_rules.month_end(days_offset = 9),  
        time_rules.market_close(minutes = 30)  
    )  
Clone Algorithm
17
Loading...
Total Returns
--
Alpha
--
Beta
--
Sharpe
--
Sortino
--
Max Drawdown
--
Benchmark Returns
--
Volatility
--
Returns 1 Month 3 Month 6 Month 12 Month
Alpha 1 Month 3 Month 6 Month 12 Month
Beta 1 Month 3 Month 6 Month 12 Month
Sharpe 1 Month 3 Month 6 Month 12 Month
Sortino 1 Month 3 Month 6 Month 12 Month
Volatility 1 Month 3 Month 6 Month 12 Month
Max Drawdown 1 Month 3 Month 6 Month 12 Month
"""
Original by: Christopher Cain, CMT & Larry Connors
Posted here: https://www.quantopian.com/posts/new-strategy-presenting-the-quality-companies-in-an-uptrend-model-1
(Dan Whitnabl version with fixed bonds weights)  
(Nathan Wells modified for performace and logging)
"""
# Quality companies in an uptrend 
import quantopian.algorithm as algo
 
# import things need to run pipeline  
from quantopian.pipeline import Pipeline
 
# import any built-in factors and filters being used  
from quantopian.pipeline.filters import Q500US, Q1500US, Q3000US, QTradableStocksUS, StaticAssets  
from quantopian.pipeline.factors import SimpleMovingAverage as SMA  
from quantopian.pipeline.factors import CustomFactor, Returns
 
# import any needed datasets  
from quantopian.pipeline.data.builtin import USEquityPricing  
from quantopian.pipeline.data.morningstar import Fundamentals as ms
 
# import optimize for trade execution  
import quantopian.optimize as opt  
# import numpy and pandas because they rock  
import numpy as np  
import pandas as pd
 
def initialize(context):  
    # Set algo 'constants'...  
    # List of bond ETFs when market is down. Can be more than one.  
    context.BONDS = [symbol('IEF'), symbol('TLT')]
 
    # Set target number of securities to hold and top ROE qty to filter  
    context.TARGET_SECURITIES = 5
    context.TOP_ROE_QTY = 50 #First sort by ROE
 
    # This is for the trend following filter  
    context.SPY = symbol('SPY')  
    context.TF_LOOKBACK = 200  
    context.TF_CURRENT_LOOKBACK = 20
 
    # This is for the determining momentum  
    context.MOMENTUM_LOOKBACK_DAYS = 126 #Momentum lookback  
    context.MOMENTUM_SKIP_DAYS = 10
    # Initialize any other variables before being used  
    context.stock_weights = pd.Series()  
    context.bond_weights = pd.Series()
 
    # Should probably comment out the slippage and using the default  
    # set_slippage(slippage.FixedSlippage(spread = 0.0))  
    # Create and attach pipeline for fetching all data  
    algo.attach_pipeline(make_pipeline(context), 'pipeline')  
    # Schedule functions  
    # Separate the stock selection from the execution for flexibility  
    schedule_function(  
        select_stocks_and_set_weights,  
        date_rules.month_end(days_offset = 11),  
        time_rules.market_close(minutes = 30)  
    )  
    schedule_function(  
        trade,  
        date_rules.month_end(days_offset = 9),  
        time_rules.market_close(minutes = 30)  
    )  
    schedule_function(record_vars, date_rules.every_day(), time_rules.market_close())  
 
def make_pipeline(context):  
    universe = Q1500US()  
    spy_ma50_slice = SMA(inputs=[USEquityPricing.close],  
                         window_length=context.TF_CURRENT_LOOKBACK)[context.SPY]  
    spy_ma200_slice = SMA(inputs=[USEquityPricing.close],  
                          window_length=context.TF_LOOKBACK)[context.SPY]  
    spy_ma_fast = SMA(inputs=[spy_ma50_slice], window_length=1)  
    spy_ma_slow = SMA(inputs=[spy_ma200_slice], window_length=1)  
    trend_up = spy_ma_fast > spy_ma_slow
 
    cash_return = ms.cash_return.latest.rank(mask=universe) #(mask=universe)  
    fcf_yield = ms.fcf_yield.latest.rank(mask=universe) #(mask=universe)  
    roic = ms.roic.latest.rank(mask=universe) #(mask=universe)  
    ltd_to_eq = ms.long_term_debt_equity_ratio.latest.rank(mask=universe) #, mask=universe)  
    value = (cash_return + fcf_yield).rank() #(mask=universe)  
    quality = roic + ltd_to_eq + value  
    # Create a 'momentum' factor. Could also have been done with a custom factor.  
    returns_overall = Returns(window_length=context.MOMENTUM_LOOKBACK_DAYS+context.MOMENTUM_SKIP_DAYS)  
    returns_recent = Returns(window_length=context.MOMENTUM_SKIP_DAYS)  
    ### momentum = returns_overall.log1p() - returns_recent.log1p()  
    momentum = returns_overall - returns_recent  
    # Filters for top quality and momentum to use in our selection criteria  
    top_quality = quality.top(context.TOP_ROE_QTY, mask=universe)  
    top_quality_momentum = momentum.top(context.TARGET_SECURITIES, mask=top_quality)  
    # Only return values we will use in our selection criteria  
    pipe = Pipeline(columns={  
                        'trend_up': trend_up,  
                        'top_quality_momentum': top_quality_momentum,  
                        },  
                    screen=top_quality_momentum
                   )  
    return pipe
 
def select_stocks_and_set_weights(context, data):  
    """  
    Select the stocks to hold based upon data fetched in pipeline.  
    Then determine weight for stocks.  
    Finally, set bond weight to 1-total stock weight to keep portfolio fully invested  
    Sets context.stock_weights and context.bond_weights used in trade function  
    """  
    # Get pipeline output and select stocks  
    df = algo.pipeline_output('pipeline')  
    current_holdings = context.portfolio.positions  
    # Define our rule to open/hold positions  
    # top momentum and don't open in a downturn but, if held, then keep  
    rule = 'top_quality_momentum & (trend_up or (not trend_up & index in @current_holdings))'   
    stocks_to_hold = df.query(rule).index  
    
    # Set desired stock weights  
    # Equally weight  
    stock_weight = 1.0 / context.TARGET_SECURITIES  
    context.stock_weights = pd.Series(index=stocks_to_hold, data=stock_weight)  
    # Set desired bond weight  
    # Open bond position to fill unused portfolio balance  
    # But always have at least 1 'share' of bonds  
    ### bond_weight = max(1.0 - context.stock_weights.sum(), stock_weight) / len(context.BONDS)  
    bond_weight = max(1.0 - context.stock_weights.sum(), 0) / len(context.BONDS)  
    context.bond_weights = pd.Series(index=context.BONDS, data=bond_weight)  
    #print("Stocks to buy " + str(stocks_to_hold))
    print("Stocks to buy: " + str([ str(s.symbol) for s in stocks_to_hold ]) )
    
    #print("Bonds weight " + str(bond_weight))
def trade(context, data):  
    """  
    Execute trades using optimize.  
    Expects securities (stocks and bonds) with weights to be in context.weights  
    """  
    # Create a single series from our stock and bond weights  
    total_weights = pd.concat([context.stock_weights, context.bond_weights])
 
    # Create a TargetWeights objective  
    target_weights = opt.TargetWeights(total_weights) 
 
    # Execute the order_optimal_portfolio method with above objective and any constraint  
    order_optimal_portfolio(  
        objective = target_weights,  
        constraints = []  
        )  
    #Log our holdings
    log.info( [ str(s.symbol) for s in sorted(context.portfolio.positions) ] )
    #print("Cash: " + str(context.portfolio.cash))
    # Record our weights for insight into stock/bond mix and impact of trend following  
    record(stocks=context.stock_weights.sum(), bonds=context.bond_weights.sum())  
def record_vars(context, data):  
    record(leverage = context.account.leverage)
    longs = shorts = 0
    for stock in context.portfolio.positions:
        if context.portfolio.positions[stock].amount > 0:
            longs += 1
        elif  context.portfolio.positions[stock].amount < 0:
            shorts += 1
    record(longs = longs)
    record(shorts = shorts)
There was a runtime error.

Anyone local north central CT and could help me set up home server and port my quantopian algos to alpaca? I'll pay hourly. I have dabbled at best here on quantopian and with python but not confident enough still but want to continue to dabble and not interested in the contest.. single father with two kids and full-time job I get exhausted looking at the tutorials right now; but want to move the needle forward. Please message me if interested. Maybe in 10 years when they can make their own meals I'll catch up.

Why all the complaining? Seems like Q was full of career paper traders. Go live or go home.

@PejH, I can't speak for everyone but I always back tested my code on here in order to stress test and then implement on paper trading with higher asset allocations. I've since moved my code to another API and are always looking for new platforms. I discovered Quantopian as an undergrad and the idea of regularly paper trading seemed dull. However being able to tack an algorithm that I can constantly build was much more interesting.

And it's not really complaining when the main problem everyone is having is we were asked "Do you like Paper Trading" and when the community replied with "Yes," Q still ended it. The moaning and groaning is rooted to the fact that an A/B decision was given and Q had the intention of ONLY picking A.

@Vladimir
Agree with you. I am wondering how many people will be interested in IBridgePy if a similar pipeline is provided by IBridgePy. We can allocate more resources if the demand is really high.
By the way, IBridgePy has started a YouTube channel to give more tutorials. https://youtu.be/xWMzTgGWv48
The latest tutorial is about trading with TD Ameritrade using IBridgePy.

DISCLAIMER: This is Dr. Hui Liu, who created IBridgePy.

@Stephen Maruschak I don't think that' s the issue at all. I think the real issue is a fundamental misunderstanding of facts. Let's try and stick to them.

At the risk of hurting some feelings, the facts are:

For continuous forward testing, Zipline is available.

If you are writing code "in order to stress test and then implement on [sic] paper trading with higher asset allocations", at the end of the day, aren't you still paper trading? Nothing is stopping you from building and tracking an algo and optimizing it for as many years as you want.... But what is the purpose of doing this? To get an allocation from Q? Or to keep stress testing and optimizing for the sake of learning?

https://www.quantopian.com/posts/using-zipline-locally-vs-quantopians-platform

https://www.zipline.io/index.html

Zipline is a Pythonic algorithmic trading library. It is an
event-driven system for backtesting. Zipline is currently used in
production as the backtesting and live-trading engine powering
Quantopian – a free, community-centered, hosted platform for building
and executing trading strategies.

Quantopian, at its core, is a slick UI wrapped around open source libraries plus a fairly decent web based IDE that integrates seamlessly for web based development, if that's your preference. I don't know about others, but I prefer PyCharm and Spyder over Q's native IDE for several reasons. Yes Q does a brilliant job at shortening the learning curve for beginners by alleviating the need to install a Python package , having to deal with the libraries and packages required to run Zipline locally, etc... But there is no path from running a simulation to running your algo in real world. And this is by design.

There is a tremendous amount of work required to port an algo developed inside Q over into real world trading, and a lot of it is a function of the broker's API.
When Q closed off outside trading with IB, that was the beginning of the end of forward testing (aka paper trading).

Q said it very clearly: "Having to support the simulation takes up resources that are needed for the platform because the live trading simulation runs separate from the main platform".

Q is looking for talented developers that 1) have a fundamental understanding of market structure, 2) can conceptualize a strategy and take it all the way from design to production, 3) have data science skills yet are creative enough to think in the abstract, 4) can source additional data sets that could augment their existing models, 5) have the resources to conduct experiments off platform and integrate positive results into their work on Q, and 6) have the desire to license their IP on an exclusive basis to Q to monetize.

The reality is most Quants that fit those requirements will rarely conform to the last one, because it's a deal breaker for many. So the talent is being farmed organically. I would love to know how the fund is performing. Have their strategies achieved capacity and risk adjusted returns in line with expectations?

I will add one more point which has not been mentioned: Quantopian Enterprise, offered by FactSet. My trader's instinct tells me everything has been a facade and a smoke screen because Q needed a path to shutting down what essentially was a free version of Quantopian Enterprise, because Enterprise came much much later but it is now a viable product that must be commercialized. I'm not certain if FactSet has added forward testing functionality yet, but early last year they told me they were considering it.

https://factset.quantopian.com/

Be grateful access to one of the most robust, powerful, resource rich algorithmic development platforms with integrated data, research, analytics and much more remains free, and for everything else that's open source, well, it's open source. Install it locally, if you can't, learn... if you don't have the patience, hire, or get your university to do it, or buy a license to Q Enterprise.... People are commenting like it's the end of the world. It's not. Nothing has changed except for Q no longer hosts the forward simulation environment, but being open source, it's still supported by an ever growing community. And if that's too much work, Quantopian Enterprise will give you access to even better FactSet data that's tuned for institutional consumption.

@ Hui Liu,

I am using your IBridgePy. I hope you could develop the multiple account trading and support fetching fundamental datas a.s.a.p. :-)

Has the paper trading infrastructure been outsourced yet? I mean the front and back-end pieces so that we can run it locally or on our own servers?

I don't mean the iPyBridge Freemium product either. I mean the full open source code needed to install and run on local servers and access using local clients.

What if any QUANTOPIAN plans are there to do this? If yes, is there a schedule with a target date?