Installing Zipline - Lord help me!

UPDATE: Zipline has been installed, but when I copy and paste the algo from GitHub into Anaconda (Spyder) it won't run. I found out it's missing a few lines of code to run:

perf_manual = algo_obj.run(data)

This brings on a new problem, 'data' not defined error.

SOLVED - Magically it finished installing, guess I needed a little more patience. Took about 15min to get past that line but on to bigger and better things here :)

Alright, so I've got Anaconda2 with python 2.7 32-bit installed. In Spyder console I've used the "conda install -c Quantopian zipline" command line. It goes through downloading a bunch of packages, then extracts the packages. It's got part way through "Unlinking packages ..." and has frozen at this line :

[ COMPLETE ]| | 0% [conda ]| | 0% [conda ]| | 0% [matplotlib ]|##### | 10% [numexpr ]|########## | 20% [numpy ]|############### | 30% [pandas ]|####################

This is the second time I've tried this now, well 3rd but that was with Anaconda 3 and Python 3.5 which I heard was even more of a pain. The previous time I believe it froze at a different line but none the less, I'm still kinda stuck.

I'm fairly new to python so if somebody has a hand to lend it'd be much appreciated. If you'd suggest I go a totally different route to install I'm all ears. As well, if a different IDE would work better, I'm open to everything.

The reason for all of this is because, from what I understand, Quantopian won't let me import a trained model, and when I try to run my training on Quantopian it says it's out of memory and to optimize my code. So figured I'd get out of Quantopian altogther and do it locally. Thanks!

9 responses

Lord help you I indeed. What are you trying to do? I found Zipline about 1,000 times too complex for my needs. And what data are you going to use? EOD? Or minute or tick or what?

When I try to train my machine learning model on Quantopian it runs out of memory. Since, as far as I know, there is no way to import a saved model into Quantopian, I'm forced to do the whole algo, and backtesting, locally. I did manage to get zipline installed but even the example in the tutorial on GitHub won't run, been trying for 4 hours now. I may not be very experienced with Python but I've been writing computer programs for 20 years, doing my best to not give up haha.

I'm happy with any data to get started. I'll be using minute data for the algo, and I can retrieve that myself if I need, I'd just like to try and make use of zipline so I don't have to build a backtesting engine from scratch. I think the tutorials on GitHub are a little outdated.

How are you going to train on minute data? You have it I guess? I eventually went a very different route and only use EOD data since my trades tend to be relatively longer term and of liquid instruments. Also I like to use futures in backtesting as well. If you want to trade short term I guess you need Q unless you want to spend a fortune on data. But what will you train the offline model with?

My main problem began with loading it onto a Windows hosted version of Anaconda - I spent literally weeks trying to fix the issues around the Compiler.

I'm a lot slicker with Python these days so could probably manage it.

If you feel you can make do on EOD data there are a number of Python based backtesters out there which beat the hell out of Q for simplicity. It's a quagmire.

Haven't trained any ML models on my setup yet but given all the software is Python depencies anyway this is not going to be a problem.

Google "python backtesting framework". There are one or two excellent frameworks out there.

I've got loads of good minute data to both backtest on and train with, it's the zipline issues that are killing me here. If you're saying there are better frameworks out there I'll take your word for it and drop Zipline / Quantopian altogether, I'm pretty much fed up with it. Thanks

I found Zipline impossible when I tried it. On the other hand I'm not sure whether any of these other frameworks handle minute data. You will have to take a look.

As I say....Maybe given I have worked very hard on Python for a year or so maybe I would not find it such a struggle now. On the other hand for me at least life's too short to want to venture into the badlands again.

If I don't see an updated installation and use tutorial from the Dev's sometime soon here I might be done with algo altogther. It's frustrating. I have a simple idea I want to test and none of the Quantopian software seems to be co-operating. Seeing as it is ML it could be trained on dairly data and still hold some weight in minute land, but I can't upload a trained model to Quantopian I don't think, not 100% sure yet though as nobody has replied to that question.

This forum does not support or give any help on Zipline itself. There is a Google group you can try but also if you think there is an error report it on the github page. You will get no response here on Zipline.

I'm tinkering around with zipline a tad bit because it is used by pylivetrader and while zipline needed to be updated due to the version that came with anaconda, zipline has been flawless. Although, in my situation I'm not attempting to add my own data, letting pylivetrader handle that.
One thing I like a lot: In initialize() can do context.set_cancel_policy(NeverCancel()and then stop or limit can also trigger in non-market hours, presuming also time_in_force = 'gtc' in alpaca.py, and in zipline/algorithm.py:

    def set_cancel_policy(self, cancel_policy):
cancel_policy = zipline.api.NeverCancel   # gh


Setup in Python has almost always pretty much been a rube goldberg device in my opinion. In this case, pip did not explain what --user does but it seemed to work magic in overcoming some errors:
pip install --upgrade --ignore-installed zipline --user
. . . and that's it, nothing after that option.
After install, try:

> pip install pipdeptree
> pipdeptree -p zipline


Then if there are any conflicting dependencies, have fun resolving them. Once done, should have a solid foundation.

In writing this, to diff for changes I had made, a new install went to C:\Users\<user>\Application Data\Python\Python36\site-packages\zipline instead of C:\Python36\Lib\site-packages where everything else is, almost as if Python has a random surprise generator to make sure we're awake. Possibly that --user switch.

Continuously amazed at Quantopian's generosity in sharing zipline with the world freely. Seems clean right now, while on the other hand, considering especially our various operating systems, Python's pip might have the greatest room for improvement in my opinion at this moment.

Zipline may well be one for the great category.