Back to Community
RSI attempt, general questions and feedback

Hi all,

Below is my first coded backtest using Quantopian*; it's an implementation of the Relative Strength Index (RSI definition), which should be familiar to most who've scratched the surface of first-level technical trading and is often cited as a useful overbought/oversold momentum oscillator. My adoption of this indicator as my first backtest is by no means an endorsement of it usefulness, it was simply a way for me to get used to the IDE with something easy as a subject. (The backtest timeframe below was crudely chosen to show good results and nothing more!)

Thoughts and feedback on the process (apologies in advance if any of this has arisen due to my lack of understanding of the system…still learning!):

The IDE

In general, I think this is excellent. The area for building up one's backtest is clean and relatively simply to use. (I do worry that for someone who has no, or little coding experience, the thought of having to actually write code to get a backtest to work, as opposed to being able to, say, drag and drop in the way that Quant Blocks works, might actually be a barrier to them using the tool at all…but more on that below**).

My biggest request for improving the IDE would be around being able to step through a program as it executes on real data. Be able to do so, and concurrently seeing the state and value of local variables, would be massively helpful in debugging and understanding, for example, why a particular order is triggered, or not triggered (etc.), at a certain point in time. Without being able to step through the program on real data, I found that debugging my code was a painful process of having the fill the script with LOADS of log.debug stuff, then run a full backtest and try to match up the Transaction Details to the Log Output in order to work out what was going with the strategy, at what point in time, and why. Step-through functionality - even if just on a static dummy set of data - would be huge.

The Backtest process

The disparity between running a backest in the IDE on daily data, and the full backest that runs against minute bars is clunky. Given the full backtest can only be run against minute bars (is this right?), then it is best to code in preparation for this, which means the "daily" backtest in the IDE becomes a bit redundant; I ended up using it purely as a way to ensure the program I was writing had no build errors (is that what it was intended for?).

The availability of more options around "frequency" in general would be a great improvement in my opinion: more options around the frequency at which the strategy is evaluated (i.e. I might like my strategy to be an hourly, or 3-hourly, or monthly strategy - as opposed to the default minute bars - and I cant be bothered (read: I don't know how) to ensure my code does this for me) and more options around the frequency of the various available transforms would be good (i.e. the mavg is currently a daily moving average, but given we're executing against minute data, it might be nice for such transforms to be of the form mavg(input_range,input_freq) with input_freq having options such as minute, hour, day, month etc. such that mavg(5,hour) would give you the 5-hour-moving average). In general, maintaining complete disconnection between the frequency at which the transforms are calculated and the frequency of the strategy evaluation would be desirable.

Which brings me onto the transforms and the bigger question alluded to **above. (I've tried to write the following sentence a few different ways, but none of them sound right, so I'm just going to straight ask it as a question instead)

Is the plan for Quantopian to be the 'custodians' of the transforms that are available to a user as defaults, and increase the suite of transforms as the platform grows? Or is this responsibility to fall to the community in general?

I ask because one of the prime attractions to new users (new users == community good) will be the variety and ease with which they can implement a large suite of (rigorously tested) time series calculations into their various strategies. If there are going to be a wide selection of them available as defaults from Quantopian, then great, but then the current selection needs to widen rapidly beyond the few simple ones currently available, for the platform to be attractive to those users who aren't prepared to code them up themselves.

If, alternatively, the community is to provide functions for such analysis, then the ease with which a new user can implement them is key. (from my experience with R, tricky package downloads and incomplete documentation are a massive turnoff for new users and you might hear them screaming "I don't want to learn computer science, I want the system to do all that for me!" before you know it!)

Overall

The above is my two cents, I hope that they are a positive contribution. Please let me know if any of them are borne out of misunderstanding. I think the system is great, has huge potential and I've found coding in Python a dream (as opposed to C++)!

*caveat: I'm not a good Python programmer so apologies if some of the twists and turns I make in my code could be completed a lot more elegantly using the various modules available had I known about them. Please highlight anything I've done that is bad/could be better.

Clone Algorithm
248
Loading...
Backtest from to with initial capital
Total Returns
--
Alpha
--
Beta
--
Sharpe
--
Sortino
--
Max Drawdown
--
Benchmark Returns
--
Volatility
--
Returns 1 Month 3 Month 6 Month 12 Month
Alpha 1 Month 3 Month 6 Month 12 Month
Beta 1 Month 3 Month 6 Month 12 Month
Sharpe 1 Month 3 Month 6 Month 12 Month
Sortino 1 Month 3 Month 6 Month 12 Month
Volatility 1 Month 3 Month 6 Month 12 Month
Max Drawdown 1 Month 3 Month 6 Month 12 Month
# Backtest ID: 508426a31706f2183b000750
This backtest was created using an older version of the backtester. Please re-run this backtest to see results using the latest backtester. Learn more about the recent changes.
There was a runtime error.
14 responses

@Tim, great idea for an algorithm, and thank you for the incredibly helpful feedback! You sparked many new ideas for me, thank you. Here are some of my thoughts:

  • I love your question about transforms because it goes to the heart of our approach. We want to provide a broad and common library, while also helping you build your own. We are looking at ta-lib as a way to rapidly accelerate the expansion of our transform library. To let you build your own transforms, we are working on an extension to the execution environment that makes it easy to work with a window of trade data. Lastly, we have opensourced our entire backtest library zipline. The spirit is to help without forcing you to use our kit only.
  • We have heard the same complaint about differing frequency between the IDE backtest and "full". We plan to update the UI so you can control bar length in both places.
    • Our new transform api will let you specify the refresh frequency for your transforms. This will enable you to divorce transforms and other logic from the underlying trade bar length, just as you've proposed. I've written a bit about the forthcoming batch transforms in the end of this thread: https://www.quantopian.com/posts/universe-selection
    • +1 for debugging and step-through. I have been dreaming of adding that kind of functionality to the IDE since the very first prototype. We haven't yet put this into our delivery plan though.
    • I can imagine a drag and drop UI that emits python code compatible with our API, but in my own experience, I've always found that graphical programming languages tend to be too rigid compared to just coding. However, reading our notes it occurred to me that a simple GUI could be a great onramp for a full programming interface - GUI generates python code. New programmers could use it like scaffolding to learn some python and our API, and then hand edit the generated code to get started. Just an idea, what do you think?

again, thank you so much for taking the time to share your algo and your ideas. Both are really interesting and helpful!

thanks,
fawce

Disclaimer

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.

As Fawce said, thanks for the thoughtful ideas here.

On your "bigger question": I think you're asking, are we going to provide a library and let people clone and edit? Or are we going to make a toolset where anyone can quickly implement and test an idea?

I'd like to provide both - I don't see a conflict between the two ideas. They are both a fair amount of work, but they work hand-in-hand. If we can provide a good library and the tools to manipulate code easily, that will unlock a lot of creativity from our members.

Did I understand the question correctly?

Thanks

Dan

For now at least

Disclaimer

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.

@fawce, @Dan, it was a pleasure, I enjoy coding algos and giving feedback: a perfect combo

@fawce - providing a broad common library whilst also allowing users to code their own transforms makes perfect sense. One the one hand, using the provided library would allow those who don't want to code transforms for whatever reason (time, knowledge etc.) to get as deep into building their strategies as possible. And on the other, the ability to customise, tinker, and create your own transforms is maintained.

If you're definitely going to expand the 'default' transforms library, I guess, to some extent, the question becomes one of how that library gets populated. I had a quick look at ta-lib and it looks reasonably deep in terms of technical indicators. I'm not sure how nicely Quantopian plays with R, but you might also want to look at Joshua Ulrich's TTR as a candidate suite of indicators.

(It also occurred to me that if there users out there busying themselves creating their own cool custom transforms, should they wish, you might be want to accept these into the default library - subject to relevant tests around robustness of code etc. - maybe in return for some sort of enhanced standing in the community (or more t-shirts!)… By being arbiter of the 'default' library Quantopian can help with things like argument-naming-consistency (i.e. years == yrs == y).)

I agree wholeheartedly that the drag & drop kind of functionality would be a great helping-held for those new to strategy writing, but it's completely not a method for those more advanced quants who wish to see (& write) what goes on under the hood. In the end, getting such functionality up and running might cost more dev time that it's worth.

I also, read through your post about the batch transforms - sounds like a great step in the right direction.

@Dan - you understood my question exactly, and I think both yours and fawce's reply definitely outline the best approach: provide as many good tools as possible, but also allow users to create their own as freely as they wish. (my only request - as a user - is maybe to think about some very broad optional 'best practice' guidelines (not very open-source, I know - sorry!) for those of us that might seek to develop production-level transforms for use by the community around naming conventions for function args etc, so that reading one user's code is as easy as reading the next user's code…)

Really enjoying using this platform - keep up the great work!

@Tim a small, but hopefully helpful, note, but you can use Python's collections module's deque, http://docs.python.org/library/collections.html#collections.deque, to implement the rolling tick history window that you use in your algorithm.

e.g.

    from collections import deque  
    num_ticks = 6  
    context.tick_history = deque([0] * num_ticks, num_ticks)  

To add a new value all you would need to do is:

    context.tick_history.appendleft(price)  

You might find it a nice little way to remove an extra line of juggling list positions.
It also provides the benefit of better memory performance, etc.

Hope that helps!

Disclaimer

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.

@Eddie - that's great, thanks very much. Think I'll be using that a lot. Really appreciate the feedback.

Hello Tim & all,

As I see it, a big advantage of Quantopian is that they have licensed a historical market dataset and are offering it free. Unfortunately, since they cannot distribute the data, they are constrained to offer limited access to it through their browser-based backtest application. To me, a better approach would be to offer users a flexible virtual/remote desktop environment with full access to view (but not download) the data. Users could then pick their analysis tools (one of which could still be the Quantopian browser-based backtest application). Quantopian folks, would this approach work technically and legally? If so, would it make business sense?

Hi @Grant,

We hear you loud and clear. My perspective is that there are three essential activities for quants:

  1. Research and exploration of source data
  2. Algorithm development, testing, and simulation
  3. Real trading

We started with #2, and we remain focused there. I believe it is critical to reach a minimum level of competency before taking on the next challenge.

I totally agree with the spirit of your idea, and I associate it with #1 above. What I hear you saying is: Give Quantopian users a more flexible environment to do research! That resonates very deeply, and it will be our goal when we turn our focus to #1.

We believe the time for data science tools in the browser has arrived. I wouldn't claim that we have given you the total freedom of analysis you want - but I think web delivery is the best way for us to do it.

Please keep sharing your thoughts and advice, it is helping and inspiring us immensely.

thanks,
fawce

Thanks Fawce,

In my opinion, there is not such a clear distinction between "1. Research and exploration of source data" and "2. Algorithm development, testing, and simulation." I'm not sure what you mean by "web delivery" but from my perspective, flexible virtual/remote desktop access to the data would be preferred over having fixed input/output GUI's in multiple browser tabs. I poked around the web and came across http://guac-dev.org/, which claims to allow "access to multiple desktops through a web browser."

It'd be better to start a separate discussion, but I think this gets back to Lou Belle's questions regarding the business thrust of Quantopian. I imagine that you need to get to "3. Real trading" ASAP so you can make some profits off of fees/commissions, right?

Hi Grant -

I am sure @fawce agrees there is no bright line between #1 and #2. As we talk to people who trade (here, in person, at PyData, through our investors, etc.) we listen to quants talk about their creative process. They start with general ideas, do some whittling down, figure out roughly where their ideas are bearing fruit, and then drill deeply and test exhaustively. We've spent a lot more calories so far on the bottom end of that process, the exhaustive testing. We're spending more calories now on the higher part, the earlier research. We absolutely want it to be an integrated process.

Can you explain to me more about the remote desktop idea? If you remoted to our desktop, it would still be our IDE. How does that work for you? I'm missing something on that one.

Real trading is high on our list, but it's not the be-all end-all. We have enough money in the bank; we're not in a race against the clock. Like any startup, we're doing the dance of limited resources and near-unlimited ideas. We want live trading, better backtester, better IDE, better community, more data sources, open-sourced data sources, more data transforms, open-sourced data transforms, and more. We're listening to our members and our market, prioritizing as best we can, and getting it done as fast as we can.

Thanks Dan,

The remote desktop idea is to give users the experience that they have the data you have licensed at their disposal and could analyze it as they wish (including but not limited to using the Quantopian IDE). Imagine a Linux remote desktop environment, for example, with a suite of programming and analysis tools. You'd need to block the means to transfer data (at least big chunks of it) from the remote desktop, to preclude the licensing issue of enabling re-distribution of the data. Each user would have some disk space on the remote system, as well. So, to the user, the experience would be as if he had purchased the historical database and loaded it onto his hard drive. In this way, very flexible screening, statistical analysis, plotting, visualization, etc. could be performed. Hope this helps...let me know if more clarification is needed.

That definitely makes sense. I think we can deliver all of that in the browser.

For the remote desktop you described, we'd have to examine and modify and whitelist and install each app in that suite of programming tools. We can deliver the same functionality, but faster, by doing it in the browser. We can install all sorts of Python libraries and make them run in the browser, from pandas to matplotlib. So I think we've got a similar vision - the question is if we can do it in the browser like we think we can.

[jumping ahead 1.5 years or so] I'm not spotting anything wrong in the code (aside from my usual pet-peeve against use of pronouns like "we" in comments) and yet those RSI values don't appear to be in the same ballpark with charting sites out there.

RSI calculations in Quantopian do not appear correct

Ok here are a couple of ways to obtain RSI values on Quantopian pretty close to charting sites, TradingView, Google, et al...

Clone Algorithm
108
Loading...
Backtest from to with initial capital
Total Returns
--
Alpha
--
Beta
--
Sharpe
--
Sortino
--
Max Drawdown
--
Benchmark Returns
--
Volatility
--
Returns 1 Month 3 Month 6 Month 12 Month
Alpha 1 Month 3 Month 6 Month 12 Month
Beta 1 Month 3 Month 6 Month 12 Month
Sharpe 1 Month 3 Month 6 Month 12 Month
Sortino 1 Month 3 Month 6 Month 12 Month
Volatility 1 Month 3 Month 6 Month 12 Month
Max Drawdown 1 Month 3 Month 6 Month 12 Month
# Backtest ID: 537543ba05d1340713897113
There was a runtime error.