Below is my first coded backtest using Quantopian*; it's an implementation of the Relative Strength Index (RSI definition), which should be familiar to most who've scratched the surface of first-level technical trading and is often cited as a useful overbought/oversold momentum oscillator. My adoption of this indicator as my first backtest is by no means an endorsement of it usefulness, it was simply a way for me to get used to the IDE with something easy as a subject. (The backtest timeframe below was crudely chosen to show good results and nothing more!)
Thoughts and feedback on the process (apologies in advance if any of this has arisen due to my lack of understanding of the system…still learning!):
In general, I think this is excellent. The area for building up one's backtest is clean and relatively simply to use. (I do worry that for someone who has no, or little coding experience, the thought of having to actually write code to get a backtest to work, as opposed to being able to, say, drag and drop in the way that Quant Blocks works, might actually be a barrier to them using the tool at all…but more on that below**).
My biggest request for improving the IDE would be around being able to step through a program as it executes on real data. Be able to do so, and concurrently seeing the state and value of local variables, would be massively helpful in debugging and understanding, for example, why a particular order is triggered, or not triggered (etc.), at a certain point in time. Without being able to step through the program on real data, I found that debugging my code was a painful process of having the fill the script with LOADS of log.debug stuff, then run a full backtest and try to match up the Transaction Details to the Log Output in order to work out what was going with the strategy, at what point in time, and why. Step-through functionality - even if just on a static dummy set of data - would be huge.
The Backtest process
The disparity between running a backest in the IDE on daily data, and the full backest that runs against minute bars is clunky. Given the full backtest can only be run against minute bars (is this right?), then it is best to code in preparation for this, which means the "daily" backtest in the IDE becomes a bit redundant; I ended up using it purely as a way to ensure the program I was writing had no build errors (is that what it was intended for?).
The availability of more options around "frequency" in general would be a great improvement in my opinion: more options around the frequency at which the strategy is evaluated (i.e. I might like my strategy to be an hourly, or 3-hourly, or monthly strategy - as opposed to the default minute bars - and I cant be bothered (read: I don't know how) to ensure my code does this for me) and more options around the frequency of the various available transforms would be good (i.e. the mavg is currently a daily moving average, but given we're executing against minute data, it might be nice for such transforms to be of the form mavg(input_range,input_freq) with input_freq having options such as minute, hour, day, month etc. such that mavg(5,hour) would give you the 5-hour-moving average). In general, maintaining complete disconnection between the frequency at which the transforms are calculated and the frequency of the strategy evaluation would be desirable.
Which brings me onto the transforms and the bigger question alluded to **above. (I've tried to write the following sentence a few different ways, but none of them sound right, so I'm just going to straight ask it as a question instead)
Is the plan for Quantopian to be the 'custodians' of the transforms that are available to a user as defaults, and increase the suite of transforms as the platform grows? Or is this responsibility to fall to the community in general?
I ask because one of the prime attractions to new users (new users == community good) will be the variety and ease with which they can implement a large suite of (rigorously tested) time series calculations into their various strategies. If there are going to be a wide selection of them available as defaults from Quantopian, then great, but then the current selection needs to widen rapidly beyond the few simple ones currently available, for the platform to be attractive to those users who aren't prepared to code them up themselves.
If, alternatively, the community is to provide functions for such analysis, then the ease with which a new user can implement them is key. (from my experience with R, tricky package downloads and incomplete documentation are a massive turnoff for new users and you might hear them screaming "I don't want to learn computer science, I want the system to do all that for me!" before you know it!)
The above is my two cents, I hope that they are a positive contribution. Please let me know if any of them are borne out of misunderstanding. I think the system is great, has huge potential and I've found coding in Python a dream (as opposed to C++)!
*caveat: I'm not a good Python programmer so apologies if some of the twists and turns I make in my code could be completed a lot more elegantly using the various modules available had I known about them. Please highlight anything I've done that is bad/could be better.
|Returns||1 Month||3 Month||6 Month||12 Month|
|Alpha||1 Month||3 Month||6 Month||12 Month|
|Beta||1 Month||3 Month||6 Month||12 Month|
|Sharpe||1 Month||3 Month||6 Month||12 Month|
|Sortino||1 Month||3 Month||6 Month||12 Month|
|Volatility||1 Month||3 Month||6 Month||12 Month|
|Max Drawdown||1 Month||3 Month||6 Month||12 Month|