Below is my first coded backtest using Quantopian*; it's an implementation of the Relative Strength Index (RSI definition), which should be familiar to most who've scratched the surface of first-level technical trading and is often cited as a useful overbought/oversold momentum oscillator. My adoption of this indicator as my first backtest is by no means an endorsement of it usefulness, it was simply a way for me to get used to the IDE with something easy as a subject. (The backtest timeframe below was crudely chosen to show good results and nothing more!)
Thoughts and feedback on the process (apologies in advance if any of this has arisen due to my lack of understanding of the system…still learning!):
In general, I think this is excellent. The area for building up one's backtest is clean and relatively simply to use. (I do worry that for someone who has no, or little coding experience, the thought of having to actually write code to get a backtest to work, as opposed to being able to, say, drag and drop in the way that Quant Blocks works, might actually be a barrier to them using the tool at all…but more on that below**).
My biggest request for improving the IDE would be around being able to step through a program as it executes on real data. Be able to do so, and concurrently seeing the state and value of local variables, would be massively helpful in debugging and understanding, for example, why a particular order is triggered, or not triggered (etc.), at a certain point in time. Without being able to step through the program on real data, I found that debugging my code was a painful process of having the fill the script with LOADS of log.debug stuff, then run a full backtest and try to match up the Transaction Details to the Log Output in order to work out what was going with the strategy, at what point in time, and why. Step-through functionality - even if just on a static dummy set of data - would be huge.
The Backtest process
The disparity between running a backest in the IDE on daily data, and the full backest that runs against minute bars is clunky. Given the full backtest can only be run against minute bars (is this right?), then it is best to code in preparation for this, which means the "daily" backtest in the IDE becomes a bit redundant; I ended up using it purely as a way to ensure the program I was writing had no build errors (is that what it was intended for?).
The availability of more options around "frequency" in general would be a great improvement in my opinion: more options around the frequency at which the strategy is evaluated (i.e. I might like my strategy to be an hourly, or 3-hourly, or monthly strategy - as opposed to the default minute bars - and I cant be bothered (read: I don't know how) to ensure my code does this for me) and more options around the frequency of the various available transforms would be good (i.e. the mavg is currently a daily moving average, but given we're executing against minute data, it might be nice for such transforms to be of the form mavg(input_range,input_freq) with input_freq having options such as minute, hour, day, month etc. such that mavg(5,hour) would give you the 5-hour-moving average). In general, maintaining complete disconnection between the frequency at which the transforms are calculated and the frequency of the strategy evaluation would be desirable.
Which brings me onto the transforms and the bigger question alluded to **above. (I've tried to write the following sentence a few different ways, but none of them sound right, so I'm just going to straight ask it as a question instead)
Is the plan for Quantopian to be the 'custodians' of the transforms that are available to a user as defaults, and increase the suite of transforms as the platform grows? Or is this responsibility to fall to the community in general?
I ask because one of the prime attractions to new users (new users == community good) will be the variety and ease with which they can implement a large suite of (rigorously tested) time series calculations into their various strategies. If there are going to be a wide selection of them available as defaults from Quantopian, then great, but then the current selection needs to widen rapidly beyond the few simple ones currently available, for the platform to be attractive to those users who aren't prepared to code them up themselves.
If, alternatively, the community is to provide functions for such analysis, then the ease with which a new user can implement them is key. (from my experience with R, tricky package downloads and incomplete documentation are a massive turnoff for new users and you might hear them screaming "I don't want to learn computer science, I want the system to do all that for me!" before you know it!)
The above is my two cents, I hope that they are a positive contribution. Please let me know if any of them are borne out of misunderstanding. I think the system is great, has huge potential and I've found coding in Python a dream (as opposed to C++)!
*caveat: I'm not a good Python programmer so apologies if some of the twists and turns I make in my code could be completed a lot more elegantly using the various modules available had I known about them. Please highlight anything I've done that is bad/could be better.
|Returns||1 Month||3 Month||6 Month||12 Month|
|Alpha||1 Month||3 Month||6 Month||12 Month|
|Beta||1 Month||3 Month||6 Month||12 Month|
|Sharpe||1 Month||3 Month||6 Month||12 Month|
|Sortino||1 Month||3 Month||6 Month||12 Month|
|Volatility||1 Month||3 Month||6 Month||12 Month|
|Max Drawdown||1 Month||3 Month||6 Month||12 Month|
# Put any initialization logic here. The context object will be passed to # the other methods in your algorithm. def initialize(context): #we're going to use the SPY ETF here, but feel free to initiate with #other securities - just change the sid number context.spy = sid(8554) #set min and max portfolio notionals context.max_notional = 1000000.1 context.min_notional = -1000000.0 #the RSI is calculated based on the security's returns over a set number #of preceding observations. Here we're setting up an array to collect the #preceding 6 ticks to allow us to calc the preceding 5 period returns context.tick_history = [0, 0, 0, 0, 0, 0] pass # Will be called on every trade event for the securities you specify. def handle_data(data, context): #set price variable price = data[context.spy].price #on each new tick we will insert the latest price into the array, extending its #lenght by 1. We'll then remove the oldest tick to return the array to its #original length context.tick_history.insert(0,price) context.tick_history.pop() preceding_prices = context.tick_history #if there are enough prices in the array, let's go! if preceding_prices[len(preceding_prices)-1] <> 0: #create and populate an array of returns from our prices returns = [(preceding_prices[i-1]-preceding_prices[i]) for i in range(1,len(preceding_prices))] #create two empty arrays, one for up moves, one for down moves up =  down =  #pass through the array of returns and place them in the corresponding up #and down move arrays for i in range(len(returns)): if returns[i] > 0: up.append(round(returns[i],2)) elif returns[i] <= 0: down.append(round(returns[i]*-1,2)) #handle empty arrays - empty arrays mean no average of returns possible if len(up) == 0: rsi = 0 elif len(down) == 0: rsi = 100 #the meat of the RSI calculation else: av_up = sum(up) / len(up) av_down = sum(down) / len(down) #some extra, probably uneccessary error handling avoidance if av_down == 0: rsi = 100 elif av_up == 0: rsi = 0 #final RSI calculation, setting the statistic to be between 0 and 100 else: rs = av_up / av_down rsi = 100 - (100 / (1 + rs)) #set default RSI for when we dont yet have enough price history else: rsi = 50 #determine current notional of positions notional = data.portfolio.positions[context.spy].amount * price #some local debugging to monitor RSI levels log.debug('RSI is') log.debug(rsi) #very un-imaginative trading strategy based around the observed RSI level # PLEASE IMPLEMENT SOMETHING MORE FUN WITH THIS!! if (rsi > 50) and notional > context.min_notional: order(context.spy,+10) elif (rsi < 50) and notional > context.min_notional: order(context.spy,-10)