Quantopian Overview

Quantopian is a platform for developing and testing quantitative trading algorithms. Our goal is to handle all the infrastructure, data, and setup tasks so that you can focus on developing and testing your investment ideas. Live trading of your algorithms is in a pilot program.

Important Concepts

Writing an algorithm in Quantopian uses two parts of our application: the IDE (interactive development environment) and backtester (testing).

Algorithms are developed in Python in our IDE. With our built-in syntax checking, validation, and smoke tests, the IDE tells you about issues before they crop up in backtesting.

Our backtester runs an event loop once per historical minute. Any calculations or orders your algorithm needs are handled on each loop. At the end of a backtest, all the relevant performance, positions, transactions, and risk data are available to you.

Data sources

We have minute bar historical data for US equities since 2002 up to the most recently completed trading day (data uploaded nightly). SPY, an ETF tracking the S&P 500, is the benchmark used for algorithms simulated in the backtest. It represents the total returns and reinvests the dividends to model the market performance. You can also set your own benchmark in your algorithm.

A minute bar is a summary of the trading activity for a security for a one-minute period, and gives you the opening price, closing price, high price, low price, and trading volume during that minute. Our US equity set is point-in-time, which is important for backtest accuracy. Since our event-based system sends trading events to you serially, your algorithm receives accurate historical data without any bias towards the present.

Our database includes all stocks that traded since 2002, even ones that are no longer traded. This is very important because it helps you avoid survivorship bias in your algorithm. Databases that omit stocks that are no longer traded ignore bankruptcies and other important events, and lead to false optimism about an algorithm. For example, LEH (Lehman Brothers) is a security in which your algorithm can trade in 2008, even though the company no longer exists in 2014; Lehman's bankruptcy was a major event that affected many algorithms at the time.

Our data uses adjusted close prices. That means that the effect of all stock splits and merger activity are applied to price and volume data. As an example: A stock is trading at $100, and has a 2:1 split. The new price is $50, and all past price data is retroactively updated 2:1 (volume data is correspondingly updated 1:2). In effect, that means you can ignore stock splits unless you are storing prices in a live trading algorithm. In a live trading algorithm, the stored variable needs to have the split applied to the price and volume. In a backtest, the split is already applied retroactively to all data.

There are many other financial data sources we'd like to incorporate, such as options, futures, and fundamental data. What kind of data sources would you like us to have? Let us know.

Fetcher - Load any CSV file

Quantopian provides historical data since 2002 for US equities in minute and daily bars. The US market data provides a backbone for financial analysis, but some of the most promising areas of research are finding signals in non-market data. Fetcher provides your algorithm with access to external time series data. Any time series that can be retrieved as a CSV file via http or https can be incorporated into a Quantopian algorithm.

Fetcher lets you load CSV files over http. To use it, invoke fetch_csv(url) in your initialize method. fetch_csv will retrieve the text of the CSV file using the (wonderful) requests module. The CSV is parsed into a pandas dataframe using pandas.io.parsers.read_csv. You may then specify your own methods to modify the entire dataframe prior to the start of the simulation. During simulation, the rows of the CSV/dataframe are streamed to your algorithm's handle_data method as additional properties of the data parameter.

Best of all, your Fetcher data will play nicely with Quantopian's other data features:

  • Your data will be streamed to your algo without look-ahead bias.
  • Use record to plot a time series of your fetcher data.
  • Create a batch_transform and make statistical models using your fetcher data.

Fetcher supports two kinds of time series:

  • Security Information: data that is about individual securities, such as short interest for a stock
  • Signals: data that stands alone, such as the Consumer Price Index, or the spot price of palladium

For Security Info, your CSV file must have a column with header of symbol which represents the symbol of that security on the date of that row. Internally, Fetcher maps the symbol to the Quantopian security id (sid). You can have many symbols in a single CSV file. However, any symbol that is not also initialized (either using sid() or set_universe()) is discarded. To access your CSV data in handle_data:

## This algo imports sample short interest data from a CSV file for one security, 
## NFLX, and plots the short interest:

def initialize(context):  
    # fetch data from a CSV file somewhere on the web.
    # Note that one of the columns must be named 'symbol' for 
    # the data to be matched to the stock symbol
    fetch_csv('https://dl.dropboxusercontent.com/u/169032081/fetcher_sample_file.csv', 
               date_column = 'Settlement Date',
               date_format = '%m/%d/%y')  
    context.stock = sid(23709)
    
def handle_data(context, data):    
    if 'Days To Cover' in data[context.stock]:        
        record(Short_Interest = data[context.stock]['Days To Cover'])

Here is the sample CSV file. Note that for the Security Info type of import, one of the columns must be 'symbol'.

Settlement Date,symbol,Days To Cover
9/30/13,NFLX,2.64484
9/13/13,NFLX,2.550829
8/30/13,NFLX,2.502331
8/15/13,NFLX,2.811858
7/31/13,NFLX,1.690317

For Signals, your CSV file does not need a symbol column. Instead, you provide it via the symbol parameter:

def initialize(context):
    fetch_csv('http://yourserver.com/cpi.csv', symbol='cpi')

def handle_data(context, data):
    # get the cpi for this date
    current_cpi = data['cpi']['value']

    # plot it
    record(cpi=current_cpi)

Using Dropbox

Many users find Dropbox to be a convenient way to access CSV files. To use Dropbox, you need to use the 'Public URL' provided by Dropbox. A common mistake is to use a URL of format https://www.dropbox.com/s/abcdefg/filename.csv, which is a URL about the file, not the file itself. Instead, you should use the Public URL which has a format similar to https://dl.dropboxusercontent.com/u/1234567/filename.csv.

Data Manipulation with Fetcher

If you produce the CSV, it is relatively easy to put the data into a good format for Fetcher. First decide if your file should be a signal or security info source, then build your columns accordingly.

However, you may not always have control over the CSV data file. It may be maintained by someone else, or you may be using a service that dynamically generates the CSV. Quandl, for example, provides a REST API to access many curated datasets as CSV. While you could download the CSV files and modify them before using them in Fetcher, you would lose the benefit of the nightly data updates. In most cases it's better to request fresh files directly from the source.

Fetcher provides two ways to alter the CSV file:

  • pre_func specifies the method you want to run on the pandas dataframe containing the CSV immediately after it was fetched from the remote server. Your method can rename columns, reformat dates, slice or select data - it just has to return a dataframe.
  • post_func is called after Fetcher has sorted the data based on your given date column. This method is intended for time series calculations you want to do on the entire dataset, such as timeshifting, calculating rolling statistics, or adding derived columns to your dataframe. Again, your method should take a dataframe and return a dataframe.

Live Trading and Fetcher

When Fetcher is used in Live Trading or Paper Trading, the fetch_csv() command is invoked once per trading day, when the algorithm warms up, before market open. It's important that the fetched data with dates in the past be maintained so that warm up can be performed properly; Quantopian does not keep a copy of your fetched data, and algorithm warmup will not work properly if past data is changed or removed. Data for 'today' and dates going forward can be added and updated.

Working With Multiple Data Frequencies

Without Fetcher data, all Quantopian simulations operate on datasources with identical frequencies. If you choose a minutely simulation, your handle_data function is called exactly once per (simulation) minute. If you choose daily, your function is similarly called once per day.

When pulling in external data, you need to be careful about the data frequency to prevent look-ahead bias. If you are fetching daily data, but are running a minutely-based algorithm, the daily row will be fetched at the beginning of the day, instead of the end of day. To guard against the bias, you need to use the post_func function. For more information see post_func API documentation or take a look at this example algorithm.

With Fetcher, you will need to have some guard code in your handle_data methods to avoid KeyError problems accessing data properties. In general, you need to check that a sid is in data before you access it, and that a property is available on a sid before you access it.:

def initialize(context):
    # a signal Fetcher
    fetch_csv('http://priceoftea.com/', symbol='tea')
    # a security info Fetcher
    fetch_csv('http://insiderselling.com')

def handle_data(context, data):
    # guard against being called before the first trade of a security
    if sid(123) in data:
        # guard against trades happening before the first insider selling event
        if 'insider' in data[sid(123)]:
            if data[sid(123)]['insider'] > 10.0:
                order(sid(123), -100)

    # signal data will pass a blank place holder if the first event has not been sent yet.
    # So, you can just guard against missing properties
    if 'price' in data['tea']:
        record(price_of_tea=data['tea']['price'])

For more information about Fetcher, go to the API documentation or look at the sample algorithms.

Paper Trading

Paper trading is also sometimes known as walk-forward testing. In paper trading, your algorithm gets live market data (actually, 15-minute delay data) and 'trades' against the live data with a simulated portfolio. This is a good test for any algorithm. If you inadvertently did overfitting during your algorithm development process, paper trading will often reveal the problem. You can simulate trades for free, with no risks, against the current market on Quantopian.

Quantopian paper trading uses the same order-fulfillment logic as a regular backtest, including the slippage model.

Before you can paper trade your algorithm, you must run a minute backtest. Go to your algorithm, choose the minute backtest option, and click the 'Full Backtest' button. Once that backtest is complete, you will see a 'Live Trade Algorithm' button. Also, on the My Algorithms list you will see a Start Trading button for any backtested algorithm. When you start paper trading, you will be prompted to specify the amount of money used in the strategy. Note that this cash amount is not enforced - it is used solely to calculate your algorithm's returns.

Live Trading

If you have received an invitation to join the live trading pilot program, when you click the Start Trading button, you will have the option to choose 'Broker'.

You will need to authenticate to your Interactive Brokers* account. (Note: Quantopian does not store your brokerage password.) For more information about live trading and how to create an IB account, see our live trading FAQ.

We strongly recommend that you run your algorithm against IB's paper trading mode before running it against a real money account. IB paper trading is a good test to help find lingering bugs. Once you're satisfied with the paper trading performance, you should stop the live algorithm and re-launch it against your real money IB account.

We are using IB's Smart Order Router algorithm for order execution. Feedback is welcome if another execution algo is preferred.

If you manually stop a live trading algorithm, this will shutdown the algorithm and prevent it from placing additional orders. This will not liquidate your portfolio and any open order may still be filled.

Quantopian's live trading program is still in a pilot mode, and the live trading platform has a number of limitations. These limitations will be reduced and removed as we keep improving the live trading platform.

  • The live trading platform does not have the capability to do a mid-day recovery from an outage. If your algo goes down mid-day (whether it's because of our bug, or your bug, or a third party issue) the algorithm won't automatically restart mid-day. If the bug was temporary, the algorithm will start the next day at market open. On restart, the data windows (batch_transform and simple transforms) will be filled with the "missing" data that passed during the previous day's outage.
  • There is no automatic liquidation on algorithm exit. If you need to liquidate your holdings, you need to do that programatically in your algorithm or in the Interactive Brokers interface.
  • If your IB login is terminated, you will need to log back in before trading continues. This could happen unexpectedly because of a server error, or it could be a planned code upgrade on our part. If a login is necessary we will contact you.
  • All open orders are cancelled at end-of-day.
  • Orders made in the last minute of the trading day will be automatically cancelled.
  • If your algorithm uses fetch_csv(), it is important that all historical data in your fetched data be accessible and unchanged. We do not keep a copy of fetched data; it is reloaded at the start of every trading day. If historical data in the fetched file is altered or removed, the algorithm will not run properly. Providing additional, new data in the file for dates in the future is fine.
  • The use of random() isn't supported because of technical limitations with our start-of-day initialization.
  • IB only allows one login at a time per account. If you try to log into IB during market hours using the same account as Quantopian, Quantopian will automatically try to force you out so that we can keep trading on your behalf. In general, we recommend that you maintain two logins, one for Quantopian and one for your use. If you need to log in to IB using the Quantopian login during market hours, you should first stop the algorithm in Quantopian.
  • The pilot program is limited to IB accounts with a portfolio value of less than $100,000.
  • The pilot program is limited to accounts based in US dollars. The account can be funded by other currencies, but the base must be US dollars.

We are working very hard at testing and improving our software; however, we certainly have software bugs. As members of the pilot program, in particular, you need to be alert to any bugs that might affect your trading. Bug reports are greatly appreciated and we will try to fix them as quickly as possible.

* Interactive Brokers LLC is not affiliated with and does not endorse or recommend Quantopian, Inc. Interactive Brokers provides execution and clearing services to customers who integrate their Interactive Brokers account with their Quantopian account. For more information regarding Interactive Brokers LLC, please visit www.interactivebrokers.com.

Privacy and Security

We take privacy and security extremely seriously. All trading algorithms and backtest results you generate inside Quantopian are owned by you. Unless you choose to share your content, your content will remain private. We will never access your data unless we are helping you troubleshoot issues.

Our databases live in Amazon's cloud infrastructure.

Specifically, our security plans include:

  • Never storing your password in plaintext in our database
  • SSL encryption for the Quantopian application
  • Secure websocket communication for the backtest data
  • Encrypting all trading algorithms before writing them to our database

We want to be very transparent about our security measures. Don't hesitate to email us with any questions or concerns.

Developing in the IDE

Quantopian's Python IDE is where you develop your trading ideas. The standard features (autosave, fullscreen, font size, color theme) help make your experience as smooth as possible.

Your work is automatically saved every 10 seconds, and you can click Save to manually save at any time. We'll also warn you if you navigate away from the IDE page while there's unsaved content.

You can invite other people to collaborate with you on an algorithm. Click the "Collaborate" button in the IDE to begin working with your friends and other Quantopian members. They will receive an email invitation to join your algorithm, and non-members will have to register before they can collaborate. You can chat with each other in the chat tab, and watch the code change in real time as each collaborator edits the strategy. Any collaborator can build the algorithm to see the backtest performance.

Highlights of collaboration:

  • This feature is in beta. Please let us know about any problems you see, or where the feature needs improvement.
  • There isn't a technical limit on how many collaborators you can have, but there is a practical limit. The more collaborators you have, the more likely that you'll notice a performance problem. We'll improve that in the future.
  • The algorithm owner has permissions that the invited collaborators do not. Only the owner can invite other collaborators, deploy the algorithm for live trading, or delete the algorithm.
  • If you don't know a member's email address, you can search their profile in the forums and send them a private user-to-user message. If they choose to share their email address with you, then you can invite them to collaborate.

API Overview

We have a simple but powerful API for you to use:

initialize(context) is the setup method for initializing state or other bookkeeping. This method is called only once at the beginning of your algorithm. context is an augmented Python dictionary used for holding state between methods. Properties can be accessed using dot notation (context.some_property).

handle_data(context, data) is called on every trading event. data is a dictionary containing your universe at that time, keyed by security ids. For example, data[sid(26578)] gets you the latest data for Google. Our backend will find all referenced securities in the algorithm and only send their trading events to this method.

Your algorithm can have other Python methods. For example, you might have handle_data call a utility method to do some calculations.

For more information, jump to the API Documentation section below.

Referencing securities

All securities have a unique security id (SID) in our system. Since symbols may be reused among exchanges, this prevents any confusion and verifies you are calling the desired security. For example G used to refer to Gilette but now refers to Genpact. Your algorithm can reference a security directly by its id, or use our sid method to look up a security by its id, symbol, or name. Using the sid method brings up a search box that shows you the top results for your search.

Help-sid

In other words, sid(24) and 24 are equivalent, and both refer to Apple.

Setting a custom benchmark

The default benchmark in your algorithm is SPY, an ETF that tracks the S&P 500. You can change it in the initialize function by using set_benchmark and passing in another security. Only one security can be used for the benchmark, and only one benchmark can be set per algorithm. If you set multiple benchmarks, the last one will be used.

Here's an example:

def initialize(context):
  set_benchmark(sid(2673)) # Tesla

def handle_data(context, data):
  ...

Checking that trade data exists for a security

Our backtester protects you from running an algorithm against securities that don't exist yet. For instance, if you try to backtest using Facebook, the backtest will fast-forward to the point where Facebook is traded. If you try to backtest both Lehman and Facebook you will get an error since Facebook wasn't traded until May 2012, many years after Lehman stopped trading.

Stocks do not necessarily trade every minute or even every day. This is especially true for thinly-traded securities, so your algorithm needs to be robust to cover these edge cases.

To check if trade data exists for a security during a given trading period, just check if it exists in the data structure that is passed to handle_data. For example, if sid in data: ... Your algorithm will check to see if the stock has data in the bar. Otherwise, it will skip the stock and continue with your algorithm calculations without receiving an error.

If your algorithm has multiple stocks, it's possible that in a trade bar, one stock has trade data while another does not. By default, your algorithm is set to set_nodata_policy(NoDataPolicy.LOG_ONLY), which logs the error, skips over the stock with no data, and exits the handle_data call. In backtesting and live trading, if you try to access trade information for a stock that had no data, the algorithm will log the information and continue running.

If you prefer to stop the algorithm entirely when you try to access missing trade data, you can use set_nodata_policy(NoDataPolicy.EXCEPTION) in initialize. Trying to access missing trade data will result in a NoTradeDataAvailableException, ending the backtest or live algorithm.

Validation

Our IDE has extensive syntax checking and validation to make sure that your algorithm is valid Python, fulfills our API, and has no obvious runtime exceptions (such as dividing by zero). You can run the validation checks by clicking on the Build button (or pressing control-B), and we'll run them automatically right before starting a new backtest.

Errors and warnings are shown in the window on the right side of the IDE. When possible, we underline the problem for you. Here's an example, where there's a missing period before vwap(30):

Help-validation

When all errors and warnings are resolved, the Build button kicks off a quick backtest. The daily data is less detailed than the second bar data, but it runs much more quickly. The quick backtest is a way to make sure that the algorithm roughly does what you want it to, without any errors.

Once the algorithm is running roughly the way you'd like, click the 'Full Backtest' button to kick off a full backtest with minute-bar data.

Ordering

Call order(sid, amount) to place a simple market order. sid is the security you wish to trade, and the amount is the number of shares you want to buy. Use a negative amount in order to sell. The method returns an order id that can be used to track the order's status. The FAQ has more detailed information about how orders are handled and filled by the backtester.

Quantopian supports four different order types:

  • market order: order(context.sid, amount) will place a simple market order.
  • limit order: Use order(context.sid, amount, limit_price=price) to place a limit order. A limit order executes at the specified price or better, if the price is reached.
  • stop order: Call order(context.sid, amount, stop_price=price) to place a stop order (also known as a stop-loss order). When the specified price is hit, the order converts into a market order.
  • stop-limit order: Call order(context.sid, amount, limit_price=price1, stop_price=price2) to place a stop-limit order. When the specified stop price is hit, the order converts into a limit order.

All orders are good-til-cancelled, except for algorithms that are connected to a brokerage. Orders placed with a brokerage (both paper trading and real-money trading) are cancelled at end-of-day. In the future we will most likely change backtest order behavior to also cancel on end-of-day, to make the backtester more accurate.

You can see the status of a specific order by calling get_order(order). Here is a code example that places an order, stores the order_id, and uses the id on subsequent calls to log the order amount and the amount filled:

# place a single order at market open.  
if exchange_time.hour ==9: and exchange_time.minute >=30:  
    context.order_id = order_target(context.aapl, 1000000)  
 
# retrieve the order placed in the first bar  
aapl_order = get_order(context.order_id)  
if aapl_order:  
    # log the order amount and the amount that is filled  
    message = 'Order for {amount} has {filled} shares filled.'  
    message = message.format(amount=aapl_order.amount, filled=aapl_order.filled)  
    log.info(message)

You can see a list of all open orders by calling get_open_orders(). This example logs all the open orders across all securities:

# retrieve all the open orders and log the total open amount  
    # for each order  
    open_orders = get_open_orders()  
    # open_orders is a dictionary keyed by sid, with values that are lists of orders.  
    if open_orders:  
        # iterate over the dictionary  
        for security, orders in open_orders.iteritems():  
            # iterate over the orders  
            for oo in orders:  
                message = 'Open order for {amount} shares in {stock}'  
                message = message.format(amount=oo.amount, stock=security)  
                log.info(message)

If you want to see the open orders for a specific stock, you can specify the security like: get_open_orders(sid=sid). Here is an example that iterates over open orders in one stock:

# retrieve all the open orders and log the total open amount  
    # for each order  
    open_aapl_orders = get_open_orders(context.aapl)  
    # open_aapl_orders is a list of order objects.  
    # iterate over the orders in aapl  
    for oo in open_aapl_orders:  
        message = 'Open order for {amount} shares in {stock}'  
        message = message.format(amount=oo.amount, stock=security)  
        log.info(message)

Orders can be cancelled by calling cancel_order(order). Orders are cancelled asynchronously.

Logging

Your algorithm can easily generate log output by using the log.error (and warn, info, debug) methods. Log output appears in the right-hand panel of the IDE or in the backtest result page.

Logging is rate-limited (throttled) for performance reasons. The basic limit is two log messages per call of initialize and handle_data. Each backtest has an additional buffer of 20 extra log messages. Once the limit is exceeded, messages are discarded until the buffer has been emptied. A message explaining that some messages were discarded is shown.

Two examples:

  • Suppose in initialize you log 22 lines. Two lines are permitted, plus the 20 extra log messages, so this works. However, a 23rd log line would be discarded.
  • Suppose in handle_data, you log three lines. Each time handle_data is called, two lines are permitted, plus one of the extra 20 is consumed. Thus, 20 calls of handle_data are logged entirely. On the 21st call two lines are logged, and the last line is discarded. Subsequent log lines are also discarded until the buffer is emptied.

Additionally, there is a per-member overall log limit. When a backtest causes the overall limit to be reached, the logs for the oldest backtest are discarded.

Event Properties

The data object in handle_data holds data for all the securities your algorithm references. Each security has several trading event properties:

Help-properties

datetime is the Python datetime object representing the timestamp of last trade of this security. Trade data is provided in UTC.

price is the price of this security at the end of this event. Most code within Quantopian operates on this property; it is the property used by default to describe the price of a security.

volume is the volume of shares traded during this event.

open_price is the price of the security at start of this event.

close_price is the price of the security at end of this event. identical to price.

high is the highest price traded during this event.

low is the lowest price traded during this event.

(mavg, returns, and vwap are transforms, which are explained in the next section.)

Simple Transforms

In addition to event properties, we provide a number of transforms so that you don't have to calculate them yourself. They're listed in the dropdown in the event properties screenshot above, but additional contextual information is shown when you use them:

Help-transforms

mavg(days) calculates the moving average of the security's price over the given number of trading days.

returns() calculates the returns since the end of the previous trading day for the specified security.

stddev(days) calculates the standard deviation of the security's price over the given number of trailing trading days. Standard deviation is calculated using Bessel's Correction. Note that mathematically there is no standard deviation for a single data point, so this function returns 'None' until the second data point.

vwap(days) calculates the volume-weighted average price over the given number of trading days.

Note that simple transforms don't have a warmup period in backtesting. As an example, if your algorithm uses mavg(3), the window for the moving average won't be filled until after the third day. The first day mavg(3) will report the first day's price, the second day will report the average of the first two days, etc. However, in paper trading and live trading the simple transforms are warmed up.

You can use the history function with rolling transforms to achieve the same calculations with faster results. These will warm up and speed up your time backtesting.

There are many other transforms we could add - which ones would you like? Let us know.

Recording and plotting variables

When backtesting, you can create time series charts by using the record method and passing series names and corresponding values using keyword arguments. Up to five series can be recorded and charted. Recording is done at day-level granularity. Recorded time series are then displayed in a chart below the performance chart.

A simple, contrived example:

def initialize(context):
    pass

def handle_data(context, data):
    # track the 20-day moving average for GOOG and AAPL
    record(goog_mavg=data[sid(26578)].mavg(20), aapl_mavg=data[sid(24)].mavg(20))

In the result for this backtest, you'll get something like this:

Help-record

Some notes:

  • Because record uses keyword arguments, variables cannot be used for series names.
  • For each series, the value at the end of each trading day becomes the recorded value for that day. This is important to remember when backtesting with minute-level data.
  • Until a series starts being recorded, it will have no value. If record(new_series=10) is done on day 10, new_series will have no value for days 1-9.
  • Each series will retain its last recorded value until a new value is used. If record(my_series=5) is done on day 5, every subsequent day will have the same value (5) for my_series until a new value is given.

To see more, check out the example algorithm at the end of the help documentation.

Set Universe

If your algorithm manually picks a few specific stocks to study, you have some bias risk in your study. To avoid this kind of bias in your choice of stocks, we provide a tool that selects a universe of stocks for you. Rather than enumerate specific securities, you provide criteria for choosing securities. This helps you avoid look-ahead bias and survivorship bias, because the criteria will be applied to the companies as they existed at the point-in-time of your simulation, rather than current values.

A universe is a set of securities defined by certain criteria. Periodically, we re-apply the criteria to update the set of qualifying securities. As a result, some securities might be added to the universe while others might be removed.

To set the universe, use the set_universe(YourUniverse) method in your algorithm's initialize method.

We currently provide a single universe implementation, DollarVolumeUniverse, which uses a percentile range for dollar volume of shares traded for securities. More explicitly, we take the trading volume of each stock and multiply it by the price of each stock to calculate the dollar volume. We then rank each stock by dollar volume, from #1 to (roughly) #8000 and convert that ranking into a percentile

To use the DollarVolumeUniverse, set the universe in the initialize method. You can set a universe interval up to 2% for backtests in minute mode or up to 10% in daily mode. Each percentile in the range will result in ~80 securities, so this universe example would consist of the ~160 securities with the highest dollar-volume volume, with changes coming on the quarter end:

set_universe(universe.DollarVolumeUniverse(floor_percentile=98.0,ceiling_percentile=100.0))

The DollarVolumeUniverse updates quarterly on the last trading day of March, June, September, and December. On these days, we use the dollar volume rank of the previous quarter to re-rank the securities. The quarterly boundaries have some specific behaviors that should be noted:

  • If a stock is not present at the beginning of a quarter (due to an IPO for instance), it is excluded from the universe for that quarter. A stock that stops trading mid-quarter (bankruptcy, delisting, etc.) is kept in the universe with a forward-fill using the last recorded price.
  • At the end of a quarter if your algorithm has a position in a stock, that stock is held in the universe even if it would have otherwise fallen out.
  • When a new stock appears in a universe with a batch transform, the historical data for that stock is loaded. For example, if you have a batch transform with a refresh window of 5 days, and a new stock is added at the quarterly refresh on March 31, then the batch transform loads the stock data back to March 26 and fills the batch transform window.
  • When a new stock appears in a universe and mavg(), vwap(), returns(), and stddev() are called, the data is not backfilled. For example, if you have a 5-day moving average and a new stock is added at the quarterly refresh on March 31, then the moving average only has one day of data on April 1.

If you also refer to specific securities using the `sid` function, those sids will always be included in the universe.

All sids are accessible via data in handle_data. For example:

def initialize(context):
    set_universe(universe.DollarVolumeUniverse(floor_percentile=98.0, ceiling_percentile=100.0))
    context.other_sids = [sid(105), sid(106)]  # add some specific securities

def handle_data(context, data):
    for sid in data:
        log.info(str(sid)) # this will print out all the securities in the universe as well as 105 and 106

Dividends

The Quantopian database holds over 150,000 dividend events dating from January 2002. While other corporate events, such as splits and mergers, are handled by adjusting historical prices and volumes, dividends are more complex. Dividends are treated as events and streamed through the performance tracking system that monitors your algorithm during a backtest. Dividend events modify the security price and the portfolio's cash balance.

Dividends specify four dates:

  • declared date is the date on which the company announced that the dividend.
  • record date is the date on which a shareholder must be recorded as an owner to receive a dividend payment. Because settlement can take 3 days, a second date is used to calculate ownership on the record date.
  • ex date is 3 trading days prior to the record date. If a holder sells their stock before this date, they are not paid the dividend. The ex date is when the price of the stock is typically most affected.
  • pay date is the date on which a shareholder receives the cash for a dividend.

Stock prices are marked down by the dividend amount on the open following the ex_date. The portfolio's cash position is increased by the amount of the dividend on the pay date. Quantopian chose this method so that cash positions are correctly maintained, which is particularly important when an algorithm is used for live trading. The downside to this method is that this can create a noticeable discontinuity in price history and a short portfolio value.

In order for your algorithm to receive dividend cash payments, you must have a long position (positive amount) in the stock as of the close of market on the trading day prior to the ex_date AND you must run the simulation through the pay date, which is typically about 60 calendar days later.

If you are short the stock at market close on the trading day prior to the ex_date, your algorithm will be required to pay the dividends due. As with long positions, the cash balance will be debited by the dividend payments on the pay date. This is to reflect the short seller's obligation to pay dividends to the entity that loaned the stock.

Special dividends (where more than 25% of the value of the company is involved) are not yet tracked in Quantopian.  There are several hundred of these over the last 11 years.  We will add these dividends to our data in the future.

Dividends are not relayed to algorithms as events that can be accessed by the API; we will add that feature in the future.

History

Data History

In many strategies, it is useful to compare the most recent bar data to previous bars. The Quantopian platform provides utilities to easily access and perform calculations on recent history.

The following code provides access to the last 20 days of price history for a universe. Specifically, this returns the closing daily price for the last 20 days, including the current price for the current day:

def initialize(context):
    set_universe(universe.DollarVolumeUniverse(90.0, 90.1))

def handle_data(context, data):
    price_history = history(bar_count=20, frequency='1d', field='price')

At this time, history is only supported for backtests in minute mode (minute bars), not backtests in daily mode (daily bars). You will need to run your backtest in minute mode but will receive trade data in daily bars. To receive minute trade data, see batch_transform().

Specifying bar_count and frequency

The bar_count specifies the number of days to include in the DataFrame returned by the history function.

Below are examples of code along with explanations of the data returned.

Day Units

  • "d" suffix to the frequency parameter
  • returns are always in daily bars and bars never span more than one trading day
  • examples:
    • history(1, "1d", "price") returns the current price.
    • history(1, "1d", "volume") returns the volume since the current day's open, even if it is partial.
    • history(2, "1d", "price") returns yesterday's close price and the current price.
    • history(6, "1d", "price") returns the prices for the previous 5 days and the current price.
  • Partial trading days are treated as a single day unit. Scheduled half day sessions, unplanned early closures for unusual circumstances, and other truncated sessions are each treated as a single trading day.

The frequency parameter "1d" specifies how often the data is sampled. At this time, "1d" is the only supported frequency.

Specifying Field

The history functions return a pandas DataFrame populated with the values for the field specified in the third parameter.

For example, history(bar_count=2, frequency='1d', field='price') returns a DataFrame populated with the price information for each stock in the universe:

The available options for the field parameter are:

  • open_price
  • high
  • low
  • close_price
  • price
  • volume

Returned Data

The returned data for daily history, for each day, is:

  • close_price: the close of the last minute bar for that day. For the current day, the most recent close is returned.
  • price: same as close_price.
  • open_price: the open of the first minute bar of the given day.
  • volume: the sum of all minute volumes. For the current day, the sum of volume thus far.
  • high: the max of the minute highs for the given day.
  • low: the min of the minute bar lows for the given day.

If the data source had minute-bar closing prices for a stock XYZ as such:

                        XYZ
2013-09-05 20:59:00     17.0
2013-09-05 21:00:00     19.0
2013-09-06 14:31:00     20.0
2013-09-06 14:32:00     18.0

The following call to history would output the following results:

history(bar_count=2, frequency='1d', field='price')

At 2013-09-06 14:31 UTC:

                       XYZ
2013-09-05 21:00:00    19.0
2013-09-06 14:31:00    20.0

One minute later, at 2013-09-06 14:32 UTC, notice that the timestamp and data for the previous day's data stays the same, but the current day's value updates to the current algo time and value for price.

                       XYZ
2013-09-05 21:00:00    19.0
2013-09-06 14:32:00    18.0

History and Backtest Start

The full history data panel is available on the first day of the backtest, so there is no need to 'warm up' the history DataFrame. The data is backfilled so that calculations can be done starting with the first call to handle_data.

Obviously, we can't load history where the requested data extends farther into the past than our database. The database limit is Jan 2, 2002 for backtesting and Mar 1, 2012 for paper trading and live trading.

Illiquidity and Forward Filling

By default, history methods will forward fill missing bars. If there is trade data missing for a security one day, the function will use the previous known price until there is new trade data.

However, it can be useful to have visibility into gaps in the trading history. We provide an option for the history methods, which disables forward filling. e.g., history(bar_count=15, frequency='1d', field='price', ffill=False) will return a DataFrame that has nan value for price for days on which a given security did not trade.

Common Usage: Current and Previous Bar

A common use case is to compare yesterday's close price with the current price.

This example compares yesterday's close (labeled prevbar) with the current price (labeled currbar) and places an order for 20 shares if the current price is above yesterday's closing price.

def initialize(context):
    set_universe(universe.DollarVolumeUniverse(90.0, 90.1))

def handle_data(context, data):
    price_history = history(bar_count=2, frequency='1d', field='price')
    for s in data:
        prev_bar = price_history[s][-2]
        curr_bar = price_history[s][-1]
        if curr_bar > prev_bar:
            order(s, 20)

Common Usage: Looking Back X Bars

It can also be useful to look further back into history for a comparison. Computing the percent change over given historical time frame requires the starting and ending price values only, and ignores intervening prices.

The following example operates over all stocks available in data, in pandas Series format to arrive at the percent change over the past 10 trading days.

def initialize(context):
    set_universe(universe.DollarVolumeUniverse(90.0, 90.1))

def handle_data(context, data):
    prices = history(bar_count=10, frequency='1d', field='price')
    pct_change = (prices.ix[-1] - prices.ix[0]) / prices.ix[0]
    log.info(pct_change)

Alternatively, leveraging the following iloc pandas DataFrame function, which returns the first and last values as a pair:

price_history.iloc[[0, -1]]

The percent change example can be re-written as:

def initialize(context):
    set_universe(universe.DollarVolumeUniverse(90.0, 90.1))

def handle_data(context, data):
    price_history = history(bar_count=10, frequency="1d", field='price')
    pct_change = price_history.iloc[[0, -1]].pct_change()
    log.info(pct_change)

The difference code example in the Current and Previous Bar section can also be written as:

def initialize(context):
    set_universe(universe.DollarVolumeUniverse(90.0, 90.1))

def handle_data(context, data):
    price_history = history(bar_count=2, frequency="1d", field='price')
    diff = price_history.iloc[[0, -1]].diff()
    log.info(diff)

Common Usage: Rolling Transforms

Rolling transform calculations such as mavg, stddev, etc. can be calculated via methods provided by pandas.

Common Usage: Standard Deviation

def initialize(context):
    set_universe(universe.DollarVolumeUniverse(90.0, 90.1))

def handle_data(context, data):
    price_history = history(bar_count=5, frequency='1d', field='price')
    log.info(price_history.std())

Common Usage: Moving Average

def initialize(context):
    set_universe(universe.DollarVolumeUniverse(90.0, 90.1))

def handle_data(context, data):
    price_history = history(bar_count=5, frequency='1d', field='price')
    log.info(price_history.mean())

Common Usage: VWAP

def initialize(context):
    set_universe(universe.DollarVolumeUniverse(90.0, 90.1))

def vwap(prices, volumes):
    return (prices * volumes).sum() / volumes.sum()

def handle_data(context, data):
    prices_a = history(15, '1d', field='price')
    volumes_a = history(15, '1d', field='volume')

    prices_b = history(30, '1d', field='price')
    volumes_b = history(30, '1d', field='volume')

    vwap_a = vwap(prices_a, volumes_a)
    vwap_b = vwap(prices_b, volumes_b)

    for s in data:
        if vwap_a[s] > vwap_b[s]:
            order(s, 50)

Common Usage: Using An External Library

Since history returns a pandas DataFrame, the values can then be passed libraries that operate on numpy and pandas data structures.

An example OLS strategy:

import statsmodels.api as sm

def ols_transform(prices, sid1, sid2):
    """
    Computes regression coefficient (slope and intercept)
    via Ordinary Least Squares between two SIDs.
    """
    p0 = prices[sid1]
    p1 = sm.add_constant(prices[sid2], prepend=True)
    return sm.OLS(p0, p1).fit().params

def initialize(context):
    context.sid1 = sid(4283) # KO
    context.sid2 = sid(5885) # PEP

def handle_data(context, data):
    price_history = history(bar_count=30, frequency='1d', field='price')
    slope, intercept = ols_transform(price_history, context.sid1, context.sid2)

Common Usage: Using TA-Lib

Since history returns a pandas DataFrame, a Series can be extracted and then passed to TA-Lib.

An example EMA calculation:

# Python TA-Lib wrapper
# https://github.com/mrjbq7/ta-lib
import talib

def initialize(context):
    context.my_sid = sid(24)

def handle_data(context, data):
    price_history = history(bar_count=30, frequency='1d', field='price')
    my_sid_series = price_history[context.my_sid]
    ema_result = talib.EMA(my_sid_series)
    record(ema=ema_result[-1])

Batch Transforms

Often, you will want to operate on a trailing window of minutely data. The work involved in creating a trailing window of data is so common, Quantopian provides a helper facility called batch_transform.

batch_transform is available in the coding environment as a python function decorator. To use it, you define a function that accepts a datapanel as its first argument, mark it with the batch_transform decorator, and then invoke the function from within handle_data using data as the first parameter. The batch_transform decorator transforms the data parameter sent to your function: data are accumulated into a pandas datapanel.

Here is a simple example that calculates the 10-day average price for all securities, running in a daily bar simulation:

# Here we create a universe of securities based on the top 0.1% most liquid by
# dollar volume traded. Approximately 7 individual securities will be used
# by the algorithm.
def initialize(context):
  set_universe(universe.DollarVolumeUniverse(99.9, 100.0))

# This method expects to receive a datapanel containing dataframes for price,
# volume, open_price, close_price, high, low, and volume. Each dataframe will
# have ten rows (1 per trading day), and about 7 columns (one for each
# stock). window_length is always in trading days.
@batch_transform(window_length=10)
def get_averages(datapanel):
  # get the dataframe of prices 
  prices = datapanel['price']
  # return a dataframe with one row showing the averages for each stock.
  return prices.mean()

def handle_data(context, data):
  # here is the magic part. We invoke the get_averages method using 
  # just the single data (a dictionary of daily bars indexed by security).
  # the decorator will convert this to a datapanel spanning the 10 day history
  # that we specified in with the window_length parameter in the decorator 
  # above.
  averages = get_averages(data)
  # add a newline to the beginning of the log line so that the column header of the 
  # is properly indented.
  log.info('\n%s' % averages) 

Minute Data in batch_transform, Rolling Updates, and refresh_period

The same code above could also be run in minutely mode. In that case, the window of data would be 10 trading days, but because updates are at minute granularity, the first and last day would be partial. For example a 10 day window would stretch from 10:32am on 3/1/2013 to 10:31am on 3/14/2013. This is in contrast to daily mode, where the trading days are always complete.

As you can see from this example, batch_transform is by default rolling - each new event passed to the function will update the window of data. We chose rolling as the default behavior because it is the most commonly needed, but sometimes it doesn't make sense to update the window on every bar: a batch transform may depend on comparing whole trading days, or complete weeks and months.

For these cases, the batch_transform provides an optional parameter refresh_period. For example, if you wanted your minutely simulation to only advance the window in whole day increments, so that the end of the window would always be through yesterday's close, you could specify the batch_transform like this:

@batch_transform(window_length=10, refresh_period=1)
def get_average(datapanel):
  ...

You may set refresh_period to any integer. Say for example, you are running a daily bar simulation and you want to update the trailing window each week:

@batch_transform(window_length=10, refresh_period=5)
def get_average(datapanel):
  ...

By default refresh_period is 0, which means 'rolling.'

Missing Bars for Batch Transforms

Stocks do not necessarily trade every minute or even every day. There are cases where the trailing window may include days that are pre-IPO or post-acquisition for a company. Stocks may be held from trading, or there simply may not be any trading activity in a day or minute. For these missing periods, the batch_transform helper will fill in missing values with the last prior value. As a result, it is still possible for the first period to have a missing value (there is no prior) or for an entire column to be missing when a stock doesn't trade at all in the entire trailing window. Missing values are represented by the pandas NA.

Pandas has excellent default handling in most aggregate functions so that NAs will not produce spurious results. Optionally, you can have the batch_transform leave the NAs instead of filling them. batch_transform will not fill the NAs if you set clean_nans to False (defaults to True). If NAs are not filled, your code needs to defensively handle them, or runtime errors will often result.

@batch_transfrom(window_length=10, clean_nans=False)
def get_averages(datapanel):
  prices = datapanel['price']
  return prices.mean()

Accumulating Custom Fields for Batch Transforms

batch_transform creates a dataframe for each property in the union of all properties for all events in data. You can extend the fields accumulated in the trailing window simply by adding those fields before sending data to the decorated function. Suppose you want to track an event, such as the stock trading above its 30 day vwap. You can set that new property on the events in data and then operate on the trailing history of the value in the batch_transform:

def initialize(context):
    set_universe(universe.DollarVolumeUniverse(99.9, 100.0))

def handle_data(context, data):
    for stock in data:
        if data[stock].price > data[stock].vwap(30):
            data[stock]['over_vwap'] = 1
        else:
            data[stock]['over_vwap'] = 0

    print_signals(data)

@batch_transform(window_length=5)
def print_signals(datapanel):
    signal = datapanel['over_vwap']
    log.info('\n%s' % signal)

Supplemental Parameters for Batch Transforms

Sometimes your batch transform needs additional parameters, outside the datapanel. For example, you might want to pass in the current portfolio or a scalar value for the calculation.

The batch_transform decorator only modifies the first parameter by accumulating data into the datapanel. Any other parameters are simply forwarded unmodified. Here is how you would pass the current portfolio to a decorated method.

def initialize(context):
    set_universe(universe.DollarVolumeUniverse(99.9, 100.0))

def handle_data(context, data):
    # here is an example of passing an extra parameter to the function.
    averages = get_averages(data, context.portfolio)
    log.info('\n%s' % averages)

# Note that the decorator does not receive the supplemental parameters, 
# the function itself does.
@batch_transform(window_length=10)
def get_averages(datapanel, portfolio):
    # get the dataframe of prices 
    prices = datapanel['price']
    # return a dataframe with one row showing the averages for each stock.
    return prices.mean()

Repeated Invocations in Batch Transforms

batch_transform is idempotent, and will cache results where possible. However, for optimal speed, you should try to use vector operations in the batch_transform - operate on the complete dataframe and return dataframes. A rule of thumb is to strive to invoke a batch transform just once per handle_data invocation.

Partially Full Trailing Windows in Batch Transforms

By default, batch_transform will return None if the trailing window is not full. For example, if you set window_length = 10 it will return None for the first 9 trading days and begin returning trailing data on the 10th trading day. This ensures that each window is equal length, and that comparisons between results are valid. However, it is sometimes useful to perform a calculation before the trailing window is completely full. This is supported by the optional parameter compute_only_full, which defaults to True. If set to False, the batch_transform will trigger the function to recalculate values with whatever data is available. In the future, batch_transform will be deprecated in support of the more robust history function, which does not require this warm-up period

Slippage Models

Slippage is where our backtester calculates the realistic impact of your orders on the execution price you receive. When you place an order for a trade, your order affects the market. Your buy order drives prices up, and your sell order drives prices down; this is generally referred to as the 'price impact' of your trade. The size of the price impact is driven by how large your order is compared to the current trading volume. The slippage method also evaluates if your order is simply too big: you can't trade more than market's volume, and generally you can't expect to trade more than a quarter of the volume. All of these concepts are wrapped into the slippage method.

Slippage must be defined in the initialize method. It has no effect if defined in handle_data(). If you do not specify a slippage method, slippage defaults to VolumeShareSlippage(volume_limit=0.25, price_impact=0.1).

To set slippage, use the set_slippage method and pass in FixedSlippage, VolumeShareSlippage, or a custom slippage model that you define.

By default, backtests use the VolumeShareSlippage model.

Slippage-main

Fixed Slippage

When using the FixedSlippage model, the size of your order does not affect the price of your trade execution. You specify a 'spread' that you think is a typical bid/ask spread to use. When you place a buy order, half of the spread is added to the price; when you place a sell order, half of the spread is subtracted from the price.

Fixed-slippage

Volume Share Slippage

In the VolumeShareSlippage model, the price you get is a function of your order size relative to the security's actual traded volume. You provide a volume_limit cap (default 0.25), which limits the proportion of volume that your order can take up per bar. For example: if the backtest is running in one-minute bars, and you place an order for 60 shares; then 100 shares trade in each of the next several minute; and the volume_limit is .25; then your trade order will be split into three orders (25 shares, 25 shares, and 10 shares).

The price impact constant (default 0.1) defines how large of an impact your order will have on the backtester's price calculation. The slippage is calculated by multiplying the price impact constant by the square of the ratio of the order to the total volume. In our previous example, for the 25-share orders, the price impact is .1 * (25/100) * (25/100), or 0.625%. For the 10-share order, the price impact is .1 * (10/100) * (10/100), or .1%.

Volumesharesslippage

Custom Slippage

You can build a custom slippage model that uses your own logic to convert a stream of orders into a stream of transactions. In the initialize() function you must specify the slippage model to be used and any special parameters that the slippage model will use. Example:

def initialize(context):
    set_slippage(MyCustomSlippage(slippage-param))

Your custom model must be a class that inherits from slippage.SlippageModel and implements process_order(self, trade_bar, order).

Each order that your algorithm places, along with the current price information for that security (the trade bar), is passed to your custom model's process_order. The logic inside that method decides whether to generate a transaction, and if so, at what amount and price. To create a transaction, use the slippage.create_transaction method

The order object has the following properties: amount (float), direction (1 for buy, -1 for sell), sid (int), stop and limit (float), and stop_reached and limit_reached (boolean). The trade_bar object is the same as data[sid] in handle_data and has open_price, close_price, high, low, volume, and sid.

The slippage.create_transaction method takes the given trade_bar, the given order, the price and amount calculated by your slippage model, and returns the newly built transaction.

Many slippage models' behavior depends on how much of the total volume traded is being captured by the algorithm. You can use self.volume_for_bar to see how many shares of the current security have been traded so far during this bar. If your algorithm has many different orders for the same stock in the same bar, this is useful for making sure you don't take an unrealistically large fraction of the traded volume.

If your slippage model doesn't place a transaction for the full amount of the order, the order stays open with an updated amount value, and will be passed to process_order on the next bar. Orders that have limits that have not been reached will not be passed to process_order. Finally, if your transaction has 0 shares or more shares than the original order amount, an exception will be thrown.

Please see the sample custom slippage model.

Commission Models

To set the cost of your trades, use the set_commission method and pass in PerShare or PerTrade. Like the slippage model, set_commission must be used in the initialize method and has no effect if used in handle_data. If you don't specify a commission, your backtest defaults to $0.03 per share.

You can define your trading cost in either dollars per share or dollars per trade.

Commission

Viewing Portfolio State

Your current portfolio state is accessible from the context object in handle_data:

Help-portfolio

To view an individual position's information, use the context.portfolio.positions dictionary:

Help-positions

Details on the portfolio and position properties can be found in the API documentation below.

Module Import

Only specific, whitelisted Python modules can be imported. If you need a module that isn't on this list, please let us know.

  • bisect
  • cmath
  • collections
  • datetime
  • functools
  • heapq
  • itertools
  • math
  • numpy
  • pandas
  • pytz
  • QSTK
  • Queue
  • random
  • re
  • scipy
  • statsmodels
  • sklearn
  • time
  • zipline

Running Backtests

You can set the start date, end date, and starting capital used by the backtest in the IDE.

Help-ide2

To create a new backtest, click the 'Run Full Backtest' button from the IDE. That button appears once your algorithm successfully validates. Press the 'Build' button to start the validation if the 'Run Full Backtest' button is not visible in the upper-right of the IDE.

We also provide a Backtests Page that has a summary of all backtests run against an algorithm. To go to the Backtests page, either click the Backtest button at the top right of the IDE, or, from the My Algorithms page click the number of backtests that have been run. The backtests page lists all the backtests that have been run for this algorithm, including any that are in progress. You can view an existing or in-progress backtest by clicking on it. Closing the browser will not stop the backtest from running. Quantopian runs in the cloud and it will continue to execute your backtest until it finishes running. If you want to stop the backtest, press the Cancel button.

Backtest results

Once a backtest starts, we load up all the trading events for the securities that your algorithm specified, and feed them to your algorithm in time order. Results will start streaming in momentarily after the backtest starts.

Here is a snapshot of a backtest results page. Mouse over each section to learn more.

Help-backtest-results
Overall results
Cumulative performance and benchmark overlay
Daily and weekly P/L
Transactions chart
Result details
Backtest settings and status

Backtest settings and status: Shows the initial settings for the backtest, the progress bar when the backtest is in progress, and the final state once the test is done. If the backtest is cancelled, exceeds its max daily loss, or have runtime errors, that information will be displayed here.

Result details: Here's where you dive into the details of your backtest results. You can examine every transaction that occurred during the backtest, see how your positions evolved over time, and look at detailed risk metrics. For the risk metrics, we show you 1, 3, 6, and 12-month windows to provide more granular breakdowns

Overall results: This is the overall performance and risk measures of your backtest. These numbers will update during the course of the backtest as new data comes in.

Cumulative performance and benchmark overlay: Shows your algorithm's performance over time (in blue) overlaid with the benchmark (in red).

Daily and weekly P/L: Shows your P/L per day or week, depending on the date range selected.

Transactions chart: Shows all the cumulative dollar value of all the buys and sells your algorithm placed, per day or week. Buys are shown as positive blue, and sells as negative reds.

API Documentation

Methods to implement

Your algorithm has to implement two methods: initialize and handle_data.

initialize(context)

Called once at the very beginning of a backtest. Your algorithm can use this method to set up any bookkeeping that you'd like.

The context object will be passed to all the other methods in your algorithm.

Parameters

context: An initialized and empty Python dictionary. The dictionary has been augmented so that properties can be accessed using dot notation as well as the traditional bracket notation.

Returns

None

Example
def initialize(context):
    context.notional_limit = 100000

handle_data(context, data)

Called whenever a market event occurs for any of your algorithm's specified securities.

Parameters

data: A dictionary containing your universe at that time, keyed by security id. It represents a snapshot of your algorithm's universe as of when this method was called. Market information about each security and transforms are all available in this object. Read more below.

context: Same context object in initialize, stores any state you've defined, and stores portfolio object.

Returns

None

Example
def handle_data(context, data):
    # all your algorithm logic here
    # ...
    order(sid(24), 100)
    # ...

Order Methods

Within your algorithm, there are some order methods you can use:

order(sid, amount, limit_price=value1, stop_price=value2)

Places an order for the specified security of the specified number of shares. Order type is inferred from the parameters used. If only sid and amount are used as parameters, the order is placed as a market order.

Parameters

sid: A security object.

amount: The integer amount of shares. Positive means buy, negative means sell.

limit_price: (optional) The price at which the limit order becomes active. If used with stop_price, the price where the limit order becomes active after stop_price is reached.

stop_price: (optional) The price at which the order converts to a market order. If used with limit_price, the price where the order converts to a limit order.

Returns

An order id.

order_percent(sid, percent, limit_price=value1, stop_price=value2)

Places an order in the specified security corresponding to the given percent of the current portfolio value, which is the sum of the positions value and ending cash balance. Placing a negative percent order will result in selling the given percent of the current portfolio value. Orders are always truncated to whole shares. Percent must be expressed as a decimal (0.50 means 50%).

Example

order_percent(sid(24), .5) will have AAPL shares 50% of current portfolio value. If AAPL is $100/share and the portfolio value is $2000, this buy 10 shares (discarding slippage and transaction cost).

Parameters

sid: A security object.

percent: The decimal equivalent to the percent of the portfolio to be invested. Positive means buy, negative means sell.

limit_price: (optional) The price at which the limit order becomes active. If used with stop_price, the price where the limit order becomes active after stop_price is reached.

stop_price: (optional) The price at which the order converts to a market order. If used with limit_price, the price where the order converts to a limit order.

Returns

An order id.

order_target(sid, target, limit_price=value1, stop_price=value2)

Places an order to adjust a position to a target number of shares. If there is no existing position in the security, an order is placed for the full target number. If there is a position in the security, an order is placed for the difference between the target number of shares and the current number of shares. Placing a negative target order will result in a short position equal to the negative number specified.

Example

If the current portfolio has 5 shares of AAPL and the target is 20 shares, order_target(sid(24), 20) orders 15 more shares of AAPL.

Parameters

sid: A security object.

target: The target integer amount of shares. Positive means long, negative means short.

limit_price: (optional) The price at which the limit order becomes active. If used with stop_price, the price where the limit order becomes active after stop_price is reached.

stop_price: (optional) The price at which the order converts to a market order. If used with limit_price, the price where the order converts to a limit order.

Returns

An order id.

order_target_value(sid, target_value, limit_price=value1, stop_price=value2)

Places an order to adjust a position to a target value. If there is no existing position in the security, an order is placed for the full target value. If there is a position in the security, an order is placed for the difference between the target value and the current position value. Placing a negative target order will result in a short position equal to the negative target value. Orders are always truncated to whole shares

Example

If the current portolio holds $500 worth of AAPL and the target is $2000, order_target_value(sid(24), 2000) orders $1500 worth of AAPL (rounded down to the nearest share).

Parameters

sid: A security object.

target_value: The target dollar amount to be invested. Positive means long, negative means short.

limit_price: (optional) The price at which the limit order becomes active. If used with stop_price, the price where the limit order becomes active after stop_price is reached.

stop_price: (optional) The price at which the order converts to a market order. If used with limit_price, the price where the order converts to a limit order.

Returns

An order id.

order_target_percent(sid, target_percent, limit_price=value1, stop_price=value2)

Place an order to adjust a position to a target percent of the current portfolio value. If there is no existing position in the security, an order is placed for the full target percentage. If there is a position in the security, an order is placed for the difference between the target percent and the current percent. Placing a negative target percent order will result in a short position equal to the negative target percent. Portfolio value is calculated as the sum of the positions value and ending cash balance. Orders are always truncated to whole shares, and percentage must be expressed as a decimal (0.50 means 50%).

Example

If the current portfolio value is 5% worth of AAPL and the target is to allocate 10% of the portfolio value to AAPL, order_target_percent(sid(24), 0.1) will place an order for the difference, in this case ordering 5% portfolio value worth of AAPL.

order_value(sid, value, limit_price=value1, stop_price=value2)

Place an order by desired value rather than desired number of shares. Placing a negative order value will result in selling the given value. Orders are always truncated to whole shares.

Example

Order AAPL worth up to $1000: order_value(sid(24), 1000). If price of AAPL is $105 a share, this would buy 9 shares, since the partial share would be truncated (discarding slippage and transaction cost).

Parameters

sid: A security object.

value: The value of this order, in dollars. Positive means long, negative means short.

limit_price: (optional) The price at which the limit order becomes active. If used with stop_price, the price where the limit order becomes active after stop_price is reached.

stop_price: (optional) The price at which the order converts to a market order. If used with limit_price, the price where the order converts to a limit order.

Returns

An order id.

cancel_order(order)

Attempts to cancel the specified order. Cancel is attempted asynchronously.

Parameters

order: Can be the order_id as a string or the order object.

Returns

None

get_open_orders(sid=sid)

If sid is None or not specified, returns all open orders. If sid is specified, returns open orders for that sid

Parameters

sid: (optional) A security object. Can be also be None.

Returns

If sid is unspecified or None, returns a dictionary keyed by security id. The dictionary contains a list of orders for each sid, oldest first. If a sid is specified, returns a list of open orders for that sid, oldest first.

get_order(order)

Returns the specified order. The order object is discarded at the end of handle_data.

Parameters

order: Can be the order_id as a string or the order object.

Returns

returns an order object that is read/writeable but is discarded at the end of handle_data.

Other Methods

Within your algorithm, there are some other methods you can use:

fetch_csv(url, pre_func=None, post_func=None, date_column='date',
           date_format='%m/%d/%y', timezone='UTC', symbol=None, mask = False, **kwargs)

Loads the given CSV file (specified by url) to be used in a backtest.

Parameters

url: A well-formed http or https url pointing to a CSV file that has a header, a date column, and a symbol column (symbol column required to match data to securities).

pre_func: (optional) A function that takes a pandas dataframe parameter (the result of pandas.io.parsers.read_csv) and returns another pandas dataframe.

post_func: (optional) A function that takes a pandas dataframe parameter and returns a dataframe.

date_column: (optional) A string identifying the column in the CSV file's header row that holds the parseable dates.

date_format: (optional) A string defining the format of the date/time information held in the date_column.

timezone: (optional) Either a pytz timezone object or a string conforming to the pytz timezone database.

symbol: (optional) If specified, the fetcher data will be treated as a signal source, and all of the data in each row will be added to the data parameter of the handle_data method. You can access all the CSV data from this source as data['symbol'].

mask: (optional) This is a boolean whose default is True. By default it will import information only for sids initialized in your algo. If set to False, it will import information for all stocks in the CSV file.

**kwargs: (optional) Additional keyword arguments that are passed to the requests.get and pandas read_csv calls. Click here to see the valid arguments.

get_datetime()

Returns the current algorithm time.

Parameters

None

Returns

Returns a Python datetime object with the current time in the algorithm. For daily data, the hours, minutes, and seconds are all 0. For minute data, it's the end of the minute bar.

history(bar_count, frequency, field, ffill=True)

Function to get a trailing window of daily data. For more complete documentation, see the History section.

Parameters

bar_count: The int number of bars returned by the history call. This includes the current bar.

frequency: The size of the bars returned by history. Currently, "1d" is the only supported frequency.

field: The data field selected from history. Currently, only the OHLCV data is supported. Options are: 'open_price', 'close_price', 'price', 'high', 'low', 'volume'.

ffill: Whether or not to forward fill the history data. If ffill is False, the return frame will have np.nan for pricing data, and 0 for volume. The default is True.

Returns

Returns a pandas DataFrame which contains bar_count rows of data for the specified field. The columns of the DataFrame are the sids in the current universe.

log.error(message), log.info(message), log.warn(message), log.debug(message)

Logs a message with the desired log level. Log messages are displayed in the backtest output screen, and we only persist the last 512 of them.

Parameters

message: The message to log.

Returns

None

record(series1_name=value1, series2_name=value2, ...)

Records the given series and values. Generates a chart for all recorded series.

Parameters

values: Keyword arguments (up to 5) specifying series and their values

Returns

None

sid(int)

Convenience method to look up a security by its id. Within the IDE, an inline search box appears showing you matches on security id, symbol, and name.

Parameters

int: The id of a security.

Returns

A security object.

Event properties and transforms

Quantopian's backend parses all the referenced securities in your algorithm and sends you the trading events for those securities. Each time any of the securities has a trading event (at most once per trading minute), handle_data is called and the data object contains all the market data for your securities.

For example, to access the market event data for AAPL, use data[sid(24)]. The following properties are supported:

datetime

DateTime: The UTC timestamp of the market event.

price

Float: The closing price of the security for the given bar.

open_price

Float: The opening price of the security for the given bar.

close_price

Float: The closing price of the security for the given bar. Identical to price.

high

Float: The highest price of the security within the given bar.

low

Float: The lowest price of the security within the given bar.

volume

Integer: The whole number of shares traded in the most recent market event for this security.

We also provide some transforms:

mavg(days)

Moving average price for the given security for the given number of trailing days.

returns()

The returns of this security since the end of the previous trading day.

stddev(days)

Standard deviation of the price of the given security for the given number of trailing days, calculated using Bessel's Correction.

vwap(days)

Volume-weighted average price for the given security for the given number of trailing days.

Order object

If you have a reference to an order object, there are several properties that might be useful:

created

Datetime: The date and time the order was created, in UTC timezone.

stop

Float: Optional stop price.

limit

Float: Optional limit price.

amount

Integer: Total shares ordered.

sid

Security object: The security being ordered.

filled

Integer: Total shares bought or sold for this order so far.

Portfolio object

The portfolio object is accessed using context.portfolio and has the following properties:

capital_used

Float: The net capital consumed (positive means spent) by buying and selling securities up to this point.

cash

Float: The current amount of cash in your portfolio.

pnl

Float: Dollar value profit and loss, for both realized and unrealized gains.

positions

Dictionary: A dictionary of all the open positions, keyed by security ID. More information about each position object can be found in the next section.

portfolio_value

Float: Sum value of all open positions and ending cash balance.

positions_value

Float: Sum value of all open positions.

returns

Float: Cumulative percentage returns for the entire portfolio up to this point. Calculated as a fraction of the starting value of the portfolio. The returns calculation includes cash and portfolio value. The number is not formatted as a percentage, so a 10% return is formatted as 0.1.

starting_cash

Float: Initial capital base for this backtest or live execution.

start_date

DateTime: UTC datetime of the beginning of this backtest's period. For live trading, this marks the UTC datetime that this algorithm started executing.

Position object

The position object represents a current open position, and is contained inside the positions dictionary. For example, if you had an open AAPL position, you'd access it using context.portfolio.positions[sid(24)]. The position object has the following properties:

amount

Integer: Whole number of shares in this position.

cost_basis

Float: The volume-weighted average price paid per share in this position.

last_sale_price

Float: Price at last sale of this security. This is identical to close_price and price.

sid

Integer: The ID of the security.

Security object

If you have a reference to a security object, there are several properties that might be useful:

sid

Integer: The id of this security.

symbol

String: The ticker symbol of this security.

security_name

String: The full name of this security.

security_start_date

Datetime: The date when this security first started trading.

security_end_date

Datetime: The date when this security stopped trading (= today for securities that are trading normally).

TA-Lib methods

TA-Lib is an open-source library to process financial data. A subset of the methods are available in the Quantopian API. If you need any TA-Lib methods that we don't currently support, let us know.

When using TA-Lib methods, create the object outside of the initialize and handle_data methods. Type ta. and a dropdown will appear:

Help-talib

You can look at our example TA-Lib algorithm.

Automatic mapping of OHLCV data

The supported TA-Lib methods all operate on a combination of open/high/low/close/volume data. Since we automatically handle those parameters, you only have to worry about method-specific parameters like time period. For example, the STOCH function uses high, low, and close.

Getting results for all sids

When using TA-Lib methods in Quantopian, we apply the methods on all the sids in your algorithm. Therefore, all the return values are dicts, keyed by sid. Most methods return a single float per sid, but some return tuples.

Daily vs minute data

Quantopian's TA-Lib methods respect the data frequency in your backtest or live algorithm. If you're backtesting with daily data, then all the time periods are calculated in days. If you're backtesting or live trading with minute data, all the time periods are in minutes.

Matching MACD values between data sources

Quantopian uses a fixed window length of 34 days every time for the calculation. Yahoo and Google use a window from the first day of the year to the current day, so it grows each day. The main advantage of always using a fixed window, instead of all the data available, is that the signal for a given day will be the same across different backtest ranges. Because of this discrepancy, the values may not match between Quantopian and external data sources. For more information see this discussion here.

Moving average types

Some of the TA-Lib methods have an integer matype parameter. Here's the list of moving average types:
0: SMA (simple)
1: EMA (exponential)
2: WMA (weighted)
3: DEMA (double exponential)
4: TEMA (triple exponential)
5: TRIMA (triangular) 
6: KAMA (Kaufman adaptive)
7: MAMA (Mesa adaptive)
8: T3 (triple exponential T3)

Method list:

ADOSC: Chaikin A/D Oscillator learn more

parameters:
fastperiod: Number of periods for the fast moving average, default is 3.
slowperiod: Number of periods for the slow moving average, default is 10.
output:
Dictionary of sid to float.

ADX: Average Directional Movement Index learn more

parameters:
timeperiod: Number of periods, default is 14.
output:
Dictionary of sid to float.

ADXR: Average Directional Movement Index Rating learn more

parameters:
timeperiod: Number of periods, default is 14.
output:
Dictionary of sid to float.

APO: Absolute Price Oscillator learn more

parameters:
fastperiod: Number of periods for the fast moving average, default is 12.
slowperiod: Number of periods for the slow moving average, default is 26.
matype: Type of moving average, default is 0.
output:
Dictionary of sid to float.

AROON: Aroon learn more

parameters:
timeperiod: Number of periods, default is 14.
output:
Dictionary of sid to tuples, where each tuple is two floats: (aroonDown, aroonUp).

AROONOSC: Aroon Oscillator learn more

parameters:
timeperiod: Number of periods, default is 14.
output:
Dictionary of sid to float.

ATR: Average True Range learn more

parameters:
timeperiod: Number of periods, default is 14.
output:
Dictionary of sid to float.

BBANDS: Bollinger Bands learn more

parameters:
timeperiod: Number of periods, default is 14.
nbdevup: Deviation multiplier for upper band, default is 2.
nbdevdn: Deviation multiplier for lower band of periods, default is 2.
matype: Type of moving average, default is 0.
output:
Dictionary of sid to tuples, where each tuple is three floats: (upperLimit, line, lowerLimit).

CCI: Commodity Channel Index learn more

parameters:
timeperiod: Number of periods, default is 14.
output:
Dictionary of sid to float.

CMO: Chande Momentum Oscillator learn more

parameters:
timeperiod: Number of periods, default is 14.
output:
Dictionary of sid to float.

CORREL: Pearson's Correlation Coefficient learn more

parameters:
timeperiod: Number of periods, default is 30.
output:
Dictionary of sid to float.

DEMA: Double Exponential Moving Average learn more

parameters:
timeperiod: Number of periods, default is 30.
output:
Dictionary of sid to float.

DX: Directional Movement Index learn more

parameters:
timeperiod: Number of periods, default is 14.
output:
Dictionary of sid to float.

EMA: Exponential Moving Average learn more

parameters:
timeperiod: Number of periods, default is 30.
output:
Dictionary of sid to float.

KAMA: Kaufman Adaptive Moving Average learn more

parameters:
timeperiod: Number of periods, default is 30.
output:
Dictionary of sid to float.

LINEARREG: Linear Regression learn more

parameters:
timeperiod: Number of periods, default is 14.
output:
Dictionary of sid to float.

LINEARREG_ANGLE: Linear Regression Angle learn more

parameters:
timeperiod: Number of periods, default is 30.
output:
Dictionary of sid to float.

LINEARREG_INTERCEPT: Linear Regression Intercept learn more

parameters:
timeperiod: Number of periods, default is 30.
output:
Dictionary of sid to float.

LINEARREG_SLOPE: Linear Regression Slope learn more

parameters:
timeperiod: Number of periods, default is 30.
output:
Dictionary of sid to float.

MA: Moving Average learn more

parameters:
timeperiod: Number of periods, default is 30.
matype: Type of moving average, default is 0.
output:
Dictionary of sid to float.

MACD: Moving Average Convergence/Divergence learn more

parameters:
fastperiod: Number of periods for the fast moving average, default is 12.
slowperiod: Number of periods for the slow moving average, default is 26.
signalperiod: Smoothing for the signal line (number of periods), default is 9.
output:
Dictionary of sid to tuples, where each tuple is three floats: (macd, macdSignal, macdHist).

MACDEXT: MACD with controllable MA type learn more

parameters:
fastperiod: Number of periods for the fast moving average, default is 12.
fastmatype: Type of moving average for the fast period, default is 0.
slowperiod: Number of periods for the slow moving average, default is 26.
slowmatype: Type of moving average for the slow period, defualt is 0.
signalperiod: Smoothing for the signal line (number of periods), default is 9.
signalmatype: Type of moving average for the signal period, default is 0.
output:
Dictionary of sid to tuples, where each tuple is three floats: (macd, macdSignal, macdHist).

MACDFIX: Moving Average Convergence/Divergence Fix 12/26 learn more

parameters:
signalperiod: Smoothing for the signal line (number of periods), default is 9.
output:
Dictionary of sid to tuples, where each tuple is three floats: (macd, macdSignal, macdHist).

MAX: Highest values over a specified period

parameters:
timeperiod: Number of periods, default is 30.
output:
Dictionary of sid to float.

MAXINDEX: Index of highest value over a specified period

parameters:
timeperiod: Number of periods, default is 30.
output:
Dictionary of sid to float.

MFI: Money Flow Index learn more

parameters:
timeperiod: Number of periods, default is 14.
output:
Dictionary of sid to float.

MIDPOINT: Midpoint over period learn more

parameters:
timeperiod: Number of periods, default is 14.
output:
Dictionary of sid to float.

MIDPRICE: Midpoint price over period learn more

parameters:
timeperiod: Number of periods, default is 14.
output:
Dictionary of sid to float.

MIN: Lowest value over a specified period

parameters:
timeperiod: Number of periods, default is 30.
output:
Dictionary of sid to float.

MIN: Index of lowest value over a specified period

parameters:
timeperiod: Number of periods, default is 30.
output:
Dictionary of sid to float.

MINMAX: Lowest and highest values over a specified period

parameters:
timeperiod: Number of periods, default is 30.
output:
Dictionary of sid to tuples, where each tuple is two floats: (min, max).

MINMAXINDEX: Indices of lowest and highest values over a specified period

parameters:
timeperiod: Number of periods, default is 30.
output:
Dictionary of sid to tuples, where each tuple is two floats: (minIndex, maxIndex).

MINUS_DI: Minus Directional Indicator learn more

parameters:
timeperiod: Number of periods, default is 14.
output:
Dictionary of sid to float.

MINUS_DM: Minus Directional Movement learn more

parameters:
timeperiod: Number of periods, default is 14.
output:
Dictionary of sid to float.

NATR: Normalized Average True Range learn more

parameters:
timeperiod: Number of periods, default is 14.
output:
Dictionary of sid to float.

PLUS_DI: Plus Directional Indicator learn more

parameters:
timeperiod: Number of periods, default is 14.
output:
Dictionary of sid to float.

PLUS_DM: Plus Directional Movement learn more

parameters:
timeperiod: Number of periods, default is 14.
output:
Dictionary of sid to float.

PPO: Percentage Price Oscillator learn more

parameters:
fastperiod: Number of periods for the fast moving average, default is 12.
slowperiod: Number of periods for the slow moving average, default is .
matype: Type of moving average, default is 0.
output:
Dictionary of sid to float.

ROC: Rate of change: ((price/prevPrice)-1)*100 learn more

parameters:
timeperiod: Number of periods, default is 10.
output:
Dictionary of sid to float.

ROCP: Rate of change percentage: (price-prevPrice)/prevPrice learn more

parameters:
timeperiod: Number of periods, default is 10.
output:
Dictionary of sid to float.

ROCP: Rate of change ratio: (price-prevPrice) learn more

parameters:
timeperiod: Number of periods, default is 10.
output:
Dictionary of sid to float.

ROCP: Rate of change ratio 100 scale: (price-prevPrice)*100 learn more

parameters:
timeperiod: Number of periods, default is 10.
output:
Dictionary of sid to float.

RSI: Relative Strength Index learn more

parameters:
timeperiod: Number of periods, default is 14.
output:
Dictionary of sid to float.

SMA: Simple Moving Average learn more

parameters:
timeperiod: Number of periods, default is 30.
output:
Dictionary of sid to float.

STDDEV: Standard Deviation learn more

parameters:
timeperiod: Number of periods, default is 5.
nbdev: Number of deviations, default is 1.
output:
Dictionary of sid to float.

STOCH: Stochastic learn more

parameters:
fastk_period: Time period for building the fast-k line, default is 5.
slowk_period: Smoothing for the slow-k line, default is 3.
slowk_matype: Type of moving average for slow-k, default is 0.
slowd_period: Smoothing for the slow-d line, default is 3.
slowd_matype: Type of moving average for slow-d, default is 0.
output:
Dictionary of sid to tuples, where each tuple is two floats: (slowK, slowD).

STOCHF: Stochastic Fast learn more

parameters:
fastk_period: Time period for building the fast-k line, default is 5.
fastd_period: Smoothing for the fast-d line, default is 3.
fastd_matype: Type of moving average for fast-d, default is 0.
output:
Dictionary of sid to tuples, where each tuple is two floats: (fastK, fastD).

STOCHRSI: Stochastic Relative Strength Index learn more

parameters:
timeperiod: Number of periods, default is 14.
fastk_period: Time period for building the fast-k line, default is 5.
fastd_period: Smoothing for the fast-d line, default is 3.
fastd_matype: Type of moving average for fast-d, default is 0.
output:
Dictionary of sid to tuples, where each tuple is two floats: (fastK, fastD).

T3: Triple Exponential Moving Average (T3) learn more

parameters:
timeperiod: Number of periods, default is 5.
vfactor: Volume factor, default is 0.7.
output:
Dictionary of sid to float.

TEMA: Triple Exponential Moving Average learn more

parameters:
timeperiod: Number of periods, default is 30.
output:
Dictionary of sid to float.

TRIMA: Triangular Moving Average learn more

parameters:
timeperiod: Number of periods, default is 30.
output:
Dictionary of sid to float.

TRIX: 1-day rate of change of a triple smooth EMA learn more

parameters:
timeperiod: Number of periods, default is 30.
output:
Dictionary of sid to float.

TSF: Time Series Forecast learn more

parameters:
timeperiod: Number of periods, default is 14.
output:
Dictionary of sid to float.

ULTOSC: Ultimate Oscillator learn more

parameters:
timeperiod1: Number of bars for the first period, default is 7.
timeperiod2: Number of bars for the second period, default is 14.
timeperiod3: Number of bars for the third period, default is 28.
output:
Dictionary of sid to float.

VAR: Variance learn more

parameters:
timeperiod: Number of periods, default is 5.
nbdev: Number of deviations, default is 1.
output:
Dictionary of sid to float.

WILLR: Williams' %R learn more

parameters:
timeperiod: Number of periods, default is 14.
output:
Dictionary of sid to float.

WMA: Weighted Moving Average learn more

parameters:
timeperiod: Number of periods, default is 30.
output:
Dictionary of sid to float.

IDE Tips and Shortcuts

You can customize the color and text size of the IDE for your algorithms. Simply click on the gear button in the top right corner and choose your settings.

Help-settings

Below are keyboard shortcuts you can use in the IDE.

Action Windows Keystroke Apple Keystroke
Build Algorithm Ctrl + B Cmd + B
Indent Ctrl + ] Cmd + ]
Outdent Ctrl + [ Cmd + [
Create/Remove Comment Ctrl + / Cmd + /
Search Code Ctrl + F Cmd + F

Need more space to see your code? Drag the bar all the way to the right to expand the code window.

Help-expandwindow

If you are building a custom graph you can record up to five variables. To view only certain variables, click on the variable name to remove it from the chart. Click it again to add it back to the graph. Want to zoom in on a certain timeperiod? Use the bar underneath the graph to select a specific time window.

Below is an example of a custom graph with all the variables selected. The following graph displays only the recorded cash value.

Help-customgraph Help-customgraph-cash

Sample Algorithms

Our first example is very basic. If you want to get started quickly, you can just copy this code into your algorithm and run a backtest. Other examples get into more advanced concepts.

Basic Algorithm

This example demos all of the basic concepts you need to write a simple algorithm that tries to capitalize on a stock's momentum. This shows you the security id (sid) function, logging, volume-weighted average price, price, and placing orders. You can clone this algorithm from the community discussion of this post.

Clone Algorithm
    # For this example, we're going to write a simple momentum script.  When the 
    # stock goes up quickly, we're going to buy; when it goes down quickly, we're
    # going to sell.  Hopefully we'll ride the waves.

    # To run an algorithm in Quantopian, you need two functions: initialize and 
    # handle_data.

def initialize(context):
    # This initialize function sets any data or variables that you'll use in
    # your algorithm.  For instance, you'll want to define the security (or 
    # securities) you want to backtest.  You'll also want to define any 
    # parameters or values you're going to use.

    # In our example, we're looking at Apple.  If you re-type this line 
    # yourself, you'll see the auto-complete that is available for the 
    # security ID.
    context.aapl = sid(24)

    # In these two lines, we set the maximum and minimum we want our algorithm 
    # to go long or short our security.  You don't have to set limits like this
    # when you write an algorithm, but it's good practice. If you indicate these
    # values, confirm that the maximum amount is identical to your capital base that
    # will run in the backtest in the right hand panel
    context.max_notional = 1000000.1
    context.min_notional = -1000000.0

def handle_data(context, data):
    # This handle_data function is where the real work is done.  Our data is
    # minute-level tick data, and each minute is called a frame.  This function
    # runs on each frame of the data.

    # We've built a handful of useful data transforms for you to use.  In this 
    # line, we're computing the volume-weighted-average-price of the security 
    # defined above, in the context.aapl variable.  For this example, we're 
    # specifying a three-day average.
    vwap = data[context.aapl].vwap(3)
    # We need a variable for the current price of the security to compare to
    # the average.
    price = data[context.aapl].price

    # Another powerful built-in feature of the Quantopian backtester is the
    # portfolio object.  The portfolio object tracks your positions, cash,
    # cost basis of specific holdings, and more.  In this line, we calculate
    # how long or short our position is at this minute.   
    notional = context.portfolio.positions[context.aapl].amount * price

    # You can use the record() method to track any custom signal. The record graph
    # will track up to five different variables. Here we record the portfolio cash value
    record(cash = context.portfolio.cash)

    # This is the meat of the algorithm, placed in this if statement.  If the
    # price of the security is .5% less than the 3-day volume weighted average
    # price AND we haven't reached our maximum short, then we call the order
    # command and sell 100 shares.  Similarly, if the stock is .5% higher than
    # the 3-day average AND we haven't reached our maximum long, then we call
    # the order command and buy 100 shares.         
    if price < vwap * 0.995 and notional > context.min_notional:
        order(context.aapl,-100)
        log.info("Selling %s" % (context.aapl))
    elif price > vwap * 1.005 and notional < context.max_notional:
        order(context.aapl,+100)
        log.info("Buying %s" % (context.aapl))

Multiple Security Example

This example shows how to initialize and trade with multiple securities. It imports the datetime and pytz libraries and uses the log function to understand the behavior. You can clone this algorithm from the community discussion of this example.

Clone Algorithm
# This example runs the same momentum play as the first sample 
# (https://www.quantopian.com/help#sample-basic), but this time it uses more
# securities during the backtest.
    
# Important note: All securities in an algorithm must be traded for the 
# entire length of the backtest.  For instance, if you try to backtest both
# Google and Facebook against 2011 data you will get an error; Facebook
# wasn't traded until 2012.

# First step is importing any needed libraries.

import datetime
import pytz

def initialize(context):
    # Here we initialize each stock.  Note that we're not storing integers; by
    # calling sid(24) we're storing the Security object.
    context.stocks = [sid(24), sid(2), sid(2673), sid(5061)]
    context.vwap = {}
    context.price = {}
 
    # Setting our maximum position size, like previous example
    context.max_notional = 1000000.1
    context.min_notional = -1000000.0

    # Initializing the time variables we use for logging
    # Convert timezone to US EST to avoid confusion
    est = pytz.timezone('EST')
    context.d=datetime.datetime(2000, 1, 1, 0, 0, 0, tzinfo=est)
   

def handle_data(context, data):
    # Initializing the position as zero at the start of each frame
    notional=0
    
    # This runs through each stock.  It computes
    # our position at the start of each frame.
    for stock in context.stocks:
        price = data[stock].price 
        notional = notional + context.portfolio.positions[stock].amount * price
        tradeday = data[stock].datetime
        
    # This runs through each stock again.  It finds the price and calculates
    # the volume-weighted average price.  If the price is moving quickly, and
    # we have not exceeded our position limits, it executes the order and
    # updates our position.
    for stock in context.stocks:   
        vwap = data[stock].vwap(3)
        price = data[stock].price  

        if price < vwap * 0.995 and notional > context.min_notional:
            order(stock,-100)
            notional = notional - price*100
        elif price > vwap * 1.005 and notional < context.max_notional:
            order(stock,+100)
            notional = notional + price*100

    # If this is the first trade of the day, it logs the notional.
    if (context.d + datetime.timedelta(days=1)) < tradeday:
        log.debug(str(notional) + ' - notional start ' + tradeday.strftime('%m/%d/%y'))
        context.d = tradeday

Batch Transform Example

This example shows how to create and use a batch transform. If the price hits the high in the trailing window, it sells; if it hits the low in the trailing window, it buys.

Clone Algorithm
# This is the standard Quantopian function that initializes your data and 
# variables. In it, we define how large of a 'bet' we're making (in dollars) and what 
# stock we're working with.
def initialize(context):
    # we will be trading in AMZN and WMT shares
    context.stocks = [sid(16841), sid(8229)]
    context.bet_amount = 100000
    context.long = 0

# This is the standard Quantopian event handling function. This function is 
# run once for each bar of data.  In this example, it the min and max 
# prices for the trailing window.  If the price exceeds the recent high, it 
# goes short; if the price dips below the recent low, it goes long. The algo
# is a contrarian/mean reversion bet.
def handle_data(context, data):
    # Until our batch transform's datapanel is full, it will return None.  Once
    # the datapanel is full, then we have a max and min to work with.
    rval = minmax(data)
    
    if rval is None:
        return
    
    maximums, minimums = rval
    
    for stock in context.stocks:
        cur_max = maximums[stock]
        cur_min = minimums[stock]
        cur_price = data[stock].price
        cur_position = context.portfolio.positions[stock]
           
        order_direction = calculate_direction(stock, cur_min, cur_max, cur_price, cur_position)
        order_amount = calculate_order_amount(context, stock, order_direction, cur_price)
        
        # Optional: uncomment the log line below if you're looking for more detail about what's 
        # going on.  It will log all the information that is a 'moving part' of this
        # algorithm. Note: if you're doing a full backtest it's a lot of log lines!
        logmsg = '\n{s}: max {m} min {i} price {p} position amount {l}\nordering {n} shares'
        log.info(logmsg.format(
            s=stock, 
            m=cur_max, 
            i=cur_min, 
            p=cur_price, 
            l=cur_position.amount,
            n=order_amount
        ))
        
        order(stock, order_amount)
        
# Here we do our test to see if we should buy or sell or do nothing.  This is
# the main part of the algorithm. Once we establish a position (long or short)
# we use the context.long variable to remember which we took.
def calculate_direction(stock, cur_min, cur_max, cur_price, cur_position):
    if cur_max is not None and cur_position.amount <= 0 and cur_price >= cur_max:
        return -1
    elif cur_min is not None and cur_position.amount >= 0 and cur_price <= cur_min:
        return 1
    
    return 0
   
# This method is purely for order management. It calculates and returns an 
# order amount to place binary bets.
# If signal_val is -1, get to a short position of -1 * context.bet_size
# If signal_val is 1, get to a long position of context.bet_size
def calculate_order_amount(context, stock, signal_val, cur_price):
    current_amount = context.portfolio.positions[stock].amount
    abs_order_amount = int(context.bet_amount / cur_price) 
    
    if signal_val == -1:
        return (-1 * abs_order_amount) - current_amount
    elif signal_val == 1:
        return abs_order_amount - current_amount
    else:
        return 0        
            
# This is our batch transform decorator/declaration. We set the 
# refresh_period and length of the window.  In this case, once per day we're loading 
# the last 10 trading days and evaluating them.
@batch_transform(refresh_period=1, window_length=10)
def minmax(datapanel):
    # We are looking for the min and the max price to return. Just because it's interesting
    # we also are logging the current price. 
    prices_df = datapanel['price']
    min_price = prices_df.min()
    max_price = prices_df.max()

    if min_price is not None and max_price is not None:
        return (max_price, min_price)
    else:
        return None

Set Universe Example

This example shows how to use the set_universe command. Here the stocks are chosen by selecting a segment of the dollar volume universe.

Clone Algorithm
# This is the standard Quantopian function that initializes your data and 
# variables. In it, we define how large of a 'bet' we're making (in dollars) and what 
# stock we're working with.
def initialize(context):
    # we want to try this on a range of highly liquid stocks
    set_universe(universe.DollarVolumeUniverse(97, 99))
    context.bet_amount = 1000
    context.count = 20

# This is the standard Quantopian event handling function. This function is 
# run once for each bar of data.  In this example, it the min and max 
# prices for the trailing window.  If the price exceeds the recent high, it 
# goes short; if the price dips below the recent low, it goes long. The algo
# is a contrarian/mean reversion bet.
def handle_data(context, data):
    # Until our batch transform's datapanel is full, it will return None.  Once
    # the datapanel is full, then we have a max and min to work with.
    
    prices = history(bar_count=20, frequency='1d', field='price')
    
    ranking = sort_returns(prices)

    if ranking is not None:
        column_name = ranking.columns[0]  
        # bottom quantile to go long
        bottom = ranking[-1*context.count:] 
        longs = bottom[bottom[column_name] < 0]
        
        # top quantile to go short
        top = ranking[:context.count]
        shorts = top[top[column_name] > 0]
        
        for stock in data.keys():
            if stock in longs.index:
                amount = calculate_order_amount(context, stock, 1, data[stock].price)
            elif stock in shorts.index:
                amount = calculate_order_amount(context, stock, -1, data[stock].price)
            else:
                amount = calculate_order_amount(context, stock, 0, data[stock].price)
                
            order(stock, amount)    
   
# This method is purely for order managment. It calculates and returns an 
# order amount to place binary bets.
# If signal_val is -1, get to a short position of -1 * context.bet_size
# If signal_val is 1, get to a long position of context.bet_size
def calculate_order_amount(context, stock, signal_val, cur_price):
    current_amount = context.portfolio.positions[stock].amount
    abs_order_amount = int(context.bet_amount / cur_price) 
    
    if signal_val == -1:
        return (-1 * abs_order_amount) - current_amount
    elif signal_val == 1:
        return abs_order_amount - current_amount
    elif signal_val == 0:
        return -1 * current_amount
    else:
        return 0        

def sort_returns(prices):
    shifted_prices = prices.shift(19)
    returns = (prices - shifted_prices) / shifted_prices
    # use a slice operator to get the most recent returns
    last_returns = returns[-1:]
    last_date = last_returns.index[-1]
    sorted_returns = last_returns.T.sort(columns=last_date, ascending=0)
    return sorted_returns

Record Variables Example

This example compares Google's 20-day moving average with its 80-day moving average and trades when the two averages cross. It records both moving averages as well as Google's share price.

Clone Algorithm
# This initialize function sets any data or variables that you'll use in
# your algorithm.
def initialize(context):
    context.sid = sid(26578)   #Google

    
# Now we get into the meat of the algorithm. 
def handle_data(context, data):
    # Create a variable for the price of the Google stock
    context.price = data[context.sid].price
    
    # Create variables to track the short and long moving averages. 
    # The short moving average tracks over 20 days and the long moving average
    # tracks over 80 days. 
    short = data[context.sid].mavg(20)
    long = data[context.sid].mavg(80)

    # If the short moving average is higher than the long moving average, then 
    # we want our portfolio to hold 500 stocks of Google
    if (short > long):
        order_target(context.sid, +500)
    
    # If the short moving average is lower than the long moving average, then
    # then we want to sell all of our Google stocks and own 0 shares
    # in the portfolio. 
    elif (short < long):
        order_target_value(context.sid, 0)

    # Record our variables to see the algo behavior. You can record up to 
    # 5 custom variables. To see only a certain variable, deselect the 
    # variable name in the custom graph in the backtest. 
    record(short_mavg = short,
        long_mavg = long,
        goog_price = context.price)

Fetcher Example

This example loads and formats two price series from Quandl. It displays the price movements and decides to buy and sell Tiffany stock based on the price of gold.

Clone Algorithm
import pandas

def rename_col(df):
    df = df.rename(columns={'New York 15:00': 'price'})
    df = df.rename(columns={'Value': 'price'})
    df = df.fillna(method='ffill')
    df = df[['price', 'sid']]
    # Correct look-ahead bias in mapping data to times   
    df = df.tshift(1, freq='b')
    log.info(' \n %s ' % df.head())
    return df
    
def initialize(context):
    # import the external data
    fetch_csv('http://www.quandl.com/api/v1/datasets/JOHNMATT/PALL.csv?trim_start=2012-01-01',
        date_column='Date',
        symbol='palladium',
        post_func=rename_col,
        date_format='%Y-%m-%d')

    fetch_csv('http://www.quandl.com/api/v1/datasets/BUNDESBANK/BBK01_WT5511.csv?trim_start=2012-01-01',
        date_column='Date',
        symbol='gold',
        post_func=rename_col,
        date_format='%Y-%m-%d')
    
    # Tiffany
    context.stock = sid(7447)

def handle_data(context, data):
    # Invest 10% of the portfolio in Tiffany stock when the price of gold is low.
    # Decrease the Tiffany position to 5% of portfolio when the price of gold is high.

    if (data['gold'].price < 1600):
       order_target_percent(context.stock, 0.10)
    if (data['gold'].price < 1750):
       order_target_percent(context.stock, 0.05)

    #record the variables   
    if 'price' in data['palladium']:
       record(palladium=data['palladium'].price, gold=data['gold'].price)

Custom Slippage Example

This example implements the fixed slippage model, but with the additional flexibility to specify a different spread for each stock.

Clone Algorithm
# Our custom slippage model
class PerStockSpreadSlippage(slippage.SlippageModel):

    # We specify the constructor so that we can pass state to this class, but this is optional.
    def __init__(self, spreads):
        # Store a dictionary of spreads, keyed by sid.
        self.spreads = spreads

    def process_order(self, trade_bar, order):
        spread = self.spreads[order.sid]
   
        # In this model, the slippage is going to be half of the spread for 
        # the particular stock
        slip_amount = spread / 2
        # Compute the price impact of the transaction. Size of price impact is 
        # proprotional to order size. 
        # A buy will increase the price, a sell will decrease it. 
        new_price = trade_bar.price + (slip_amount * order.direction)

        log.info('executing order ' + str(trade_bar.sid) + ' stock bar price: ' + \
                 str(trade_bar.price) + ' and trade executes at: ' + str(new_price))

        # Create the transaction using the new price we've calculated.
        return slippage.create_transaction(
            trade_bar,
            order,
            new_price,
            order.amount
        )

def initialize(context):
    # Provide the bid-ask spread for each of the securities in the universe.
    spreads = {
        sid(24): 0.05,
        sid(3766): 0.08
    }
   
    # Initialize slippage settings given the parameters of our model
    set_slippage(PerStockSpreadSlippage(spreads))


def handle_data(context, data):
    # We want to own 100 shares of each stock in our universe
    for stock in data:
            order_target(stock, 100)
            log.info('placing market order for ' + str(stock.symbol) + ' at price ' \
                     + str(data[stock].price))
            

TA-Lib Example

This example uses TA-Lib's RSI method.

Clone Algorithm
# This example algorithm uses the Relative Strength Index indicator as a buy/sell signal.
# When the RSI is over 70, a stock can be seen as overbought and it's time to sell.
# When the RSI is below 30, a stock can be seen as oversold and it's time to buy.

import numpy as np
import math

# Set up the Relative Strength Index indicator.
# It will calculate the RSI over a trailing 14-day window.
rsi = ta.RSI(timeperiod=14)

def initialize(context):
    context.max_notional = 100000
    context.goog = sid(26578)
    context.LOW_RSI = 30
    context.HIGH_RSI = 70

def handle_data(context, data):
    # check the RSI of Google.
    # rsi_data is a dictionary of sid -> rsi_value
    rsi_data = rsi(data)
    goog_rsi = rsi_data[context.goog]

    # check how many shares of Google we currently own
    current_google_shares = context.portfolio.positions[context.goog].amount

    # until 14 time periods have gone by, the rsi value will be numpy.nan
    if not np.isnan(goog_rsi):
        if goog_rsi > context.HIGH_RSI and current_google_shares > 0:
            # RSI is above 70 and we own GOOG, time to sell.
            order(context.goog, -1 * current_google_shares)
            log.info('RSI is at ' + str(goog_rsi) + ', selling ' + str(current_google_shares) + ' shares')
        elif goog_rsi < context.LOW_RSI and current_google_shares == 0:
            # RSI is below 30 and we don't have any GOOG, time to buy.
            num_shares = math.floor(context.max_notional / data[context.goog].close_price)
            order(context.goog, num_shares)
            log.info('RSI is at ' + str(goog_rsi) + ', buying ' + str(num_shares)  + ' shares')

    # record the current RSI value and the current price of GOOG.
    record(googrsi=goog_rsi, googprice=data[context.goog].close_price)

Portfolio Allocation Example

This example uses order_target_percent order method.

Clone Algorithm
def initialize(context):
    context.stocks = [sid(1419),sid(12652)]

def handle_data(context, data):
    # This will order as many shares as needed to
    # achieve the desired portfolio allocation.
    # In our case, we end up with 20% allocation for
    # one stock and 80% allocation for the other stock.
    order_target_percent(context.stocks[0], .2)
    order_target_percent(context.stocks[1], .8)

    # Plot portfolio allocations
    pv = float(context.portfolio.portfolio_value)
    portfolio_allocations = []
    for stock in context.stocks:
        pos = context.portfolio.positions[stock]
        portfolio_allocations.append(
            pos.last_sale_price * pos.amount / pv * 100
        )

    record(perc_stock_0=portfolio_allocations[0],
           perc_stock_1=portfolio_allocations[1])

History Example

This example uses the history function.

Clone Algorithm
# Standard Deviation Using History
# Use history() to calculate the standard deviation of the days' closing
# prices of the last 10 trading days, including price at the time of 
# calculation.
def initialize(context):
    # this example works on Apple's data
    context.aapl = sid(24)

def handle_data(context, data):
    # use history to pull the last 10 days of price
    price_history = history(bar_count=10, frequency='1d', field='price')
    # calculate the standard deviation using std()
    std = price_history.std()
    # record the standard deviation as a custom signal
    record(std=std[context.aapl])