Optimize

The quantopian.optimize module provides tools for defining and solving portfolio optimization problems directly. You can interact with the Optimize API by providing an Objective and a list of Constraint objects to an API function that runs a portfolio optimization. Optimize hides most of the complex mathematics of portfolio optimization, allowing you to think in terms of high-level concepts like “maximize expected returns” and “constrain sector exposure” instead of the actual abstract matrix products.

Below is an example optimization expressed using the Optimize API:

import quantopian.optimize as opt

objective = opt.MaximizeAlpha(expected_returns)
constraints = [
    opt.MaxGrossExposure(W_max),
    opt.PositionConcentration(min_weights, max_weights),
]
optimal_weights = opt.calculate_optimal_portfolio(objective, constraints)

Running Optimizations

There are three ways to optimize your portfolio with Optimize:

  1. calculate_optimal_portfolio() solves optimizations (calculates an optimal portfolio, but does not place any orders) - used in Research or the IDE.
  2. order_optimal_portfolio() runs the same optimization as calculate_optimal_portfolio() and places the orders necessary to achieve that portfolio - used in the IDE.
  3. run_optimization() performs the same optimization as calculate_optimal_portfolio() but returns an OptimizationResult with additional information - used in both Research and the IDE.

Note

When placing orders with Optimize, it is generally recommended to use order_optimal_portfolio(). However, run_optimization() can also be useful for debugging optimizations that are failing or producing unexpected results.

All types of portfolio optimizations require an objective and constraints. You can think of the constraints as a "box". Within the "box" defined by the constraints, the Optimize API will look for the portfolio that generates the maximum value of the objective function. As such, you must always have exactly one objective, but it is possible to have any number of constraints (though having too many or too few constraints may result in an InfeasibleConstraints error).

Objectives

Every portfolio optimization requires an Objective to tell the optimizer what function should be maximized by the new portfolio.

There are currently two available objectives:

TargetWeights is used by algorithms that explicitly construct their own target portfolios. It is useful, for example, in algorithms that identify a list of target assets (e.g. a list of 50 longs and 50 shorts) and want to target an equal weighted or factor weighted portfolio of those assets. TargetWeights takes a Series mapping assets to target weights for those assets. It finds an array of portfolio weights that is as close as possible (measured by Euclidean Distance) to the targets.

MaximizeAlpha is used by algorithms that try to predict expected returns. MaximizeAlpha takes a Series mapping assets to "alpha" values for each asset, and it finds an array of new portfolio weights that maximizes the sum of each asset's weight times its alpha value.

Note

MaximizeAlpha does not result in asset weights proportional to their alpha values. Rather, it maximizes the sum of each asset's weight times its alpha value. As such, highest ranked assets usually end up taking up as much weight as the constraints will allow.

Constraints

It's often necessary to enforce constraints on the results produced by a portfolio optimizer. When using MaximizeAlpha, for example, it's necessary to enforce a constraint on the total value of all long and short positions. If no such constraint is provided, the optimizer will fail trying to put "infinite" capacity into every asset with a nonzero alpha value.

You can tell the optimizer about the constraints on your portfolio by passing a list of Constraint objects when running an optimization. For example, to require the optimizer to produce a portfolio with gross exposure less than or equal to the current portfolio value, you can supply a MaxGrossExposure constraint:

import quantopian.optimize as opt

objective = opt.MaximizeAlpha(calculate_alphas())
constraints = [opt.MaxGrossExposure(1.0)]
order_optimal_portfolio(objective, constraints)

The most commonly-used constraints are as follows:

  • MaxGrossExposure constrains a portfolio's gross exposure (i.e., the sum of the absolute value of the portfolio's positions) to be less than a percentage of the current portfolio value.
  • NetExposure constrains a portfolio's net exposure (i.e. the value of the portfolio's longs minus the value of its shorts) to fall between two percentages of the current portfolio value.
  • PositionConcentration constrains a portfolio's exposure to each individual asset in the portfolio.
  • NetGroupExposure constrains a portfolio's net exposure to a set of market sub-groups (e.g. sectors or industries).
  • FactorExposure constrains a portfolio's net weighted exposure to a set of Risk Factors.

See the API reference for a complete listing of all available constraints.

Debugging Optimizations

One issue you may encounter when using the Optimize API is that it's possible to accidentally ask the optimizer to solve problems for which no solution exists. There are two common ways this can happen:

  1. There is no possible portfolio that satisfies all required constraints. When this happens, the optimizer raises an InfeasibleConstraints exception.
  2. The constraints supplied to the optimization fail to enforce an upper bound on the objective function being maximized. When this happens, the optimizer raises an UnboundedObjective exception.

Debugging an UnboundedObjective error is usually straightforward. UnboundedObjective is most commonly encountered when using the MaximizeAlpha objective without any other constraints. Since MaximizeAlpha tries to put as much capital as possible to the assets with the largest alpha values, additional constraints are necessary to prevent the optimizer from trying to allocate "infinite" capital.

Debugging an InfeasibleConstraints can be more challenging. If the optimizer raises InfeasibleConstraints, it means that every possible set of portfolio weights violates at least one of the constraints. Since different portfolios may violate different constraints, it may be difficult to identify a single constraint as the source.

One useful observation when debugging InfeasibleConstraints errors is that there are certain special portfolios that one might expect to be feasible. One might expect, for example, that it is always possible to liquidate existing positions to produce an "empty" portfolio. When an InfeasibleConstraints error is raised, the optimizer examines a small number of these special portfolios and produces an error message detailing the constraints that were violated by each special portfolio. The portfolios currently examined by the optimizer for diagnostics are:

  1. The current portfolio.
  2. An empty portfolio.
  3. The target portfolio (only applies when using the TargetWeights objective).

For example, suppose we have a portfolio that currently has long and short positions worth 10% of our portfolio value in AAPL and MSFT, respectively. The following optimization will fail with an InfeasibleConstraints error:

import quantopian.optimize as opt

# Target a portfolio that closes out our MSFT position.
objective = opt.TargetWeights({AAPL: 0.1, MSFT: 0.0})

# Require the portfolio to be within 1% of dollar neutral.
# Our target portfolio violates this constraint.
dollar_neutral = opt.DollarNeutral(tolerance=0.01)

# Require the portfolio to hold exactly 10% AAPL.
# An empty portfolio would violate this constraint.
must_long_AAPL = opt.FixedWeight(AAPL, 0.10)

# Don't allow shorting MSFT.
# Our current portfolio violates this constraint.
cannot_short_MSFT = opt.LongOnly(MSFT)

constraints = [dollar_neutral, must_long_AAPL, cannot_short_MSFT]

opt.calculate_optimal_portfolio(objective, constraints)

The resulting error will contain the following message:

The attempted optimization failed because no portfolio could be found that
satisfied all required constraints.

The following special portfolios were spot checked and found to be in violation
of at least one constraint:

Target Portfolio (as provided to TargetWeights):

   Would violate DollarNeutral() because:
      Net exposure (0.1) would be greater than max net exposure (0.01).

Current Portfolio (at the time of the optimization):

   Would violate LongOnly(Equity(5061 [MSFT])) because:
      New weight for Equity(5061 [MSFT]) (-0.1) would be less than 0.

Empty Portfolio (no positions):

   Would violate FixedWeight(Equity(24 [AAPL])) because:
      New weight (0.0) would not equal required weight (0.1).

This error message tells us that each portfolio tested by the optimizer violated a different constraint. The target portfolio violated the DollarNeutral constraint because it had a net exposure of 10% of our portfolio value. The current portfolio violated the LongOnly constraint because it had a short position in MSFT. The empty portfolio violated the FixedWeight constraint on AAPL because it gave AAPL a weight of 0.0.