Back to Community
Evolutionary Strategy

Genetic algorithms are algorithms that mimic natural selection. This is a simple evolving algorithm that trades top stocks. Essentially, momentum strategies are randomly generated. Based on how those strategies would perform over a period of time (30 days), the best performers, or parents, are selected. Using attributes these parents have, new algorithms are generated that have similar attributes to the parents. This process is then repeated. Trades are made using the overall best performing algorithm.

Here's a cool example of a genetic algorithm: http://rednuht.org/genetic_cars_2/

Although the strategies being evolved are basic and don't perform great, this is just meant to be an example. I think there are a lot of ways one could extend this, like moving away from momentum or importing relevant data from a CSV file. There are also some variables that can easily be adjusted that may lead to better results, and the code is commented. Clone this, play around with it, and let me know what you think!

Clone Algorithm
352
Loading...
Backtest from to with initial capital
Total Returns
--
Alpha
--
Beta
--
Sharpe
--
Sortino
--
Max Drawdown
--
Benchmark Returns
--
Volatility
--
Returns 1 Month 3 Month 6 Month 12 Month
Alpha 1 Month 3 Month 6 Month 12 Month
Beta 1 Month 3 Month 6 Month 12 Month
Sharpe 1 Month 3 Month 6 Month 12 Month
Sortino 1 Month 3 Month 6 Month 12 Month
Volatility 1 Month 3 Month 6 Month 12 Month
Max Drawdown 1 Month 3 Month 6 Month 12 Month
# 1. Warm up for 70 days
# 2. Generate 5 random momentum strategies
# 3. Run these strategies for 30 days
# 4. Pick the best of the strategies that have been run, and generate 5 new strategies derived from the top performers
# 5. Continue with this process, and make trades based off the strategy that performed the best in the past

# The random strategies have 4 "genes" in their "genome"
# derivatives determines, generally speaking, if we want to take action when the price has just a positive/negative first derivative or both a positive/negative first and second derivative
# vwap1 is the main vwap
# vwap2 is used if we have derivatives = 2
# shortlong determines whether we go short or long on upswings/downswings

# For each strategy "in testing", search through stocks that fit the criteria and take action
# Using the strategy that performed the best, make trades

import random
from collections import deque

# Initialize the needed starting variables
def initialize(context):
    set_universe(universe.DollarVolumeUniverse(99.5, 100))
    
    set_slippage(slippage.FixedSlippage(spread=0))
    set_commission(commission.PerTrade(cost=0))
    
    context.target_duration = 30 # How long to test strategies for
    context.num_parents = 4 # Number of parents each new species has
    context.generation_size = 5 # How many species to generate per generation
    
    context.frames = deque(maxlen=70) # Store past frames for past prices
    
    context.species = [] # All species are stored here
    new_generation(context) # Create a new generation (starts testing on 70th day)

# Creates a new generation of size context.generation_size based off of top performers from past generations
def new_generation(context):
    possible_parents = []
    for spec in context.species:
        if spec['run_duration'] >= context.target_duration: possible_parents.append(spec)
    parents = sorted(possible_parents, key=lambda k: k['value'])[-context.num_parents:]
    
    for i in range(context.generation_size):
        
        # Create new species with 4 genes from parents and a new random gene
        random_gene = random.randrange(4)
        
        if random_gene == 0 or len(parents) < context.num_parents:
            derivatives = random.randint(1, 2)
        else:
            derivatives = parents[random.randrange(context.num_parents)]['derivatives']
        
        if random_gene == 1 or len(parents) < context.num_parents:
            vwap1 = random.randint(3, 70)
        else:
            vwap1 = parents[random.randrange(context.num_parents)]['vwap1']

        if random_gene == 2 or len(parents) < context.num_parents:
            vwap2 = random.randint(2, vwap1-1)
        else:
            vwap2_poss = []
            for spec in parents:
                if spec['vwap2'] < vwap1: vwap2_poss.append(spec['vwap2'])
            if len(vwap2_poss) > 0:
                vwap2 = random.choice(vwap2_poss)
            else:
                vwap2 = random.randint(2, vwap1-1)
            
        if random_gene == 3 or len(parents) < context.num_parents:
            shortlong = random.randrange(2)
        else:
            shortlong = parents[random.randrange(context.num_parents)]['shortlong']
        
        # For each species, store its genes, how long it's been run for, its value (starts at 100, similar to $100), and its positions
        context.species.append({'derivatives':derivatives, 'vwap1':vwap1, 'vwap2':vwap2, 'shortlong':shortlong, 'run_duration':0, 'value':100, 'current_longs':[], 'current_shorts':[]})
    return None

# Run a specific strategy species, used on strategies with run_duration less than context.target_duration and also the best overall returner
def run_strategy(key, context, data):
    strat = context.species[key]
    for sid in data:
        if strat['derivatives'] == 1:
            # If price is more or less than vwap, take action
            if data[sid].price > dynamicvwap(context, sid, strat['vwap1']):
                if strat['shortlong'] == 0 and sid not in strat['current_longs']:
                    strat['current_longs'].append(sid)
                    if sid in strat['current_shorts']:
                        strat['current_shorts'].remove(sid)
                elif strat['shortlong'] == 1 and sid not in strat['current_shorts']:
                    strat['current_shorts'].append(sid)
                    if sid in strat['current_longs']:
                        strat['current_longs'].remove(sid)
            elif data[sid].price < dynamicvwap(context, sid, strat['vwap1']):
                if strat['shortlong'] == 1 and sid not in strat['current_longs']:
                    strat['current_longs'].append(sid)
                    if sid in strat['current_shorts']:
                        strat['current_shorts'].remove(sid)
                elif strat['shortlong'] == 0 and sid not in strat['current_shorts']:
                    strat['current_shorts'].append(sid)
                    if sid in strat['current_longs']:
                        strat['current_longs'].remove(sid)

        else:
            # If two derivatives in agreement, take action
            if data[sid].price > dynamicvwap(context, sid, strat['vwap2']) and dynamicvwap(context, sid, strat['vwap2']) > dynamicvwap(context, sid, strat['vwap1']):
                if strat['shortlong'] == 0 and sid not in strat['current_longs']:
                    strat['current_longs'].append(sid)
                    if sid in strat['current_shorts']:
                        strat['current_shorts'].remove(sid)
                elif strat['shortlong'] == 1 and sid not in strat['current_shorts']:
                    strat['current_shorts'].append(sid)
                    if sid in strat['current_longs']:
                        strat['current_longs'].remove(sid)
            elif data[sid].price < dynamicvwap(context, sid, strat['vwap2']) and dynamicvwap(context, sid, strat['vwap2']) < dynamicvwap(context, sid, strat['vwap1']):
                if strat['shortlong'] == 1 and sid not in strat['current_longs']:
                    strat['current_longs'].append(sid)
                    if sid in strat['current_shorts']:
                        strat['current_shorts'].remove(sid)
                elif strat['shortlong'] == 0 and sid not in strat['current_shorts']:
                    strat['current_shorts'].append(sid)
                    if sid in strat['current_longs']:
                        strat['current_longs'].remove(sid)
    return None

# Updates returns after a strategy is run
def update_returns(key, context, data):
    strat = context.species[key]
    daily_total_returns = 0
    for sid in data:
        if sid in strat['current_longs']:
            daily_total_returns += data[sid].returns()
        elif sid in strat['current_shorts']:
            daily_total_returns -= data[sid].returns()
    if len(strat['current_longs']) + len(strat['current_shorts']) > 0:
        strat['value'] *= (daily_total_returns/(len(strat['current_longs']) + len(strat['current_shorts'])) + 1)
    return None

# We need a special volume-weighted average price function since our num_bars isn't static
def dynamicvwap(context, sid, num_bars):
    addedprices = 0
    count = 0
    for i in range(1, num_bars+1):
        bar = context.frames[-i]
        addedprices += bar[sid].price
        count += 1
    return addedprices/count
    
# This is called once per bar
def handle_data(context, data):
    context.frames.append(data) # Update the recent bars
    top_returner = {}

    # If we have enough past data:
    if len(context.frames) == 70:
        new_gen = 1
        
        # Find the top performer
        tested_strats = []
        for strat in context.species:
            if strat['run_duration'] >= context.target_duration:
                tested_strats.append(strat)
        top_returners = sorted(tested_strats, key=lambda k: k['value'])
        if top_returners:
            top_returner = top_returners[-1]
        
        # For each species, check if it needs to be run and then run it
        for i in range(len(context.species)):
            if context.species[i]['run_duration'] < context.target_duration or context.species[i] == top_returner:
                
                # Update returns, but not for the top performer to prevent bias
                if context.species[i] != top_returner:
                    update_returns(i, context, data)
                    context.species[i]['run_duration'] += 1
                    new_gen = 0
                run_strategy(i, context, data)
        
        if new_gen == 1:
            log.info('Current top performer:')
            log.info('derivatives: '+str(top_returner['derivatives']))
            log.info('vwap1: '+str(top_returner['vwap1']))
            log.info('vwap2: '+str(top_returner['vwap2']))
            log.info('shortlong: '+str(top_returner['shortlong']))
            log.info('# of positions: '+str(len(top_returner['current_shorts'])+len(top_returner['current_longs'])))
            log.info('Value: '+str(top_returner['value']))
            log.info('-------------')
            
            new_generation(context) # Create new generation
        
        longs = []
        shorts = []
        context.portfolio.portfolio_value
        # Make trades based on the top performer
        # Buy and sell shares in 2:1 leverage with amount proportional to our starting cash
        if top_returner:
            for sid in data:
                if sid in top_returner['current_longs']:
                    longs.append(sid)
                elif sid in top_returner['current_shorts']:
                    shorts.append(sid)

            for sid in data:
                if sid in longs:
                    order_target_percent(sid, 1.0/len(longs))
                elif sid in shorts:
                    order_target_percent(sid, -1.0/len(shorts))
                else:
                    order_target(sid, 0)
    try: top_returner['value']
    except: top_returner['value'] = 0
    
    record(num_species=len(context.species), top_value=top_returner['value'])         
There was a runtime error.
Disclaimer

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.

8 responses

thanks your sharing!

some interesting reference:
http://www.thertrader.com/2014/03/14/using-genetic-algorithms-in-quantitative-trading/

Application of Genetic Algorithms to Quantitative Asset Allocation Models

genetic programming libraries in Python:
Pyvolution: Very modular and very extensible evolutionary algorithms framework, with complete documentation, Apache License 2.0
deap: Distributed Evolutionary Algorithms in Python, GNU Lesser GPL
pySTEP: Python Strongly Typed gEnetic Programming, MIT License
Pyevolve
PyRobot: Evolutionary Algorithms (GA + GP) Modules, Open Source
PonyGEa small, one source file implementation of GE, with an interactive graphics demo application GNU GPL v3
inspyred: Biologically inspired computation encompasses a broad range of algorithms including evolutionary computation, swarm intelligence, and neural networks, GNU GPL v3
DRP: Directed Ruby Programming, Genetic Programming & Grammatical Evolution Library, GNU GPL

http://www.jonathankinlay.com/

Jonathan Kinlay has been posting recently about the practices and pitfalls of genetic programming for algorithmic trading. I attempted a system based on it about ten years ago, but never really got anywhere.

I do know that for nonlinear global optimization problems, we had good results (in derivatives model fitting) with particle swarm differential evolution. Perhaps that could be adapted to trading system search problems.

Ty for sharing that is actually my area of research, using a few years of historical data I used to need over 1 day to compute a model I wonder how would quantopian and python deal with big models.

We have tried genetic programming and the results when adjusted for data snooping are terrible. For starters who are not familiar with data mining bias see this blog: http://www.priceactionlab.com/Blog/2012/06/fooled-by-randomness-through-selection-bias/

@SImon: Dr. Kinlay makes a few good points but the system he posts at the end of his blog has 240 trades in nearly 25 years. The small number of trades is one indication of an extreme fit. The point is that if his system was a top performer of a genetic programming algorithm, then it is probably random even if the out of sample performance looks nice because his selection ignores all those systems with bad out of sample performance.

I agree 100%, I think it's mostly a dead end, but perhaps differential evolution of deep neural nets might fit real relationships? Super tricky to avoid mining bias though, so many degrees of freedom. Marco de Prado's technique for backtest overfitting might help...

GP is just an optimization algorithm, it is as good as its fitness function and the model you are trying to optimize. If the model is over fitting then your fitness function is not doing its job properly if the model is not evolving then the strategy being optimized is not good. I see GP more as a tool-box.

Very nice Gus. Plenty of food for thought here. Been looking at Genotick recently - written in Java and so a bit of a slog for me. But if you are adept at Java you might like to take a look.Genotick

I am trying to do something similar using genetic programming, but trying to create a more turn-key web app approach.

https://www.gpsignals.com

Would love some feedback, as I would like to expand the data series and techniques that are available to the framework.
also trying to learn Python to do some Quantopian stuff (I am a Java guy).