Back to Community
Reverse engineer future stock prices

Attached is a backtest of God trading. Basically algo looks what tomorrow's prices will be and then every day buys the stock that will grow the most.

Made me think what tools would you use to reverse engineer this? What indicators or factors contributed the most to the ensemble that lead to these stocks being the right picks? Is there a thought process that can lead to some interesting conclusions?

Clone Algorithm
20
Loading...
Backtest from to with initial capital
Total Returns
--
Alpha
--
Beta
--
Sharpe
--
Sortino
--
Max Drawdown
--
Benchmark Returns
--
Volatility
--
Returns 1 Month 3 Month 6 Month 12 Month
Alpha 1 Month 3 Month 6 Month 12 Month
Beta 1 Month 3 Month 6 Month 12 Month
Sharpe 1 Month 3 Month 6 Month 12 Month
Sortino 1 Month 3 Month 6 Month 12 Month
Volatility 1 Month 3 Month 6 Month 12 Month
Max Drawdown 1 Month 3 Month 6 Month 12 Month
"""
This is a template algorithm on Quantopian for you to adapt and fill in.
"""
import quantopian.algorithm as algo
from quantopian.pipeline import Pipeline
from quantopian.pipeline.data.builtin import USEquityPricing
from quantopian.pipeline.filters import QTradableStocksUS


def initialize(context):
    """
    Called once at the start of the algorithm.
    """
    # Rebalance every day, 1 hour after market open.
    algo.schedule_function(
        rebalance,
        algo.date_rules.every_day(),
        algo.time_rules.market_open(hours=1),
    )

    # Record tracking variables at the end of each day.
    algo.schedule_function(
        record_vars,
        algo.date_rules.every_day(),
        algo.time_rules.market_close(),
    )

    # Create our dynamic stock selector.
    algo.attach_pipeline(make_pipeline(), 'pipeline')


def make_pipeline():
    """
    A function to create our dynamic stock selector (pipeline). Documentation
    on pipeline can be found here:
    https://www.quantopian.com/help#pipeline-title
    """

    # Base universe set to the QTradableStocksUS
    base_universe = QTradableStocksUS()

    # Factor of yesterday's close price.
    yesterday_close = USEquityPricing.close.latest

    pipe = Pipeline(
        columns={
            'close': yesterday_close,
        },
        screen=base_universe
    )
    return pipe


def before_trading_start(context, data):
    """
    Called every day before market open.
    """
    context.output = algo.pipeline_output('pipeline')

    # These are the securities that we are interested in trading each day.
    context.security_list = context.output.index


def rebalance(context, data):
    """
    Execute orders according to our schedule_function() timing.
    """
    hist = data.history(context.security_list, 'price', -390, '1m')  #look into tomorrow
    returns = ((hist.ix[-1] - hist.ix[0])/ hist.ix[0]).dropna().sort_values(ascending=False)
    
    stock=returns.head(1).index[0]
    order_target_percent(stock, 1.0)
    
    for stock in context.portfolio.positions:
            order_target_percent(stock, 0)
    pass


def record_vars(context, data):
    """
    Plot variables at the end of each day.
    """
    pass


def handle_data(context, data):
    """
    Called every minute.
    """
    pass
There was a runtime error.
16 responses

God is making a boat load of money! Are negative bar counts in data.history supposed to be permitted? Seems like a backtest software bug.

Hahaha, awesome!! Here's a long/short version to make it market neutral. Would make a killing in the contest! ;)

Clone Algorithm
15
Loading...
Backtest from to with initial capital
Total Returns
--
Alpha
--
Beta
--
Sharpe
--
Sortino
--
Max Drawdown
--
Benchmark Returns
--
Volatility
--
Returns 1 Month 3 Month 6 Month 12 Month
Alpha 1 Month 3 Month 6 Month 12 Month
Beta 1 Month 3 Month 6 Month 12 Month
Sharpe 1 Month 3 Month 6 Month 12 Month
Sortino 1 Month 3 Month 6 Month 12 Month
Volatility 1 Month 3 Month 6 Month 12 Month
Max Drawdown 1 Month 3 Month 6 Month 12 Month
"""
This is a template algorithm on Quantopian for you to adapt and fill in.
"""
import quantopian.algorithm as algo
from quantopian.pipeline import Pipeline
from quantopian.pipeline.data.builtin import USEquityPricing
from quantopian.pipeline.filters import QTradableStocksUS


def initialize(context):
    """
    Called once at the start of the algorithm.
    """
    # Rebalance every day, 1 hour after market open.
    algo.schedule_function(
        rebalance,
        algo.date_rules.every_day(),
        algo.time_rules.market_open(hours=1),
    )

    # Record tracking variables at the end of each day.
    algo.schedule_function(
        record_vars,
        algo.date_rules.every_day(),
        algo.time_rules.market_close(),
    )

    # Create our dynamic stock selector.
    algo.attach_pipeline(make_pipeline(), 'pipeline')


def make_pipeline():
    """
    A function to create our dynamic stock selector (pipeline). Documentation
    on pipeline can be found here:
    https://www.quantopian.com/help#pipeline-title
    """

    # Base universe set to the QTradableStocksUS
    base_universe = QTradableStocksUS()

    # Factor of yesterday's close price.
    yesterday_close = USEquityPricing.close.latest

    pipe = Pipeline(
        columns={
            'close': yesterday_close,
        },
        screen=base_universe
    )
    return pipe


def before_trading_start(context, data):
    """
    Called every day before market open.
    """
    context.output = algo.pipeline_output('pipeline')

    # These are the securities that we are interested in trading each day.
    context.security_list = context.output.index


def rebalance(context, data):
    """
    Execute orders according to our schedule_function() timing.
    """
    hist = data.history(context.security_list, 'price', -390, '1m')  #look into tomorrow
    returns = ((hist.ix[-1] - hist.ix[0])/ hist.ix[0]).dropna().sort_values(ascending=False)
    
    stock=returns.head(1).index[0]
    order_target_percent(stock, 0.5)
    
    stock=returns.tail(1).index[0]
    order_target_percent(stock, -0.5)
    
    for stock in context.portfolio.positions:
            order_target_percent(stock, 0)
    pass


def record_vars(context, data):
    """
    Plot variables at the end of each day.
    """
    pass


def handle_data(context, data):
    """
    Called every minute.
    """
    pass
There was a runtime error.

Definitely appears to be a bug. I sent a note to Q support.

Yes, certainly a bug. Thanks for the heads up!

Disclaimer

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.

This is how the Doctor can afford a TARDIS.

Bug fix in zipline here: https://github.com/quantopian/zipline/pull/2164

Thanks all,
Josh

I see this as a feature ("God mode") rather then a bug.

Reverse engineering it the new backtest mode reveals that this trade has biggest exposure to the volatility factor.

Yet its only about 10% exposure. This means there is a yet undiscovered factor accounting for 90% of exposure. Let's call it the God factor...

Hi vladimir,

I can appreciate that perspective. Generally we provide unfettered access to data in research notebooks and work to enforce a lack of look-ahead bias in the backtester. Which is why we patched this bug/feature.

Thanks
Josh

Suppose someone stumbled upon this bug, and has a contest entry running that utilizes it. When the fix goes through, will it crash their algo? Or is the the underlying API "frozen" for contest entries?

I do not believe they can run a contest entry with this because in paper trading there is no "future" history.

Looks like it's been patched now. It was fun as long it lasted. :)

@Grant: Vladimir is right about the old contest; the negative window length would not have worked. For the daily contest, it would have been possible to look ahead to the minutes later in the day (since participating entries are scored from a backtest). The contest code base is updated regularly as we make improvements to the backtester. The negative window length bug was patched earlier this week both in the backtester and in the daily contest, so contest algorithms would have raised an exception with a message like: ValueError: bar_count must be >= 1, but got -390. I went back and reviewed some of the results from earlier this week in the contest and it looks like no one was exploiting that bug.

Disclaimer

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.

@vladimir,

Reverse engineering it the new backtest mode reveals that this trade has biggest exposure to the volatility factor.

Yet its only about 10% exposure. This means there is a yet undiscovered factor accounting for 90% of exposure. Let's call it the God factor...

It would be nice if you can share the "God mode" full tearsheet so we can all see the various exposures of all the God factors. Thanks.

Can you view this one? Not bad exposure values given that it only holds 1 long and 1 short per day. Enjoy! :)

Loading notebook preview...
Notebook previews are currently unavailable.

Yep thats what I've seen as well. It just too noisy to make any conclusions. Reverse engineering approach still does feel like an interesting starting point for some kind of analysis.

If volatility has the biggest exposure at 10% in "God mode", it basically confirms an old but proven stock market trading principle...high returns = high risks, low returns = low risks. With perfect hindsight or "Gode mode", as you call it, we can see that profits are derived proportionately with risks as represented by volatility. With 90% other attributable factors being noisy and many, volatility sticks out like a sore thumb, making 10% exposure highly significant.

This is quite different from what Q and their investors are looking for in their market neutral long short equity strategy which seeks low volatility and steady low returns that is protected against market movements and shocks. I am guessing that Point72, being a multi-strategy hedge fund manager is exploring this market neutral long short equity strategy as a diversification component of their overall strategy.

If the aim is to get an allocation, then refocuse on what they are looking for, one notch above risk free rate returns and risks. Trying to chase high returns with very low volatility/risks is like finding the "holy grail".