Back to Community
The next tearsheet challenge is here: come up with factors for the guidance dataset

Our recent estimates data challenge was a huge success, so we are excited to run the next one with very similar parameters. This round we are looking for your best guidance factors. We think that this challenge will be more difficult than the last ones and more creativity will be required in finding alpha, but we are looking forward to seeing what you come up with.

Here are the rules, pretty much the same as before:

  • There is no submission or live-updated leaderboard like for the contest. To enter this challenge, simply post an alpha tearsheet as a reply to this thread. For this, you would run a backtest on your factor and run the alpha notebook which loads in your backtest results.
  • The deadline to submit a factor is October 21, 2019.
  • There is no hold-out testing, just post your best factor starting on June 1, 2015 until Oct 1, 2018.
  • We will look at all tearsheet submissions and manually determine 5 best algorithms according to our discretion. Each winner gets a $100 prize.
  • There is no limit on the number of submissions.

Algorithm requirements to enter the challenge:

  • Use the guidance data set as your primary signal source. It is OK to combine guidance signals with other sources if there are predictive interactions between them.
  • Use TargetWeights in the optimizer and do not put any constraints on common risk exposures.
  • Use the QTU.

When selecting a winner, we will primarily look at:

  • Specific Sharpe Ratio (IR) over the first 5 days in the alpha decay analysis (higher is better).
  • Turnover (lower is better).
  • Not driven mainly by common risk (but no reason to try and artificially reduce your exposures, ideally your idea is dissimilar enough from common factors that it will be naturally uncorrelated).
  • Universe size (larger is better)
  • For more examples on what we look for, check out our last live tearsheet review.

These rules are mostly derived from our updated guidelines on getting an allocation and are based on many community members' feedback. Thank you for all your input and creative suggestions. If this challenge is well-received, we will continue to offer more experimental challenges.
Ultimately, we want to keep improving our ability to find your best ideas and fund them!

Good luck and happy coding!

Disclaimer

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.

84 responses

Thomas,

As it stated above this challenge is for guidance dataset.
But only 17% of QTU has not null guidance records.
Is this challenge only for this universe of around 380 stocks?

Loading notebook preview...
Notebook previews are currently unavailable.

Vladimir: Yes, that's correct as most companies wouldn't provide guidance estimates.

Single Guidance derived factor:

Loading notebook preview...
Notebook previews are currently unavailable.

First Entry!!

Loading notebook preview...
Notebook previews are currently unavailable.

Is it allowed to mix the Guidance factor with other factors in order to obtain a larger universe (within the QTU)?

In the meanwhile, single factor algo.
First Commit :)

Loading notebook preview...
Notebook previews are currently unavailable.

@Gregorio: You can mix in other factors but only if there are predictive interactions between your non-guidance and your guidance factor. Thus, you won't be able to increase your universe size this way as all factors need to start with guidance.

Thank you Thomas, now everything is clear to me.
By the way, the factor that I posted is based on Guidance, so it is compliant with the rules on this contest.

@Gregorio and @Shiv,

Nice first cut for a single guidance factor. However, I noticed that your number of positions exceeds 500 names while the guidance datasets have only approximately 400 names. I suggest you go back to your code and check why this is happening. I'm sure with a slight modification in the way you process the factor, you'll be good to go. Good Luck!

@James
Thanks for your feedback. I checked the code for being sure and actually, in my opinion, it meets the contest criteria stated by Thomas.
However, the ultimate decision will be taken by Quantopian of course: in case it won't meet the criteria, I will be disqualified for sure.

Happy coding :)

Multi factor algo
-> First Commit

Loading notebook preview...
Notebook previews are currently unavailable.

Single factor algo
Second Commit

Loading notebook preview...
Notebook previews are currently unavailable.

My second model. Struggling to score all 350+ names with Guidance data. :(

Loading notebook preview...
Notebook previews are currently unavailable.

Here's my first take on a Guidance strategy.

Loading notebook preview...
Notebook previews are currently unavailable.

@Arun, You have posted the incorrect notebook. Use this one: alpha tearsheet

Another attempt with one additional factor.

Loading notebook preview...
Notebook previews are currently unavailable.

My third model, multi factor model.

Loading notebook preview...
Notebook previews are currently unavailable.

My fourth model, again multi factor.

Loading notebook preview...
Notebook previews are currently unavailable.

I will make also my fifth submission.
This one is less performing but it has a larger universe and (I think) it is compliant with the rules of this contest.

Loading notebook preview...
Notebook previews are currently unavailable.

My first cut.

Loading notebook preview...
Notebook previews are currently unavailable.

Revised Model 01

Loading notebook preview...
Notebook previews are currently unavailable.

Actuals & Guidance Model #1

Loading notebook preview...
Notebook previews are currently unavailable.

Model 2

Loading notebook preview...
Notebook previews are currently unavailable.

Third model's a charm?

Loading notebook preview...
Notebook previews are currently unavailable.

Actuals & Guidance Model #2

Loading notebook preview...
Notebook previews are currently unavailable.

Actuals & Guidance Model #3

Loading notebook preview...
Notebook previews are currently unavailable.

@Thomas Can we re use this one as a template?

Clone Algorithm
3
Loading...
Backtest from to with initial capital
Total Returns
--
Alpha
--
Beta
--
Sharpe
--
Sortino
--
Max Drawdown
--
Benchmark Returns
--
Volatility
--
Returns 1 Month 3 Month 6 Month 12 Month
Alpha 1 Month 3 Month 6 Month 12 Month
Beta 1 Month 3 Month 6 Month 12 Month
Sharpe 1 Month 3 Month 6 Month 12 Month
Sortino 1 Month 3 Month 6 Month 12 Month
Volatility 1 Month 3 Month 6 Month 12 Month
Max Drawdown 1 Month 3 Month 6 Month 12 Month
"""
Sample algo using estimates datasets for mini-contest
(Consensus, Actuals, Long Term, Broker Recommendations, Guidance)

This template is based on an example algorithm from Daniel Cascio.
"""

# Import required pipeline methods
from quantopian.pipeline import Pipeline
from quantopian.algorithm import attach_pipeline, pipeline_output

# Import any built in filters and/or factors
from quantopian.pipeline.filters import QTradableStocksUS

# Import datasets being used
from quantopian.pipeline.data.factset.estimates import (PeriodicConsensus, 
                                                        Actuals, 
                                                        ConsensusRecommendations as Consensus, 
                                                        LongTermConsensus, 
                                                        Guidance)

# Import optimize
import quantopian.optimize as opt

import pandas as pd

def initialize(context):
    # Normally a contest algo uses the default commission and slippage
    # This is unique and only required for this 'mini-contest'
    set_commission(commission.PerShare(cost=0.000, min_trade_cost=0))    
    set_slippage(slippage.FixedSlippage(spread=0))

    attach_pipeline(make_pipeline(context), 'sales_estimate_pipeline') 

    # Place orders towards the end of each day
    schedule_function(rebalance, date_rules.every_day(), time_rules.market_close(hours=2))
               
    # Record any custom data at the end of each day    
    schedule_function(record_positions, date_rules.every_day(), time_rules.market_close())

        
def create_factor():
    # Use the QTradableStocksUS universe as a base
    qtu = QTradableStocksUS()
    
    # Create an alpha factor
    # Must use at least one of the estimates datasets as the primary signal
    # (Consensus, Actuals, Long term, Broker recommendations, Guidance)
    # Replace this logic with your own factor(s).
    up = PeriodicConsensus.slice('SALES', 'qf', 1).up.latest
    down = PeriodicConsensus.slice('SALES', 'qf', 1).down.latest
    num_est = PeriodicConsensus.slice('SALES', 'qf', 1).num_est.latest
    alpha_factor = (up - down) / num_est
        
    # Filter out securities with very few estimates or factor is invalid
    screen = qtu & alpha_factor.isfinite() & (num_est > 2)
    
    return alpha_factor, screen

def make_pipeline(context):  
    alpha_factor, screen = create_factor()
    
    # Winsorize to remove extreme outliers
    alpha_winsorized = alpha_factor.winsorize(min_percentile=0.05,
                                              max_percentile=0.95,
                                              mask=screen)
    
    # Zscore to get long and short (positive and negative) alphas to use as weights
    alpha_zscore = alpha_winsorized.zscore()
    
    return Pipeline(columns={'alpha_factor': alpha_zscore}, 
                    screen=screen)

def rebalance(context, data): 
    # Get the alpha factor data from the pipeline output
    output = pipeline_output('sales_estimate_pipeline')
    alpha_factor = output.alpha_factor
    
    # Weight securities by their alpha factor
    # Divide by the abs of total weight to create a leverage of 1
    weights = alpha_factor / alpha_factor.abs().sum() 
    
    # Must use TargetWeights as an objective
    order_optimal_portfolio(
        objective=opt.TargetWeights(weights),
        constraints=[],
    )
    
def record_positions(context, data):
    pos = pd.Series()
    for position in context.portfolio.positions.itervalues():
        pos.loc[position.sid] = position.amount
        
    pos /= pos.abs().sum()
        
    quantiles = pos.quantile([.05, .25, .5, .75, .95]) * 100
    record(q05=quantiles[.05])
    record(q25=quantiles[.25])
    record(q50=quantiles[.5])
    record(q75=quantiles[.75])
    record(q95=quantiles[.95])
There was a runtime error.

I was curious as to what ranges of universe size to expect, so I wrote this little notebook to as a tool to estimate exactly that. Depending on how you do your factor combination, you may or may not drop vs. fill NaNs. In either case, this can help you to make sure your universe isn't wildly larger than expected!

Loading notebook preview...
Notebook previews are currently unavailable.

@Shiv Thanks for pointing that out.

Natural set, profitable and stable.

Loading notebook preview...
Notebook previews are currently unavailable.

Natural set with artwork.

Loading notebook preview...
Notebook previews are currently unavailable.

My 2nd submission using only 1 Guidance Factor. Not the highest IR in the forum but the exposures were decent.

Loading notebook preview...
Notebook previews are currently unavailable.

Model 02 - Two Guidance Factors and Actuals
Note: High Returns (70% - with 300 assets)

Loading notebook preview...
Notebook previews are currently unavailable.

Model 3

Loading notebook preview...
Notebook previews are currently unavailable.

Model 4

Loading notebook preview...
Notebook previews are currently unavailable.

Model 03 - More Stocks

Loading notebook preview...
Notebook previews are currently unavailable.

My final contribution to this challenge:

Loading notebook preview...
Notebook previews are currently unavailable.

Model 04

Loading notebook preview...
Notebook previews are currently unavailable.

Model 5 - ML

Loading notebook preview...
Notebook previews are currently unavailable.

Model 6 - ML (au natural)

Loading notebook preview...
Notebook previews are currently unavailable.

Tried to keep it simple here. Pure guidance. No actuals, estimates, etc. Sharpe isn't comparable to other users' entries, but I tried to keep my factors as clean as possible.

Loading notebook preview...
Notebook previews are currently unavailable.

My first attempt with 560 guidance universe.

Loading notebook preview...
Notebook previews are currently unavailable.

First attempt

Loading notebook preview...
Notebook previews are currently unavailable.

Pure guidance v2. No other datasets used.

Loading notebook preview...
Notebook previews are currently unavailable.

Pure guidance v3. Same factors as v2 but a wider universe.

Loading notebook preview...
Notebook previews are currently unavailable.

Submission # 3. This one exclusively uses 2 Pure Guidance Factors.

Loading notebook preview...
Notebook previews are currently unavailable.

4 Guidance factors - > removed one questionable factor and avoided overfitting to make it as predictive as possible

Loading notebook preview...
Notebook previews are currently unavailable.

Attempt 2

Loading notebook preview...
Notebook previews are currently unavailable.

My first submission. Not pretty but different. Backtest looks better than this notebook!
Uses one guidance factor only.

Start date 2015-06-02
End date 2018-10-01
Total months 40
Backtest
Annual return 7.8%
Cumulative returns 28.4%
Annual volatility 4.3%
Sharpe ratio 1.75
Calmar ratio 3.14
Stability 0.98
Max drawdown -2.5%
Omega ratio 1.37
Sortino ratio 2.66
Skew -0.10
Kurtosis 15.17
Tail ratio 1.12
Daily value at risk -0.5%
Gross leverage 1.01
Daily turnover 9.0%
Alpha 0.06
Beta 0.10

Loading notebook preview...
Notebook previews are currently unavailable.

Attempt 3

Loading notebook preview...
Notebook previews are currently unavailable.

Attempt 4

Loading notebook preview...
Notebook previews are currently unavailable.

first attempt

Loading notebook preview...
Notebook previews are currently unavailable.

Last Attempt!
This has only 3 factors with reduced drawdown, higher IR an relatively smaller universe in comparison to previous entries. Thanks!

Loading notebook preview...
Notebook previews are currently unavailable.

My Challenge Submission #2

Loading notebook preview...
Notebook previews are currently unavailable.

Attempt 5.

Loading notebook preview...
Notebook previews are currently unavailable.

Some participants in this challenge are concentrating on universe size (larger is better)
This algo has average set size 650.

Loading notebook preview...
Notebook previews are currently unavailable.

My 4th model.

[Edit: Improved 4th model below.]

Loading notebook preview...
Notebook previews are currently unavailable.

second attempt

Loading notebook preview...
Notebook previews are currently unavailable.

third and last

Loading notebook preview...
Notebook previews are currently unavailable.

My very first contribution. Thank you Quantopian and its members for creating this eco system for learning Quant finance.

Loading notebook preview...
Notebook previews are currently unavailable.

Second model

Loading notebook preview...
Notebook previews are currently unavailable.

Here is my attempt to utilize intersection of QTU with union of four Guidance qf

Loading notebook preview...
Notebook previews are currently unavailable.

Objective: Develop as pure Guidance factor as possible and uncorrelated signal to rest of the peers

Loading notebook preview...
Notebook previews are currently unavailable.

4th model refined. Factors use both quarterly and annual EPS and Sales guidance.

Loading notebook preview...
Notebook previews are currently unavailable.

Intersection of QTU with union of two Guidance quarterly factors.

Loading notebook preview...
Notebook previews are currently unavailable.

First entry

Loading notebook preview...
Notebook previews are currently unavailable.

Second entry using different technique to rank guidance estimates

Loading notebook preview...
Notebook previews are currently unavailable.

Version 2 of second entry

Loading notebook preview...
Notebook previews are currently unavailable.

Better late than never...combined guidance and consensus surprises.
PercHoldings requires investigation.

Loading notebook preview...
Notebook previews are currently unavailable.

Thanks for submitting everyone, we're very excited about all the entrants to the challenge! Over the next few weeks we will look through all of them and then do another live webinar to review the submissions and announce the winners. Stay tuned!

ok this is it for me

Loading notebook preview...
Notebook previews are currently unavailable.

Here is my attempt to use the intersection of QTU with a combination of four Guidance quarterly and annual factors.
Thanks to Joakim Arvidsson for the tip.

Loading notebook preview...
Notebook previews are currently unavailable.

Increased universe size.

Loading notebook preview...
Notebook previews are currently unavailable.

Unfortunately, I didn't have time to prepare something in time. But for the sake of the exercise, here is my contribution. The theme was to generate a signal for every asset in the Quantopian Universe.

Loading notebook preview...
Notebook previews are currently unavailable.

The tearsheet review and winners announcement webinar for this challenge will be at 9am EST on Monday, November 18th, 2019. You can watch the webinar here: http://bit.ly/guidanceweb.

Disclaimer

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.

Our tearsheet review and winners announcement webinar is happening now. Tune in at http://bit.ly/guidanceweb.

Hi Thomas and Ang ,

Just had the chance to watch the tearsheet review. Although ranked number 6, I am glad to be mentioned multiple times and the potential opportunity of licenced algos, particularly for the fact that three of my four models landed in 3 different algo clusters, and two of them continually performed well in the out-sample data. I am relatively new as a python programmer and learned a lot from everyone here at the Q site. Thanks a lot for the opportunities and am looking forward to contribute more in the near future!

@Kuwa Wen: Surprise! You are also getting a $100 honorary prize (in addition to us wanting to license the factor from you) because we really liked your algorithm and want to welcome new users that show great potential like yourself. Glad to hear that your experience here so far was a positive one of learning.

The other winners are:

  • Vedran Rusman
  • Vladimir
  • Indigo Monkey
  • James Villa
  • Antony Jackson

Please join me in congratulating them on the amazing intellectual work that goes into cracking these challenges. This of course extends to everyone who submitted to this challenge, so thank you all and we hope to see you all for next challenge: https://www.quantopian.com/posts/financials-tearsheet-challenge.

Using cluster correlation to weed out other equally good entries in the same cluster doesn't make clear cut sense. If one entry is correlated with another entry but is an independent submission, why they should get a pie of the prize or the allocation.

At the end of the day, it's Q's discretion but it undermines the effort made. Do you agree?

@Thomas Wiecki,
То restore a justice, may I request to share my second prize with others in my correlation cluster?

A quick reality check about the "out of sample" period.

Participants in these challenges are bright enough to submit to the main contest to establish how the algorithm performs in the "hold out" data.

With this in mind, this new criterion for licensing puts more weight on the most recent 1 year of data. This may be a good thing --- a test of whether the idea has not been crowded out --- but we should perhaps exercise caution in viewing it as an out-of-sample test.

@Thomas
Can you share how you combined the algorithms? I think this would be interest to everyone on this thread.

@Antony: Yes, you're right, although the number of times it can be tested is limited.

@Arun: I just equal weighted them.

Thomas,

I have revisited my submission which was eliminated based on the mean of alpha decay in earlier tear sheet review and tried improving it.
Is there any chance to re look at same. Attached the improved version tear sheet.

Appreciate the feedback.

Loading notebook preview...
Notebook previews are currently unavailable.