Back to Community
Higher IC doesn't lead to better backtest performance

I have been researching multiple alpha factors and with each additional alpha factor, the IC goes up. However, the backtest results don't reflect the improved predictability.

I have two scenarios with combination of around 14 alpha factors each. In the results below, Scenario 1 has better metrics but its backtest performance is far inferior in comparison to Scenario 2.

How do I explain this performance gap?
Is the portfolio construction process sub-optimal? I am currently using Maximize Alpha with vanilla risk exposure and leverage constraints.

Scenario - 1 (Forward 5 Day Returns)
Date Range: Jan 2015 - May 2019
Num. Factors (14)
Linear Combination Coefficient (1.0)
Alpha (0.066)
Beta (-0.237)
Mean IC (0.026)
IC Std (0.139)
Risk Adj. IC (0.189)
t-stat (6.207)
Spread-bps (19.482)
Backtest Performance - Total Return - 34.84%/Sharpe - 0.95/ Max Drawdown - 12.84%

Scenario - 2 (Forward 5 Day Returns)
Date Range: Jan 2015 - May 2019
Num. Factors (14)
Linear Combination Coefficient (1.0)
Alpha (0.063)
Beta (-0.244)
Mean IC (0.023)
IC Std (0.146)
Risk Adj. IC (0.156)
t-stat (5.132)
Spread-bps (16.725)
Backtest Performance - Total Return - 58.96%/Sharpe - 1.30/Max Drawdown - 9.53%

Average correlation between net alpha (simple linear combination of factors) from scenario1 and scenario2 ~ 0.87

5 responses

I'm pretty sure that IC only takes into account accuracy, not profitability.

e.g.

Algo 1:
Trade 1: -$5
Trade 2: +$5

Profit: $0
IC: 0.0

Algo 2:
Trade 1: -$5
Trade 2: +$1,000

Profit: $995
IC: 0.0

I agree partly.

In context of Alphalens, I guess the definition of IC is correlation of exposure (equal weighted?) and holding period returns. And If it goes higher, it means that predictability improves (or accuracy as suggested above)

Ideally, improved predictability should lead to improved profitability (As optimizer will appropriately chose weights to maximize alpha). And as a researcher, the goal should be improve predictability as suggested by Q in some posts as well (otherwise it will lead to overfitting).

Seems like portfolio construction introduces the performance gap as it has to satisfy constraints that are not taken into account in Alphalens. Could that be or is it something else?

Having an IC standard deviation 5 times larger than your IC mean does not make the IC mean that useful, especially, when your mean IC is close to zero.

What is the typical Mean IC (st. dev.) one should expect in alpha prediction? Are these IC number meaningless?

Each factor should past the IC t-stat test or pval, so that the IC number is not meaningless. Using a noisy factor to predict the future based on past results will lead to varying results good and bad. The mean IC you are getting are typical for 5 day time frame.