Back to Community
Why Does TA-Lib Method EMA Have the Same Result as SMA? It looks not right.

Why does the TA-Lib method EMA return the same result as SMA? It looks not right for EMA. See the following sample code to test EMA and SMA suing Apple daily price (3/01/2014 to 4/17/2014). All the three MAs have the same result.

sma = ta.SMA(timeperiod=10)  
ema = ta.EMA(timeperiod=10)  
ma = ta.MA (timeperiod=10,matype = 1 )

def initialize(context):  
    context.stock = sid(24) # Apple

# Will be called on every trade event for the securities you specify.  
def handle_data(context, data):  
    stock = context.stock  
    ema_data = ema(data)  
    sma_data = sma(data)  
    ma_data = ma(data)  
    price = data[stock].price  
    curr_sma = sma_data[stock]  
    curr_ema = ema_data[stock]  
    curr_ma = ma_data[stock]

    log.info("price: " + str(price) +"  sma: " + str(curr_sma) + "  ema: " + str(curr_ema) + "  ma: " + str(curr_ma) )  
    return  
20 responses

Below is the output for the test code:

014-03-03handle_data:24INFOprice: 527.75  sma: nan  ema: nan  ma: nan  
2014-03-04handle_data:24INFOprice: 531.24  sma: nan  ema: nan  ma: nan  
2014-03-05handle_data:24INFOprice: 532.36  sma: nan  ema: nan  ma: nan  
2014-03-06handle_data:24INFOprice: 530.82  sma: nan  ema: nan  ma: nan  
2014-03-07handle_data:24INFOprice: 530.43  sma: nan  ema: nan  ma: nan  
2014-03-10handle_data:24INFOprice: 530.92  sma: nan  ema: nan  ma: nan  
2014-03-11handle_data:24INFOprice: 536.1099  sma: nan  ema: nan  ma: nan  
2014-03-12handle_data:24INFOprice: 536.6  sma: nan  ema: nan  ma: nan  
2014-03-13handle_data:24INFOprice: 530.68  sma: nan  ema: nan  ma: nan  
2014-03-14handle_data:24INFOprice: 524.66  sma: 531.15699  ema: 531.15699  ma: 531.15699  
2014-03-17handle_data:24INFOprice: 526.54  sma: 531.03599  ema: 531.03599  ma: 531.03599  
2014-03-18handle_data:24INFOprice: 531.365  sma: 531.04849  ema: 531.04849  ma: 531.04849  
2014-03-19handle_data:24INFOprice: 531.26  sma: 530.93849  ema: 530.93849  ma: 530.93849  
2014-03-20handle_data:24INFOprice: 528.76  sma: 530.73249  ema: 530.73249  ma: 530.73249  
2014-03-21handle_data:24INFOprice: 531.63  sma: 530.85249  ema: 530.85249  ma: 530.85249  
2014-03-24handle_data:24INFOprice: 539.15  sma: 531.67549  ema: 531.67549  ma: 531.67549  
2014-03-25handle_data:24INFOprice: 545.0  sma: 532.5645  ema: 532.5645  ma: 532.5645  
2014-03-26handle_data:24INFOprice: 539.78  sma: 532.8825  ema: 532.8825  ma: 532.8825  
2014-03-27handle_data:24INFOprice: 537.425  sma: 533.557  ema: 533.557  ma: 533.557  
2014-03-28handle_data:24INFOprice: 536.86  sma: 534.777  ema: 534.777  ma: 534.777  
2014-03-31handle_data:24INFOprice: 536.74  sma: 535.797  ema: 535.797  ma: 535.797  
2014-04-01handle_data:24INFOprice: 541.74  sma: 536.8345  ema: 536.8345  ma: 536.8345  
2014-04-02handle_data:24INFOprice: 542.55  sma: 537.9635  ema: 537.9635  ma: 537.9635  
2014-04-03handle_data:24INFOprice: 538.825  sma: 538.97  ema: 538.97  ma: 538.97  
2014-04-04handle_data:24INFOprice: 531.83  sma: 538.99  ema: 538.99  ma: 538.99  
2014-04-07handle_data:24INFOprice: 523.548  sma: 537.4298  ema: 537.4298  ma: 537.4298  
2014-04-08handle_data:24INFOprice: 523.43  sma: 535.2728  ema: 535.2728  ma: 535.2728  
2014-04-09handle_data:24INFOprice: 530.31  sma: 534.3258  ema: 534.3258  ma: 534.3258  
2014-04-10handle_data:24INFOprice: 523.48  sma: 532.9313  ema: 532.9313  ma: 532.9313  
2014-04-11handle_data:24INFOprice: 519.64  sma: 531.2093  ema: 531.2093  ma: 531.2093  
2014-04-14handle_data:24INFOprice: 521.68  sma: 529.7033  ema: 529.7033  ma: 529.7033  
2014-04-15handle_data:24INFOprice: 517.9  sma: 527.3193  ema: 527.3193  ma: 527.3193  
2014-04-16handle_data:24INFOprice: 518.95  sma: 524.9593  ema: 524.9593  ma: 524.9593  
2014-04-17handle_data:24INFOprice: 524.99  sma: 523.5758  ema: 523.5758  ma: 523.5758  

fyi a couple days ago i reported a bug to quantopian , that's perhaps related. data[sid].mavg(7) gives the same result as data[sid].mavg(30)

no reply from quantopian yet, but i submitted the bug report friday night

Indeed, I did noticed this error few months ago. It is not related to quantopian, but ta-lib (the c underlying library). I tried to contact them, but got no response.

See the issue I raised on the python ta-lib wrapper :
https://github.com/mrjbq7/ta-lib/issues/52

fyi the mitigation to this bug is to compute the values manually, though you either need to use the history / batch transforms, or store the prices manually in an array. (and use numpy)

Makes you concerned that if averages are wrong how much other underlying match is wrong. I'd hate to figure out some logic is incorrect as it takes a live portfolio and drives the equity to $0

I use the dotnet version of TA-LIB and the values are different.

The example on github may not be the best example. It looks like the both the SMA and EMA on a linear series will converge to the same values. So a short MA period will converge fast and the values will be the same. I used a longer period (30) on a longer data series and the moving average values for SMA and EMA differ initially then converge to the same value.

For the Python TA-Lib wrapper 0.4.8, the SMA and EMA are different (see below). I am wondering how Quantopain integrates TA-Lib into this platform. My big concern is how can we sure that all other TA indicators based EMA, i.e. MACD, have the CORRECT values.

 SMA = [       nan        nan        nan        nan        nan        nan  
        nan        nan        nan  531.15699  531.03599  531.04849  
  530.93849  530.73249  530.85249  531.67549  532.5645   532.8825   533.557  
  534.777    535.797    536.8345   537.9635   538.97     538.99     537.4298  
  535.2728   534.3258   532.9313   531.2093   529.7033   527.3193   524.9593  
  523.5758 ]

EMA = [          nan           nan           nan           nan           nan  
           nan           nan           nan           nan  531.15699  
  530.31753727  530.50798504  530.64471503  530.30203957  530.54348692  
  532.10830748  534.45225158  535.42093311  535.78530891  535.98070729  
  536.11876051  537.14080405  538.12429422  538.25169527  537.08411432  
  534.62300262  532.58791124  532.17374556  530.59306455  528.60159827  
  527.34312585  525.62619388  524.41234045  524.51736946]  

perhaps you should manually compute everything.

fyi i updated my framework's StandardTechnicalIndicators to manually compute. you may consider doing something similar: https://www.quantopian.com/posts/a-r-and-d-framework-for-quantopian

also if one of the quantopian people read this, I'm pretty sure the data[sid].mavg() function worked properly a couple months ago. so perhaps there's an older version of the codebase you could revert to.

Hello Luke/Jason,

I'm not so sure you should be so sceptical about TA-Lib (the library). It's very widely used and something as basic as a miscalculation of EMA would have been found years ago since ta_EMA.c dates from 2000 or earlier. The library is used by many commercial platforms - sometimes publicised as such and sometimes not - including Dukascopy's Forex platforms.

Here is a test in IPython:

import datetime  
import numpy as np  
import pandas as pd  
import pandas.io.data  
import talib

aapl = pd.io.data.get_data_yahoo('AAPL',start=datetime.datetime(2006, 10, 1), end  =datetime.datetime(2006, 12, 31))  
print talib.SMA(np.array(aapl['Close']), 50)  
print talib.EMA(np.array(aapl['Close']), 50)  

The output is:

[     nan      nan      nan      nan      nan      nan      nan      nan
      nan      nan      nan      nan      nan      nan      nan      nan  
      nan      nan      nan      nan      nan      nan      nan      nan  
      nan      nan      nan      nan      nan      nan      nan      nan  
      nan      nan      nan      nan      nan      nan      nan      nan  
      nan      nan      nan      nan      nan      nan      nan      nan  
      nan  82.332   82.5576  82.857   83.1204  83.3782  83.6032  83.8368  
  84.0558  84.2492  84.388   84.5178  84.6402  84.7718  84.978 ]

[         nan          nan          nan          nan          nan
          nan          nan          nan          nan          nan  
          nan          nan          nan          nan          nan  
          nan          nan          nan          nan          nan  
          nan          nan          nan          nan          nan  
          nan          nan          nan          nan          nan  
          nan          nan          nan          nan          nan  
          nan          nan          nan          nan          nan  
          nan          nan          nan          nan          nan  
          nan          nan          nan          nan  82.332       82.48133333  
  82.7389281   82.96681328  83.15321276  83.24406716  83.36429982  
  83.41903316  83.39867892  83.3516719   83.27944947  83.21045145  
  83.11866904  83.18617222]  

Another thought occurs to me specific - and uncommon - to the Quantopian implementation of TA-Lib (the library) in that it is implemented to avoid the 'memory effect' of indicators like EMA (see http://ta-lib.org/d_api/ta_setunstableperiod.html). I don't know enough about this but it feels like this may be involved somehow.

P.

Hello Luke,

These are the results from IPython with the same data as your test i.e. Apple daily price (3/01/2014 to 4/17/2014):

aapl = pd.io.data.get_data_yahoo('AAPL',start=datetime.datetime(2014, 03, 1), end  =datetime.datetime(2014, 04, 17)  

SMA
[ nan nan nan nan nan nan nan nan nan 531.151 531.049 531.065 530.955 530.75 530.993 531.82
532.71 533.027 533.708 534.925 535.925 536.95 538.079 539.088
538.983 537.411 535.256 534.31 532.912 531.187 529.681 527.312
524.958 523.573]

EMA
[ nan nan nan nan nan nan nan nan nan 531.151 530.349
530.54009091 530.67098347 530.31262284 530.77760051 532.30712769
534.61310447 535.55254002 535.89935093 536.07401439 536.19510269
537.1869022 538.16201089 538.27619073 537.10233787 534.62373098
532.59032535 532.17753892 530.59616821 528.59868308 527.3407407
525.63515148 524.43057849 524.52320058]

I'm concluding that there is no issue with the EMA calculations in TA-Lib (the library) or introduced by talib (the wrapper). I'm going to go a little further and say that the re-writing of TA-Lib (the library) functions is entirely wasted effort - unless of course someone is going to do ALL of them in which case I look forward to seeing the final results in several years time. Here is such an ongoing (stalled since 2012?) effort: https://code.google.com/p/ultra-finance/wiki/pyTaLib

P.

OK, i did an experiment and I see that for my issue, it's me misunderstanding the data[sid].mavg() function. it gives results for DAYS, not frames. so on day 1 all mavg values will be the same.

this also may be the same thing with ta lib... it gives mavg for days and not minutes?

here is my experiment:

Clone Algorithm
15
Loading...
Backtest from to with initial capital
Total Returns
--
Alpha
--
Beta
--
Sharpe
--
Sortino
--
Max Drawdown
--
Benchmark Returns
--
Volatility
--
Returns 1 Month 3 Month 6 Month 12 Month
Alpha 1 Month 3 Month 6 Month 12 Month
Beta 1 Month 3 Month 6 Month 12 Month
Sharpe 1 Month 3 Month 6 Month 12 Month
Sortino 1 Month 3 Month 6 Month 12 Month
Volatility 1 Month 3 Month 6 Month 12 Month
Max Drawdown 1 Month 3 Month 6 Month 12 Month
# Backtest ID: 5356449615d5400720848c65
There was a runtime error.

actually for the talib, you'll have the same results on timestep 0 and 1 anyway, because you only have 1 sample to average

Hello Jason,

TA-Lib can be imported and applied to 'history' data (see attached). I can't recall the status of the mavg() method - is this 'warmed up' in live but not backtesting?

P.

Clone Algorithm
10
Loading...
Backtest from to with initial capital
Total Returns
--
Alpha
--
Beta
--
Sharpe
--
Sortino
--
Max Drawdown
--
Benchmark Returns
--
Volatility
--
Returns 1 Month 3 Month 6 Month 12 Month
Alpha 1 Month 3 Month 6 Month 12 Month
Beta 1 Month 3 Month 6 Month 12 Month
Sharpe 1 Month 3 Month 6 Month 12 Month
Sortino 1 Month 3 Month 6 Month 12 Month
Volatility 1 Month 3 Month 6 Month 12 Month
Max Drawdown 1 Month 3 Month 6 Month 12 Month
# Backtest ID: 5356525fbbc4d0072415f85b
There was a runtime error.

Hello Peter,

Thanks for your input. I totally agree with you about the reputation of the TA-Lib. If you read my original post and follow-up replies, you will find out that I am not sceptical about TA-Lib itself. I am questioning about the Quantopain's implementation of the TA-Lib. The test result presented in the begin of this post clearly showed that the Quantopain's TA-lib EMA is not right. Since the EMA and SMA have the exactly same values, I don't think that undesired EMA result is caused by the 'memory effect' or the the 34-day fixed window length approach.

Because so many posts concern about the discrepancy of the some indicators, i.e. RSI, MACD, from other systems, and all the explanations of the discrepancies on this website are due to the 'memory effect' or the 34-day fixed window length algorithm implementation. This inspire me to check what the 34-day fixed window length algorithm of EMA is and see how it effects other indicators derived from EMA. It was really surprise for me when I saw the Quantopain's TA-Lib implementation of SAM and EMA have the exact same result. I wish Quantopain can tell me why and help me to eliminate my doubt about other Indicators which are based on the EMA.

BTW, regarding the 34-day fixed window length algorithm for EMA, since there are so many ways to implement. For the clarification purpose, could the people from Quantopain explain us what is the exactly 34-day fixed window length algorithm of EMA you used in Quantopain system?

Thanks,

I noticed this issue a while ago when I first read the Common Usage: Using TA-Lib section of the Quantopian Overview.

Bottom line: Don't use history windows to generate exponential moving average calculations (e.g. EMA, MACD, etc)

Common Usage: Using TA-Lib Since history returns a pandas DataFrame, a Series can be extracted and then passed to TA-Lib.

An example EMA calculation:

# Python TA-Lib wrapper  
# https://github.com/mrjbq7/ta-lib  
import talib

def initialize(context):  
    context.my_sid = sid(24)

def handle_data(context, data):  
    price_history = history(bar_count=30, frequency='1d', field='price')  
    my_sid_series = price_history[context.my_sid]  
    ema_result = talib.EMA(my_sid_series)  
    record(ema=ema_result[-1])  

All exponential moving averages have permanent 'memory'. In order to calculate a 60 day EMA, for example, I only need the value of the current observations (e.g. closing price), the most recent EMA value (which contains the memory of all past values with exponentially decaying weights), and the decay factor alpha (e.g. 1 / 60 in this example). Although EMA values tend to converge over time (how quickly depends on the alpha parameter), users will obtain different EMA values depending on when they started their observations. Today's observation of the 60-day EMA for SPY will be different if I use one year of SPY history for one example and two year's of history for the other.

The history implementation above, however, will have a subtle drift. When the EMA figure is calculated in TA-Lib, you will get NA until you have enough observations (e.g. 60 data points in a 60 day EMA). On the 60th day, TA-Lib gives you the simple average of these data points for the EMA. This value is used to 'seed' the EMA calculation for all values subsequently returned. Note that the 60-day EMA and simple MA (SMA) will be identical on this initial data point of the 60th observation.

HOWEVER, If one were to use a 60-day history window to generate both the EMA and the SMA in the example above, then the values will be identical throughout the testing window. This is because you are losing the memory properties of the EMA by using a sliding history window and just using the most recent values (which in the extreme simply gives you the simple moving average).

Hope this helps to clarify.

I still don't understand why SMA and EMA values below are identical, even when you run the algorithm for a backtesting window of 3 years. Surely that can't be right, no?

Clone Algorithm
5
Loading...
Backtest from to with initial capital
Total Returns
--
Alpha
--
Beta
--
Sharpe
--
Sortino
--
Max Drawdown
--
Benchmark Returns
--
Volatility
--
Returns 1 Month 3 Month 6 Month 12 Month
Alpha 1 Month 3 Month 6 Month 12 Month
Beta 1 Month 3 Month 6 Month 12 Month
Sharpe 1 Month 3 Month 6 Month 12 Month
Sortino 1 Month 3 Month 6 Month 12 Month
Volatility 1 Month 3 Month 6 Month 12 Month
Max Drawdown 1 Month 3 Month 6 Month 12 Month
# Backtest ID: 5479ad934d3fa108f95c2567
There was a runtime error.

Mete,

An EMA of window 30 requires more than 30 days of data to be any different from an SMA.

EDIT: It seems if you don't specify 'timeperiod' then the default window for SMA and EMA is 30, so the algo in the help doc "example EMA calculation" is the same as an SMA because the history() function is being called with only 30 days.

Clone Algorithm
22
Loading...
Backtest from to with initial capital
Total Returns
--
Alpha
--
Beta
--
Sharpe
--
Sortino
--
Max Drawdown
--
Benchmark Returns
--
Volatility
--
Returns 1 Month 3 Month 6 Month 12 Month
Alpha 1 Month 3 Month 6 Month 12 Month
Beta 1 Month 3 Month 6 Month 12 Month
Sharpe 1 Month 3 Month 6 Month 12 Month
Sortino 1 Month 3 Month 6 Month 12 Month
Volatility 1 Month 3 Month 6 Month 12 Month
Max Drawdown 1 Month 3 Month 6 Month 12 Month
# Backtest ID: 5479e7fb527d140900b539e1
There was a runtime error.

Hi James, thanks for the nice example, SMA and EMA difference is clear now. I think the example EMA calculation in the help docs is quite misleading, as it's simply an SMA example, rather than EMA example. They should have used a different bar_count or a different window size for it to make sense as an EMA example.

Agreed - I've emailed them about it

It's not a bug from TA-Lib. I'm using TA-Lib with C# and I was experiencing same symptons. Checking the internet I was thinking it was a problem about the setunstableperiod method. But now I've got it. I made a mistake about ordering the data. I was putting the newest on the first position but to works correctly you need to get the oldest as the first one.

List rates = (from Rate r in context.Rates
where r.Security.Symbol == setup.Symbol &&
r.Timeframe == setup.Timeframe
orderby r.DateTime ascending
select r).ToList();

it as descending