Firstly. thanks for your acknowledgement regarding " .... what both you and I think is the crux of formulating a successful trading system" Basically with that piece of info, if known reliably, everything else gets remarkably easy, as long as one has the discipline to adhere to the caveat: "... and if in doubt then just stay out" :-)
I have read all of John Ehlers books and experimented quite a bit with all of the ideas in them. His first work on MESA was adapted from a technique used in geophysical processing for oil exploration (also an area that I worked in). Most of Ehlers filtering ideas and things like Hilbert transforms come directly from Electrical Engineering & DSP. All of these techniques are valid, widely used and well documented in Engineering literature & textbooks. Ehlers innovation was to apply these ideas to trading, based on the notion that bars of trading data are like digitally sampled signals. In that area Ehlers has certainly created a niche and a reputation for himself. I like a lot of his ideas. The ones related to low-pass filtering are particularly good. Butterworth filters are well known within Electrical Engineering, but Ehlers modified 2nd order Butterworth filter is the best minimum distortion, maximally flat within its passband, compact, minimum lag smoothing filter that I know of (even better than his "Super-Smoother").
I understand Ehlers ideas very well, and in particular his considerations of signal-to-noise. However I do have two mild criticisms of his work. The first is a simple practical one. In the signals that Electrical Engineers deal with there are often many cycles (for example of a carrier wave with some fixed frequency) with an amplitude or frequency that is modulated at a slower rate (for example by an audio signal in AM or FM radio). All the DSP ideas that Ehlers uses work very well in those situations. However in trading the difference is that we do not have a large number (e.g. hundreds or thousands) of underlying cycles to be able to process. In fact the most we usually ever see is about two cycles with a decaying envelope in the case of a classical triangle pattern. In fact often we don't even get one full cycle of price data before traders figure out what is happening, trade in anticipation of the cycle completion, and thereby destroy it. This represents a major limitation and, at least as i see it, the first breakdown in the assumption that securities price data can be adequately treated with conventional Engineering DSP methods.
The second problem (criticism) that i have is a little more subtle. In conventional DSP we usually have a data stream that is a mixture of a signal which is either made by humans (e.g. voice, music, the trajectory of a vehicle or projectile, etc) or by nature (e.g. the geology of a sedimentary basin containing oil, etc) in some reasonably uniform way, and in either case there are some inherent regularities because of the process that generated the signal. Then, contaminating the wanted signal, there is inevitably added some unwanted noise, so what we observe is a mixture of signal + noise. Usually we can easily tell what is signal and what is noise, and the job of filtering or DSP is to separate them. In the context of trading this is NOT so easy; What really is "signal" and what really is "noise"? This is not just a philosophical question. Some people say that in trading there is no such thing as "noise". Experts in Price Action Trading, such as Al Brooks for example, contend that ALL price bars contain meaningful information for trading and none of them are "noise".
Personally i also have another philosophical issue with the idea of "signal". In trading data, we have (mostly reasonably) sentient beings or their algos, continually trying to outguess what the market is about to do next and responding as fast as they can within their own individual time-frames. This degree of responsiveness of the target in trading is very different to the usual "signals" that Engineering DSP methods have to deal with. Just imagine if the Earth's geology, or a piece of music was self-aware and was continually trying to "trick" the Engineering-types into mis-predicting it!! Traders need to be very careful about what they call "signal" and what they call "noise".
With regard to Ehlers quote that you provide, @James, the words that stand out most to me are: " ... accurate measurements of the dominant cycle ..."
A simple and often quite useful conceptual model of market data is that it consists of a trend component, a cycle component, and noise. It is often quite a good model and one i have experimented with a lot. I have read Ehler's comments that he believes that usually there is only ONE dominant cycle. I think this is his preferred conceptual model rather than a statement of fact. Even if it were true, then the period (and/or phase) of the "dominant cycle" are non-stationary and keep drifting. Anyone who has seriously tried to use the trend+cycle+noise model for trading has found that determining the varying cycle period is difficult. Anyone who has done careful spectral analysis of market data has seen that there is usually more than one significant cycle period in play, and these are not always just the harmonics that give rise to all the usual Fourier synthesis effects like double tops, classical H&S patterns, etc.
My own experience with Ehlers Hilbert transform (which I coded from one of his books) is that for trading data it just does not work as well as it does in its more usual domain of applicability in Electrical Engineering DSP, and again the reasons for that are as mentioned earlier.
Although Ehlers may seem enigmatic in some ways, i think there is an underlying explanation. Although Ehlers may (or may not, i don't know) be a trader, first and foremost he is an author, and presumably also a consultant and still a marketer of his software. As with any kind of marketing, ideas that have been around for a long time often benefit from some rejuvenation or change. MESA has now been around for a long time and in the trading area MESA is very much Ehlers software. Maybe he just figures it is time for something new.
My conclusion as to why Ehlers wrote more than 10 pages about H in his book and then most ambiguously concluded that " it is useless" / "it is useful" / "it is not", was probably he thought that SOMEONE would be interested in it and it might help to sell his book (which it did .... to me at least ;-))
Autocorrelation periodograms are interesting. In EasyLanguage in one of his books, but not very difficult to re-write in other languages, Ehlers has some code that produces interesting 2-D visual display plots. I tried to improve on it using python with some of its scientific libraries. Although conceptually easy, i found in practice that it was difficult to get from a frequency vs time display to a period vs time display in python, but that was probably just a reflection of my own very limited skills with python library tools. However that's probably getting a bit off topic now. If you want to take up the topic of periodograms further offline, then most welcome to email me at [email protected] or alternatively let's break from this post about H and start a new one.
Cheers, all the best, Tony