Hi @James, thanks for your nice "welcome back". Yes, i'm very much still around, but have been out of the loop here at Q since the end of last year. Looking forward to getting back into it again ASAP. Cheers!
;-)) well even if you categorically disagree with my bold-faced statement that "automated optimizations are great for running quick scans to see the ranges of parameter values that DON'T work", actually I think you may find that we are both very much on the same page!
I deliberately chose to express my comment in a way that may seem somewhat provocative, and I will explain why. The optimizers in the commercial packages are great tools for sorting through a lot of ideas quickly and putting them into to different groups, right? I do that too. As we both know, some of those ideas get kept, some get quickly thrown out. For me too, this is also usually an important step on the "long and painful R&D road". However what I see lots of beginners doing (and I used to do it myself) was working and re-working the optimizer to get the "best" solution and then imagining that I had something real ... instead of just a useless illusion based on over-fitting. From your follow-up comments, you are obviously way past that. Nowadays, after a few decades, I like to work some of the conventional ideas of trading system design & development from a "backwards" perspective, i.e. filtering to get rid of as many bad ideas as fast as possible, rather than filtering to try find good ideas. Its mostly just a matter of perspective & efficiency, and in the end I think we are at a similar place.
I concur with all your comments about your own findings, with just one possible exception. That is the topic of conclusively identifying variables that drive performance. In addition to the fact that this may change over time, particularly with regard to Style (e.g. one year "Growth" is the most fashionable, another year it is "Value", etc), this problem mostly (I think) comes down conceptually to identifying Information Content based on some form of Feature Extraction process. Yes, I'm in this for the money too, not for intellectual wanking. In the past I played around a lot with NN, GA, different forms of ML, PSA & other stuff in addition to conventional optimization. I never got as far as I had hoped in conclusively identifying drivers in non-linear systems. If you have any comments or would like to discuss & share ideas offline, yes, please. "Over coffee in your office" might perhaps be a bit difficult as I choose to live on Penang Island, Malaysia, but please feel welcome to contact me if you want: [email protected]
Cheers, all the best, TonyM.