Technical Analysis Difficulties: Overoptimization

Exploring The Various Psychological Reasons Behind Technical Analysis Difficulties

One Reason Why Technical Analysis Is So Difficult

When you show a stock chart to 10 different analysts you are more than likely to see 10 different conclusions about where it is going. Yes, technical analysis is difficult not only because of the mountain of tools and indicators involved but to make things even harder, the psychology of the person performing the technical analysis makes a huge difference as well! In fact, two persons employing the exact same indicators and tools look at the exact same chart on the exact same time frame is more than likely going to have two different conclusions! One of the factors that makes technical analysis so difficult for beginners and veterans alike is the tendency to try to overoptimize your setups in order to find the "Holy Grail".

Overoptimization - When Does It Happen in Technical Analysis?

Now, assuming that you have overcome all of the rest of the issues outlined in the Options Insights For Veteran Traders page and have reached a stage where you are able to use a good technical analysis set of tools and indicators to arrive at a decently high level of accuracy, this is when the tendency to overoptimize and break the whole game happens. In fact, this is the exact polar opposite of the tendency to Rely on a Singular Indicator.

First of all, what is a "decently high level of accuracy" in technical analysis?

In my experiences through the decades, a decently high level of accuracy is about a 70% to 80% accuracy of predicting future stock movement. In fact, academically, anything above 60% accuracy is deemed to be a cash cow where you simply have to follow for the long term to result in a net profit. However, this is when most beginners options traders and veteran options traders alike start to "get smart" and try to overoptimize their setups.

What Exactly is Overoptimization in Technical Analysis?

Optimization in the simplest form is attempting to add more and more indicators to an existing technical analysis setup in hope that by doing so, accuracy increases and eventually, a 100% winning holy grail combination of technical indicators results. The real danger in doing so is not that you won't eventually arrive at an almost 100% fitting and winning combination on a specific stock chart but that the 100% fit ONLY HAPPENS for that specific stock chart and only for its historical data, not even new data going forward for the same stock! Its like drawing an outline on an existing cartoon picture, its not hard to do. But now, try fitting that exact same outline on another cartoon picture! That right there is the problem! You THINK you've got it but all you really got is a lie that works only for that particular stock chart you were working on (stock + timeframe). The moment you try it on another timeframe or another stock / timeframe, it falls apart drastically. In fact, it won't even work as good as what you originally had before you tried to overoptimize it. Optimization is like climbing a pyramid. When you try to get too close to the peak, you fall off the other side instead.

Overoptimization - The Search For The Holy Grail

What leads to overoptimization is really a mental one, the search for the elusive and non existent "Holy Grail". The refusal to accept having to manage risks and learn smart money management strategies that allows a real options trader to minimize losses on losing trades and maximize profits on winning trades. The dream that once you "get it", you can put all your money into it and compound it into millions in a few short month and "turn your life around". In fact, we can see that there are two problematic mindsets at work here that leads to the tendency to overoptimize in technical analysis; One, the search for the Holy Grail. Two, the gambler mindset where you just want to hit a jackpot to turn your life around. Both of these mindsets are exactly the kind of mindset that leads to every problem in options trading and investment overall. Meaning, these are exactly the mindset which will lead an options trader to failure, not only through overoptimization, but leading to every single mistake in the book. The bottom line here really is this; If you cannot afford to lose, you cannot afford to win. Accepting that there is no holy grail and that we will always need to be smart with risk management is how professional traders survive in the long term, not some magical 100% winning formula. Professionals never need a 100% winning formula to win for the long term, on the other hand, traders who seek only the 100% winning formula never wins.

Overoptimization - What Is The Solution?

So, what is the solution to the problem of overoptimization in technical analysis? Should we never try to optimize our setups?

This is the tricky part, one should always seek to optimize the technical analysis setups we use especially for different stocks and timeframes. However, as I have said above, the problem is more of a mental one. If you have the two mental problems outlined above, you would most likely overoptimize. However, if you are simply going through the normal process of getting to a healthy accuracy and tweaking it to better suit changing conditions and stock behaviors without the idea of trying to create a holy grail, most of the time you would be fine. One simple guideline really is, you should never need more than 3 indicators for good technical analysis. However, even this is too simplistic a guideline but may serve as a good red flag for beginner options traders. When you try to add more and more to your technical analysis, you know you are crossing over to the "twilight zone" and it should raise a red flag in your mind.

Ultimately, investment is a highly mental game rather than a methodological one. The mental can make or break the game and all options traders need to acknowledge this as a fact and develop an awareness and an emphasis to understanding more about this aspect before trying to pile on more and more technical knowledge.

Explore another very common psychological reason which makes technical analysis so difficult: Hope vs Reality.