Analysis of Directional Movement as a Complex System
Continuing on my work in the recent white paper, Directional Movement Indicators using n Range, this researcher was trying to use DM to generate earlier and more reliable trading signals.
Directional Movement (DM) Indicators have long been established as an effective method of identifying and measuring trends in the stock market. We had discovered that using a range of n values, we could generate much earlier and better signals than we could with any particular value of n. We also discovered that with minor variations in the parameters of our signal-generator we could make these signals happen earlier and earlier, even to the point of being able to predate (or perhaps predict) a trend by two or three days. Unfortunately, while we could create almost any set of signals we wanted in a given period, elsewhere in that dataset, we were generating other signals that were not as good, and many that were downright wrong.
Specifically, it was the closing signals that proved the most difficult to generate accurately. Any time is a good time to get into a trend, getting out before the trend reverses and you lose your profit is problematic. And so we had at last discovered what seemed to be brick wall; a direct, inverse relationship between how early a signal was generated and how reliable it was.
Having already established that any derivative of price is just that, a derivative of price, we can now extrapolate that any signal being generated by any technical indicator (MACD, RSI, or even candlesticks) would have the same inverse relationship of speed versus accuracy. (Note: Quantifying this relationship, boiling it down to some kind of formula or perhaps a percentage should be the subject of future research.)
At this point it was decided that if there was a pattern to DM (as there clearly appeared to be on our charts) perhaps we should employ some sort of automated pattern recognition system to find it. We used an AI system called a digital neural network, more specifically, a Multilayer Perceptron (MLP).
My set of first experiments focused on recreating the success of our Ranged n Signals, only this time, without the errors. To do this I created a dataset from real-world stock data, specifically, the input was the last twenty daily DI values, and the example output was mathematically generated signals from a cross of DI(5) and DI(20). But then the dataset was “cleansed” of signals that turned out to be false in retrospect. The results were encouraging, with an 80% success rate in training, but this dropped to a 65% to 70% when testing.
There were a number of problems with this approach, among them; signals only work in pairs (an open and a close) therefore we can only gauge the quality of the open signal by the timeliness of its corresponding close signal. This had us penalizing many perfectly good open signals because a close signal was late. Identical inputs (two open signals) led to opposite outputs (one was good, the other bad). In short, the real-world dataset, contradicted itself about 15% of the time.
The second set of experiments simply attempted to give the Perceptron the last twenty days of DI values and asked it for an average of the next four. These Perceptrons proved very difficult to train. After several attempts, I began a detailed statistical analysis of the dataset.
I had assembled a total of 1.8 million trading-days. The each stock of the S&P 500 going back to 1985. Even then, the data-points occupied only about 30% of the available dataspace, a vast majority of possible combinations of DM values simply never happened. Over half the days were clustered in middle of the dataspace (with very little DI movement) and almost a quarter were in the exact center (zero DI movement). Real-world data is usually a little lop-sided but not like this.
This is analogous to drawing a series of cards from a deck then asking the computer which card comes next, with high cards indicating upward price movement and low cards indicating downward. It is, of course, the aces and kings (One’s and thirteen’s) that we want to know about. But in order to imitate real-world stock data, in our deck of cards many of the threes, twos, and jacks have been removed and its heavily stacked with fives and sevens. On the order of ninety-to-one.
In this case, the Perceptron starts to answer “six” every time. It may not be right, but it is always the least wrong. (Literally, it reaches a minimum mean-squared-error.) A really good Perceptron may even learn to count the fives and sevens, but kings and aces are a statistical anomaly, to be ignored. They are the exception and not the rule.
Indeed, that is the lesson to be learned from the second set of AI experiments, the Perceptrons I created all constantly guessed that your trend ends now, that tomorrow’s DM is zero. And a detailed examination 1.8 million trading days reveals that that is indeed true a majority of the time.
So, a quick glance at a Ranged DI chart (or any other chart) can easily tell you if a given stock is in a trend or not, that is, if you have Momentum. But even my most advanced pattern-recognition systems cannot tell you when your trend is going to end.
Martin Pring says “a trend continues until it ends.” (Thanks for that, Martin) Issac Newton was a little more specific: He said that an object in motion remains in motion until acted upon by another force.
So I began to look at
market as being influenced by two basic types of forces. Trending Forces,
“momentum,” those forces that tend to “keep a trend going” (Perhaps the singular
Trending Force is more accurate.) and Non-Trending Forces which encompass
everything else. (Note that these are not Counter-Trending Forces,
because they do not necessarily oppose the trend at any given moment, which we
will discuss later.)
Market Psychology: “Surely the price of Company X can’t go any higher than it now, because of Y” or conversely “Surely the price of Company X can’t go any lower, because of Z” or perhaps “Everyone else is buying/selling Company X, so I’ll do the same/the exact opposite.”
News: “Company X has a new product that will change the world” or conversely “Company X is being sued over outrageous claims recently made about their new product.”
Intelligent Design: In its most innocent, the “Strong Hands Theory” at its most malevolent, outright market manipulation. (It happens.)
The “Reflexive” Properties of the Market: The idea that it doesn’t always “act” as much as it “reacts” (and/or overreacts).
So the question we’d like to answer is how do these Trending and Non-Trending forces interact to alter the price of a given security. And to this let’s, for a moment, imagine a universe in which there are no Non-Trending Forces.
In such a universe, there are no thinking participants, but rather computers trade stocks based only on the input of yesterday’s, high, low and close and the assumption that yesterday’s trend continues today.
As such, if the price of Company X increased .7% yesterday, the machines assume that it will do so today, and are willing to buy and/or sell the stock at slightly higher prices. In doing so, the prophecy is self-fulfilled, and the price rises .7% today, and does the same tomorrow. As the simulation continues we see that the stock price trajectories have become linear (the literal definition of “going ballistic”) and trends continue without the slightest variation forever.
It is important to note that, if momentum was the only force applied to a stock (or any single force is applied to any single object) that object’s trajectory would be linear, perfectly straight. And stock price movement is anything but perfectly straight.
With this mental game, we can see that the day-to-day fluctuations of a stock’s price, even minor ones that are well within a “trend” are in fact the result of Non-Trending Forces. It is the Non-Trending Forces that start up the trend, it is Non-Trending Forces keep the trend going and, of course, it is the Non-Trending Forces that end that trend.
So now, when we look at a Ranged n DM chart, we start to see new things. We can see trends ending all the time, without warning. But we can see Non-Trending Forces that often start up a new trend in the same direction. We can also how see how longer values of n are fooled into displaying that situation as one continuous trend.
Momentum seems to be the weakest of all market forces. It seems to be what is left when all other forces are nominal. Even a breath of a Non-Trending Force can easily overwhelm a trend; a whispered comment, a headline, a financial disclosure, or a fifty-two week high can slam a trend closed faster than you can say “sell.”
Directional Movement Indicators are an excellent way to measure a trend and are a pretty good barometer of Trending Force, however weak it may be. And the opening (Buy or Sell Short) signals generated from them can very good. (It’s almost always a good time to get into a trend.)
But I don’t think it’s possible for DM to ever provide a well-timed closing signal. Even in Martin Pring’s own book, the illustrations of DM signals clearly show almost all the profit from the trade evaporating before you get a close signal. This is because your trend is ended by invisible, unpredictable, and unmeasurable (?*) Non-Trending Forces, and you only find out n days later.
The things we see that look like closing signals on our charts are either a trend that has already reversed, in which case we are too late, or it is merely a weakening of the trend, and thus, a higher probability that Non-Trending Forces will overwhelm that trend. However, there is a fifty-fifty chance that those Non-Trending Forces will be in the direction of your last trend, in which case you have just closed your position far too early. And despite my work on the subject there seems to be no hint within DM Indicators as to which is which. They are mathematically identical conditions, which is why the Perceptrons only performed as well as they did.
*Measuring the immeasurable:
Could we measure Non-Trending Force?
Perhaps we can, in a way. It seems to me that there might be a way to calculate how susceptible a stock is to Non-Trending Force, by measuring how that given stock doesn’t trend.
That it to say, how this stock price varies from its own trends, or a volatility of the DM. Some method of adding up a score of “trendablity.” A score that would be very high for an Apple or a KMart and very low for a Microsoft.