By Vlada Raicevic*
I thought I would write down a few thoughts on a topic that piques the curiosity of many traders. I will also provide some basic back tests on the ES to give an example of what kind of results one can expect.
If you have been involved with technical trading for any length of time, you have no doubt heard about the concept of Program Trading. The idea is simple, but the execution can be anywhere from puzzling to mind-numbing in complexity. Program Trading can be summed up thusly: Software does the trading for you. This is software that you write yourself, hire somebody to write for you or that you buy from an individual or corporation. The basis, or the triggers used to enter and exit trades, can be as simple as price moving above or below a moving average, which can take no more than 50 - 100 lines of code or as incredibly complex as systems that are written by mathematicians using artificial intelligence systems that require tens of thousands of lines of programming.
The latter of the two includes systems that cost millions of dollars to code and require a fairly large staff of systems engineers to maintain. It is difficult to get much information from trading houses, hedge funds and others who use such systems, and, while they will say very little about how much of their trading and profits (or losses) are due to program trading, we can only assume that they generate enough profits to be worth the expense of maintaining and improving these large systems.
While thinking about this article, I came up with several titles. For example, "Basics of Back Testing" and "What Is Program Trading?", but the idea that burns in the hearts of all traders is really this: “Money for Nothing,” or, in other words, coming up with a simple system of trading that you can program and set loose to generate profits for nothing more than the cost of commissions. After the initial investment of setting this into motion, all you have to do is sit in your hot tub and wait for the profits to roll in. It is the trader’s Holy Grail.
How I Started Back Testing
Last year, I got the program trading glint in my eye when I signed up with eSignal and found that their new version allowed programming of simple-to-complex systems that can be back tested. What is back testing? It is simply this: You program an idea, for example, the idea of going long when price closes above the 50ema, and, then, you test it using data from the past. Reports are generated, giving you a variety of statistics, and the ever-important net amount from the trades.
The first thing I did was back test the concept of CCI crossing the centerline for long and short trades. The results I experienced with these tests were AMAZING! I was going to be RICH! I started daydreaming about long vacations in the south of Spain and a new motorcycle, and, and.....and I found a slight programming error in my code. I ran the tests again and received results that were just plain pitiful. Dang!
Looks as though my next trip to Spain was going to be like my last one: Strap on a backpack and go to cheap 2-star hotels. Oh, well! You still meet more interesting people in places like that. But, darn it all, I really wanted that new Italian Moto Guzzi motorcycle I've been drooling over, so I persevered.
Over the past year, I've learned quite a bit about back testing and have had some strong preconceptions shattered. For example, I always thought that a trailing stop would give much better results than just a SAR (Stop and Reverse) system, but months of testing showed me that my expectations were wrong. Even though I used a hundred different variations on the trailing stop, it never gave better numbers than a simple SAR. It surprised me greatly, but the numbers didn't lie.
Let's say you have a simple idea: Go long when the MACD (12, 26, 8) fast line crosses over the slow line to the upside, and go short when it crosses to the downside.
1. You code a simple program and run it against a daily chart, a 120-minute chart, 60-minute chart and so on, all the way to a 5-minute chart. Then, you make a small table of the results containing information, such as number of trades, percent of positive versus negative trades, net totals and so on.
2. You then change the signal smoothing from 8 to 6, and, using a MACD (12, 26, 6), you rerun the tests. If the results are worse, you turn around and try increasing the signal smoothing and rerun the tests using a MACD (12, 26, 10). If the results improve, you continue adding to the signal smoothing until the results stop improving. Slowly, you modify this number, and, perhaps, the slow and fast length of the MACD until you get a setting and a time frame that give you the best results.
Even though this sounds incredibly tedious, and, sometimes, it can be, it is more often than not an interesting puzzle. You have a finite amount of numbers to use, and you just need to find the right combination of these numbers to maximize your return. You also have the concept of Money for Nothing propelling you along. These tests can take anywhere from 10 seconds to perform on a daily chart, to several minutes when running the same test on a 5-minute chart. This is because a daily chart may generate 60 trades for the test while a 5-minute chart may generate 1,200 trades for the same period.
I keep a sketch book and doodle while I run the tests. I also keep some weights in my trading room and do some lifting, sit-ups and so on between tests. After trading all day, the last thing I want to do is sit immobile all night running back tests.
When the first set of tests is done, you may find that something like a MACD (11, 17, 6) gives the best results for trading IBM on a 15-minute chart. Excellent! You may now decide to sit in front of the monitor all day, everyday, and trade IBM by using this as one of your key indicators. Then, you notice something interesting: The crossing of the MACD lines over the centerline is also a good signal for entering a trade.
3. You modify your code a bit, so it takes 100 shares of IBM long when the MACD lines cross to the upside and then adds 100 shares long when those lines cross the centerline. Run a test and see what kind of results you get.
4. More ideas: How about testing just the crossing of the centerline? Are the results better? What about moving the stop to break even once price moves in your direction by some set amount.
These types of tests are limited only by your imagination. "But, wait," you say, "It is also limited by my inability to program. I'm no code jockey. Please don't make me go back to school!" Indeed, this can be a bit of an overwhelming issue, but eSignal and other trading software that allows program trading and back testing, have hundreds of programs already written, templates that contain most of the code and even automated software that asks you a series of questions and then creates the code for you.
Keep in mind, though, that back testing is a slippery fish. It continues to surprise me with results that are completely unexpected. I may get terrific results testing an idea on IBM but horrible results when testing it on MSFT. Whatever idea you have, it will work much better with certain types of stocks than it will with others. My focus has been to try to develop a trading system to trade the ES, but, for months, I was frustrated with better results when running tests against KLAC, one of my testing stocks.
Here are some things I noticed in testing:
1. Never assume something. These tests are meant to stop you from making assumptions. When testing, don't skip steps; the one setting you didn't try could be the one that gave the best results.
2. Run your tests on a number of stocks, and mix them up so that you have some highly volatile stocks and some low-volatility stocks. Your specific idea may work horribly on a high-volatility issue such as ES but extremely well on a stodgy old boat such as MO.
3. Consider programming your ideas, so you close out all positions at the end of the day. I found that most large losses occurred due to gaps, sometimes as much as 40% of total losses for the entire test.
4. Keep in mind that you are testing a strictly literal idea. Meaning that, unlike when you sit there trading, the machine does not know that it shouldn't go long because strong resistance is just above. All it knows is that the automated signal is given.
5. Mix your ideas. Two slightly good ideas may become a great trading tool when used together.
When testing some idea, I recommend trying the following cases:
- Move a stop to break even once your trade has moved into the money.
- Test with trailing stops. Each time price closes an amount “x” in your direction, move the stop by that much. Some trading systems work well this way.
- Test the idea of hard profits. Each time your trade goes into the money by “n” amount, just take the profit.
That last one has given me some of my best back testing results yet. What happens is this: You come up with an idea that often gives you a great entry and often gives you a certain profit, but the market takes most of it away in whipsaws.
So, for example, you test with more contracts but always just take the profit as soon as you get 2 points on ES. You end up missing the big ones, but all those little ones add up into a bigger pile. Or, you take 3 contracts at that 2-point profit and let the last one run. The number of permutations are endless and only limited by how much doodling or sit-ups you can handle during back testing before going mad.
Back Testing Examples
I wrote a little program to test a basic premise that many have heard of -- trading when moving averages cross.
Example 1:
Basic Premise: Go long when the 13ema crosses over the 21ema moving average; go short when the 13ema crosses below the 21ema.
Test Criteria: No stops, simple SAR (stop and reverse) trading, 60 days of data, raw results, no included commission costs
Test Object: ES, using 1 contract; trade taken at close of period when signal is generated; net is number of total points from trading
120-Minute
· Trades: 7
· Net: -27.50
· Percent Profitable: 14.28
60-Minute
· Trades: 12
· Net: +116.75
· Percent Profitable: 50
30-Minute
· Trades: 39
· Net: +106
· Percent Profitable: 38.46
Looking at the results, you see that the 120-minute example is behaving very poorly. So, we take a look at the charts to see what happened. Red shows the duration of the short trades, and green shows the duration of the long trades.
So, our results are misleading because the 120-minute chart did not catch the long selloff in January as the 60-minute did. This is due to the eSignal limitation of 60 days of intraday data, and, hence, the 120-minute chart did not get the crossover trigger for that short. eSignal will soon be giving users up to a year's worth of intraday data, and I look forward to rerunning a number of my back testing code using a much larger pool of test data.
Example 2:
Basic Premise: Go long when the 9ema crosses over the 17ema moving average; go short when the 9ema crosses below the 17ema.
Test Criteria: No stops, simple SAR (stop and reverse) trading, 60 days of data, raw results, no included commission costs 
Test Object: ES, using 1 contract; trade taken at close of period when signal is generated; net is number of total points from trading
120-Minute
· Trades: 7
· Net: +17.75
· Percent Profitable: 42.85
60-Minute
· Trades: 16
· Net: +149.50
· Percent Profitable: 50
30-Minute
· Trades: 30
· Net: +96.25
· Percent Profitable: 50
You look at those 60-minute chart results and think, “Zowee!” Remember: This is only 60 days’ worth of data. It looks promising, but I would like to see a minimum of a year's worth of data before becoming too excited. Still, it DOES tickle the greed bone in all of us, eh?
Next is a chart and the actual trades themselves for that back test. Note that the extremely good result is due to the large initial profit from the January 2003 drop. The 30-minute test had twice the trades but a lower profit, indicating that, once more trades were placed into the statistical pool, profits might start to flatten.
From this point, you can continue to back test using various moving averages and types (mix exponential with simple or weighted moving averages) and test using stops and so on.
You can use back testing to find out how well your signals actually work, or, over time, to help you create that Holy Grail: An automated trading system that, over a period of time, generates money. Let me say that last part again, "OVER A PERIOD OF TIME GENERATES MONEY."
Every trading system works well in one kind of environment and works poorly in another. Over a period of time, say 6 months, a simple system may generate $3,000 for you, but, during the first 4 weeks, you may get a drawdown of -$1,500 due to market conditions. If you code a system and then let your program loose to start automatically trading, you have to have the conviction that, over time, it will make money for you, even as you watch in horror when the losses mount those first 4 weeks. To have this conviction in your system, you must run numerous back tests that show you that, over time, you will make money.
Probably!
Even back testing is not a guarantee for future results because the market conditions for the past 3 years may have been completely different from what you are going to encounter in the next 3 years. However, back testing is the only way to go about this process, and, as long as you understand that there are no guarantees, you can at least be somewhat comfortable that you have some hard numbers behind your trading ideas.
*Article reprinted (and modified) with permission from OptionInvestor.com
