Automated Forex Back-Testing
Automated back-testing is the most well-known method for testing a trading system. Most forex traders are aware of the fact that it is possible to do some automated testing on a trading system. However, most forex traders use discretionary, or manual, trading systems, so automated back-testing is not the ideal method for back-testing; it does not closely duplicate the discretionary trading most traders engage in. There are many reasons for discouraging automated back-testing for discretionary traders:
- There may be too much human interpretation in the trading system. Automated back-testing does not allow for the human interpretation of trade signals.
- The trading system may involve variables that are not available on the price chart (news releases, economic data records, interpretation of world events, etc.).
- It is impossible to automate the trading system (fuzzy logic, parameters difficult to define, etc.).
- You may not be able to articulate your trading system. Automated back-testing is only possible for trading systems with clearly defined rules.
Most forex traders should not use automated back-testing. Automated back-testing is appropriate for those traders who use automated trading systems. Automated trading systems, known as trading robots, or expert advisors, are popular with forex traders. However, most traders are more comfortable with discretionary trading systems. So, while it may be possible to use automated back-testing, it may not be appropriate for most traders.
Here is a test to help you decide whether automated back-testing is for you. If you turn on your automated trading system and can allow it to trade without any of your intervention for a month, then automated backtesting may be for you. If your system needs your input for any reason, then you are a discretionary trader, and you should use manual back-testing or manual back-testing with software. There are other disadvantages to automated back-testing. It does not allow you to gain experience with the trading system. You will not gain the expertise that you would if you manually tested the system because the computer takes all of the trades over the course of the back-testing.
Automated back-testing will not give you experience trading your system through many market conditions. Automated back-testing will probably not highlight the weaknesses of your trading system; these weaknesses will be readily apparent when manual back-testing is done. However, with automated back-testing, the weaknesses are a bit more difficult to identify. In short, automated back-testing is really only an option for those traders who are not using a discretionary trading system. There are unique pitfalls associated with automated back-testing. For example, it is quite easy to use too many variables in an automated trading system. Using too many variables often implies that there are too many indicators in the trading system. Seasoned traders understand how simple trading systems are robust, and may be applied to many markets over varying timeframes. (All of the naked-trading systems in this book are incredibly simple and robust.) It is difficult for some automated-system developers to keep their trading systems simple and robust; the temptation to add more indicators and rules is great.
It is exceptionally easy for automated traders to use too many indicators when developing and testing a system. Adding too many variables to any trading system increases the chances that the trading system will work quite well on one data set but will not work well on another.With too many variables the system is likely to do exceptionally well during some market conditions and then break down and perform poorly when market conditions change. This is a very real risk with automated back-testing. It is almost too easy for a trader to decide to add more conditions and indicators to the trading system, which will increase the profitability of the trading system over the course of the historical data in the back-test, often making the system look very good. However, these results often completely fall apart after you apply this very same trading system to a different data set or to future market conditions. In a very real sense, the Achilles heel of automated back-testing is that it is too easy. Because the automated back-tests can be generated so quickly, the automated back-tester will go overboard tweaking and testing the system. The end result is often a system that works exceptionally well on the historical data in the back-test but falls apart completely in live market conditions.
There is another issue that often comes up with automated backtesting. This is the postdictive error, a fancy way of saying that a trading system uses future information to make a decision in the present time. This is actually something that is related to the hindsight bias. Traders backtesting automated trading systems must be extremely careful. It is possible for the back-tester to pull data from the future without knowing about it. This is obviously a very big problem, because as a rule, future data is not generally available, despite any claims to the contrary by technical indicator salespeople (psychics and palm readers are the obvious exceptions). When future data is used in an automated back-test, the system looks incredible, but once the trading system is applied to real market conditions, without the critical (future) data, the system falls apart.
There is one advantage to automated back-testing, that is, it allows you to quickly determine whether an automated trading strategy is viable. The back-test may be done in seconds, which is a true advantage for the automated trader. Remember, the automated trader will not gain the same expertise and comfort level as the trader who uses manual back-testing. This is the biggest weakness of automated back-testing, the back-tests are cheap. There is no experience gained each time a back-test is run. The manual back-testing trader accumulates experience every time a trade setup is executed. This experience should not be discounted. An automated back-test may clarify the worthiness of an automated trading system, often involving hundreds, or thousands of trades, but remember the trades are all taken by computer and will not lead to trader expertise. In this sense the trades are wasted, the trader does not accumulate experience during the back-testing, this is only possible with manual back-testing. If you use automated trading systems, then, perhaps, automated backtesting is appropriate, but if you use manual trading systems, it may be best to avoid automated back-testing.