9 Steps to program profitable trading strategies
Programming a profitable trading strategy is not trivial. It is also not particularly hard – at least the programming part. We develop hundreds of setups that have a statistically proven significance at least in a specific timeframe. There are a number of steps we follow when we develop a profitable trading strategy.
Pretense: You must know what you are doing
New to programming? Never traded before? All that will be in your way. While trading strategies are inherently not complex in terms of programming, one still must program and know how to deal with the specialties of markets. Our blog has many examples of functionality that it trivial for a human and not trivial to program – like maintaining a calendar of trading times so a strategy does not trade overt the market close on shorter trading days.
There is quite a lot of such items that a new developer will have to master to make a reliable trading strategy – and all these are out of scope for this blog post – a different blog post will look at those. We assume that a developer has mastered how to write a trading strategy and not looks for a successful one. If you are not there – expect some pain in the road ahead. Things are not as easy as they look, and tools like NinjaTrader are, in part, a trap – they hide the complexity, only for it to come back at a time and show some really ugly behavior.
Step 1: Come up with a trading idea
This is not development. This is earlier. A trading strategy should be based on a concept. You look at something and think “hey, this could make money”. Then you sit down and paper-trade it for some time. NinjaTrader, with all the ridiculous issues they have for automated trading strategies, is a great tool for this because the limited programmability also allows one to customize indicators to give alarms. If the work of strategy concept development and programming is split between people – like in our little operation – then this step is done by the trader side, not the developer.
An idea should always be sound and have a concept behind it – not something born in code, but something born out of… well, an idea. “Have indicator, look for profit” is not a good idea to start with. Observing a behavior or having a concept how to use an indicator in certain situations is. For example:
“I like the Fisher Transform - nice indicator. Maybe there is a way to use it to enter trades in trend reversals at one of the trend extremes”. That is the core of a trading idea. This is actually how one of our strategies did start.
Step 2: Define the trading rules for your new strategy
From an idea, move to having defined rules for an initial version of the strategy. There already may be some tuning here – parameters, but mostly also trading rules. They must be written down in a way that you can hand them over to a separate person for implementing. This can be quite short, or it can take a week or two, while the strategy is still paper traded.
Step 3: Program the initial version of the strategy
The next step is to program an initial version of the strategy. This can take some time – there are a great many things in trading that computers do better than humans. There are also a lot of things that humans do easy but that are a lot of work to program into a computer – chart pattern recognition is a very good example of this.
Step 4: Validate it trades as required
The next step is tedious, and a step that is very often omitted. It is, though, extremely critical. In order to make sure the strategy is traded as intended, every single trade over a larger time period must be validated. In fact, every symbol must be validated because the strategy may well miss trades due to a coding error. This includes as a shortcut a lot of statistical analysis for fast validation, for example: Are there trades on every day of the week? Are there trades long and short? Do trades follow risk (allowed loss)? Are trades kept open over the weekend or over sessions? All these questions point to significant programming problems.
For strategy analysis, we generally are not interested at all for the PNL unless we have validated that the trades do make sense and follow the strategy. In quite some cases we found significant outlier trades (big wins or losses) that were purely the result of programming errors and totally skewed the result
Step 5: Optimize and look at the result
The next step is then to optimize the strategy. This is a part of walk forward analysis, brute force parameter optimization but also a more detailed definition of trading logic. There is a lot of statistical analysis going on at this stage.
Step 6: Check defined special time intervals for behavior in specific market conditions
We have a list of predetermined time intervals that are then manually checked for trades. These are either “signature” trading periods (days the stock market went only in one direction) or dates of abnormal market behavior. Regardless of the financial outcome, we want to know that the strategy trades made sense in these dates.
Step 7: Refine trading logic as needed
Finally. The second development iteration starts. Yes, the strategy is nice and dandy, but maybe the trading logic can be optimized. Not the parameters. Maybe it is time to add a timeout to trades and close loosing trades after some time? Maybe it is better to limit trading hours and only trade during a specific time of the day. This is a lot of iterations – and all starting with a repeat of step 4 or 5. This takes time – both, human (for development) as well as technical (optimizing capacity in our data center). There are not too many optimization here, but a lot of back tests as various approaches to the trading rules are tried out and analyzed in the various market conditions. Obviously – this is the moment one can make big mistakes and fall into the trap of curve fitting.
Step 8: Implement walk forward analysis
The next step is now to move towards a formal walk forward analysis. The strategy has proven to be worth the effort. In our case this means it gets a simulation identifier assigned, so that we can store the back testing and optimization results long term. Our analysis is very database driven, meaning that we generate literals hundreds of gigabytes of data per day. Until now, the strategy back testing results were not considered worth keeping – now we move the simulation to a more sophisticated and long term part of our framework. As Pardo points out in his book about strategy development found out – Walk Forward Analysis is processor intensive. In our case – we need to rerun it weekly. But – we also have the infrastructure in place to update the data with just the next week of simulation results.
Step 9: Paper-Trade the strategy in real time
The final step now is to move the strategy forward into what we call staging. In this level a strategy is trading against a simulator in real time for some timeframe until we are comfortable all back tested results are validated. In case of Forex this may (fast) result in the strategy trading a very small account real money (Oanda allows trading down to 1 USD / 1 EUR), so the financial risk of a real money test is extremely low. We will also run back tests for these periods and compare the trading results, validating that the expectations of the back test are realistic. If, for example, we get significant different executions than in the back test, this will be investigated.
And then – it is time to put real money on the table
Once 2-3 months have passed, the strategy is then ready to get real money allocated. For us, this does not mean it will be traded, but it means that we will now try to integrate the strategy into a strategy portfolio so that it can get money – because we always trade groups of strategies. Depending on how good the strategy looks, and how well it fits with its characterization, it may now move to real money relatively fast.
Where is the secret to get hundreds of good setups?
Simple. Hard work. 80 strategies in 4 markets are 320 good setups, with all parameters. They also likely are another 150 strategies that get eliminated early. At the end, it is about having a lot of diverse ideas (which can be simple – and the internet is full of them) and then sitting down and doing these steps. One after the next. A good infrastructure helps – we manage terabyte of back testing results (our current capacity is 3000 GB) and have no problem keeping nearly 200 cores permanently busy running back tests. Something we could not do on most retail software. But it also takes a focused approach – one just has to run one setup after the next through the whole sequence of steps. Hard work – both by humans and a computer – are needed to come up with working solutions.