Renko and Backtests - it can be a very wrong result

Renko based strategies can be tricky to backtest, especially when you use a bad exchange simulator such as NinjaTrader. It is not a programming error – it is an optimization in their backtest design that makes Renko very hard to backtest. If one does not know how the simulator works, then a Renko based strategy will give always wonderful results, then fail in the real world.

The main issue why this happens is optimization. NinjaTrader, for example, has decided that a backtest is not using tick data but bard. 1 Minute. 10 seconds. Renko. Bars. The simulator never sees bid, ask or ticks. Only bars. The advantages are clear – you test with a lot less data. This is important, more so when you have bad programmers and / or a bad architecture that does not allow you to use multiple computers for a backtest. This is one side of the problem with backtesting Renko based strategies, and it has many takers. TradeStation? Guilty. It is actually a valid optimization – the problem is that many developers are not really knowledgeable in the internals of the software they write.

The Renko problem is symptomatic why we decided to always run tick based backtests. In fact, our backtest engine does not even know what a bar is – it reacts on the event stream from a market data playback, only, and all higher functions are done in the strategy. Need a Renko bar in the strategy? The strategy can get one, the simulator will not care.

What is the problem?

Price Jumps. A price jump and a bar based backtesting engine will give you executions that could never ever have happened. In case of NinjaTrader you can add some nice additional bugs, but the core problem is an incompatibility. A Renko bar based strategy and a Renko bar based backtesting engine will always give you good and unrealistic executions.

Again, this in the core is not a bug. The bar based backtest was done to be faster, and this is a valid optimization – not everyone is willing to throw many computers on a problem. But it is an abstraction and in the case of Renko bars, this abstraction gets ugly.

Imagine: A commodity trading at 100, with a tick size of 1. A Renko bar with a box size of 5. And a jump. The price trades at 100, then the next trade happens at 119. Good? This happens, sometimes. More if you choose a very small Renko bar size of for example 1 or 2 ticks.

The problem is: the Renko bar generator will generate the following bars: 100-105, 105-110, 110-115 and 115-119 (still open as the price did not hit 120). 4 bars. The jump disappears.

The strategy now may see the move in 3 bars (the 4th is still open). And it can decide once the price starts moving to buy at the market. It does so because the first bar closes higher than – let’s say 103 (this is an example after all).

So, at the end of the first bar the simulator executes a buy at the market order at 105. The simulator gives this price. Then the next price executes, 105 to 110. But these are empty trades. 0 volume in the bar. One can argue this is a bug and a 0 volume bar would not allow an execution. But the simulator has no idea where the price will go (it does not see the ticks) and not returning the fill at this bar will result In a VERY different behaviour between simulator and real world, in terms of code timing. A big problem.

And this is the exact problem of simulating Renko based strategies with a tool like NinjaTrader, but in general with any bar based simulator. Which is why our internal Reflexo framework only works on ticks in the simulator.

As a note to all NinjaTrader users – one can work around it. First, there is a “Better Renko” that forms the boxes with open/close but leaves high and low realistic, so one can wee when the price jumps. Second, one can always trade against a 1 tick bar (not analysing it, just trading it) or two (bid/ask separate). This means that the strategy, executing against a 1 tick bar, would get a more realistic execution. It uses a lot more memory though, is a lot slower and takes more coding. But it is necessary in order to backtest Renko based strategies realistically in NinjaTrader. It also exposes another bug – but that is another story for another blog post.

How do you backtest? Is your platform showing the same behaviour? Or are you like us, backtesting always against a full tick stream to avoid issues like this?