The Trade-Robots State of Affairs 2015-04
End of April 2014 – and we are seriously behind in our plans. Time for another update. Generally, we are in full swing with our new trading infrastructure and reworking some other parts of the trading framework, following some issues (in performance and flexibility) that we found. We ended some major consulting work (that had serious impacts on our available time) and right now code changes happen with frightening pace. Which is good – it moves us forward.
The new trading agent: on hold, planned for end of Mai
We just happened to not find the time for this, putting our trading on Oanda on hold. May, latest June, will see this happening.
What we did instead: the Reflexo Symbology
We reworked out internal symbol and instrument handling to follow the new standard symbology we have developed. Putting Forex into the Reflexo framework was (is) a little harder than we thought and required significant reprogramming to fix design decisions. It if funny how sometimes a shortcut comes back to haunt you - in this case the fact that the original code always had a front month mechanism, even for instruments without...
What we did instead: Data fixes and… a new Simulator
Yes, that is likely version 6, but the new simulator is more pessimistic, more accurate, half as large (in lines of code) and twice as fast. This, and some changes on handling of our data files… resulted in our standard benchmark (simulating one week trading with a specific strategy in a specific symbol) to go from 50 seconds to 29. Not bad. We also found and fixed some issues in our historical data, totally rewriting the data extraction from Nanex. A lot of code moved from C++ to C# - and we now have totally accurate data during the week. In depth testing showed some issues with Nanex data in the way we handled it (daily export) having a non-complete order book at the start of data tape. All these issues are now fixed.
What we do instead: Code refactoring for strategies
Reflexo has two parts – the core framework and the extensions in the strategies. Over time, a lot of functionality that should be in the core was added to the external extensions. There is a NetTectureStrategy class (NetTecture – that is us, as in the company behind trading robots) that does things like a trading calendar (which should be in the core framework). The reasons? Limited time on the person doing the framework development. The result – ugly redundant code.
This is changing. Now. A major code refactoring is in the works. Code is cleaned up, functionality is redistributed. Code is moving from our own extensions to the core framework whenever it makes sense. Central code in the Reflexo framework is reorganized to remove redundancies that have crept in. While we touch and touched a lot of areas, most work is done around market definitions – things like trading times in a centralized calendar for example.
What we do instead: A new analysis framework is coming
This is one of the other elements in Reflexo framework now due for an overhaul. The old framework lacked certain flexibility and was too monolithic for my liking. It was a great prototype – but it showed problems down the road, resulting in a lot of performance issues and lot of non-reusable work. It was a great prototype – I am proud of the work we have done with it and the work our partner did when he designed it. But like many prototypes it shows issues after some time and it it simply time to take the lesson learned and apply it.
Right now a new framework is in the making – which should be finished next week, with standard indicators coming a week later. This one is focused on being modular, and thus ultimately faster. It also allows indicators to be tied to other indicators, something the other framework had issues on. You could not easily make a SMA based on an EMA, for example (or calculate the standard deviation of a moving average). The new framework allows this. I think I nailed down most of the issues – with a quite non trivial amount of work with .NET generics – to allow an indicator to take data from any series (ticks, bars, another indicator) while following object oriented design principles. A detailed blog post will follow – particularly also outlining the reasons we do not use something like QuantLib, a well-established analysis library (and the main reasons is: performance during optimizations and graphing abilities…).
Next on the list: Backtest Analysis
Once the new analysis framework is in place, it is time to redo every single of the more than 120 strategies we have, optimizing their code. And then making new optimizations. And then doing better analysis for them. As we now move to trade a lot of them live – we need to provide the capabilities to make back tests that “move forward” in time. Making a new optimization for the last week every week and adding this data to the database. Walk forward optimization requires it – we can hardly efficiently make a multi month optimization every week. Wasted money.
This requires doing a lot more work on the database and web front end. A week or two of work should start fixing this nicely. This will go hand in hand with a major rework of our Reflexo Trading Portal in general – we definitely have to add the capability to see a standardized graph of the prices around a trade in the UI (so one can see how good an execution was etc.). All this work is not exactly complicated – but it is quite a lot and will manipulate a lot of data ;)
Next on the List: Trading Agent
Then, finally, around end of Mai, it is time to move back to the trading agent and get it working. Oanda first, planned for beginning of June. The account is funded. The API is open. We just need to use it. And that requires a lot more work to be done. Accounting for the trading and the still unsolved problem of how to actually manage hundreds of strategies. A separate blog post (or more) are needed here. This is a really critical point – which no software I have seen so far has tacked in a decent way.
Next on the List: .NET 4.6 – a major upgrade
And then, looking for a July implementation, is the upgrade to .NET 4.6 – including performance tests, upgrading our projects to Visual Studio 2015. July is the release date for Windows 10, which we look forward to quite a lot. ASP.NET 4.6/5.0 will do nicely for our portal work – as soon as certain third party extensions are available that we rely upon. The new JIT (just in time compiler) should give us a good performance boost - .NET 64 bit was so far a performance stepchild. A lot of the new .NET features will require going through all of our code and reworking it. It is not a lot – the core trading framework is quite compact. Still, a week or two of work and all this should be done. And then there is the move to the new source control environment. Personally I look forward a lot to the new TFS version.
And then – A major site update and talking about trading results
Once all this is done – it is time to get serious. Trading has to resume as soon as possible – right now we are totally offline in an awkward combination of decisions and reworks. Even backtesting has stopped, waiting for the new analysis framework to be finished and our strategies being recoded.
It is time to fill this site up with some real content. A UI upgrade will also come (mostly invisible, just the usual maintenance done, with new framework versions being used, the CMS getting a minor version upgrade). But it is time, then to spend some time and start putting “real content” on the site. Not just a blog, but also some services people can use daily. Strategy information. Our plan is to open our large pool of strategies for outside trading at some point in 2015 – and that will need more information.
So far the site is stable at around 530 visitors a month without posts published for months. Time to start moving that forward.
My thanks to all that contacted us.
I had some good discussions, some good contacts and met some really nice people. Best of all was the contact with Empirica Empirica, a polish software company doing a platform for automated trading that has so a totally different focus than us that we could really complement each other. We spent the better part of 2 days discussing our approaches and I think it was a great time. A blog post will follow.