Pardo, Robert: Evaluation and Optimization of Trading Strategies
There are unlimited books about trading, and most are not worth the paper they are written on. There are, though, very few books that talk about how to approach trading system development from a systematic angle, about the pitfalls to avoid. And very few of those books are written by proven survivors in this field.
This book was written to provide a clear-cut and speciﬁc road map for
the trader who wants to transform a trading idea into a tested,
veriﬁed, properly capitalized, and proﬁtable automated trading
Pardo’s book “The Evaluation and Optimization of Trading Strategies“ is one of it. And it is a required central reading in our operation. Not only that, but we are building and rebuilding our infrastructure around his concepts – because they are sound. This book is a core of our library now. It should be in your library, too.
Do you do Walk Forward Analysis?
Robert Pardo is a big fan of it to the degree many say he invented it. A claim I cannot sensibly judge. But he makes good points why this is a good approach – a well as pointing out most software does a woefully bad job of supporting this. Walk Forward Analysis means that you do not optimize all – you optimize in slices, then trade for some time, then optimize again. You optimized for example 12 months, trade for 1, optimize the last 12, and trade for 1, and so forth. This means parameters change slowly, but they do change when market patterns change. This is a permanent rolling blind data test. One that needs what is called an “Objective Function”, responsible to select one of the many strategy variants as the best one. Yes, this may be “largest profit” but more complex scenarios make more sense, to avoid a singular highest profit spike. The one best result may be a surrounded by losses – and thus be risky. It is better to look for a stable plateau.
I would like to make one thing perfectly clear from the outset—and
also forewarn anyone who did not like the ﬁrst edition from buying
this edition—I still believe that the method that I created,
pioneered, and still use to this day, Walk-Forward Analysis (WFA) to
be the only 99 percent foolproof method of optimizing a trading
strategy. The only model that I trust that does not use WFA is the
model that requires no optimization.
This is an interesting concept because it is sound – and ignored. Not supported by any commercial tool, and contrary to traders running their own infrastructure (like we do with our Reflexo framework), this is a big limitation. Go figure – seems software providers should read some trading books. The chapters about Walk Forward Analysis are one of the highlights of this book, going from page 237 to 262. Yes, it is not long (and embedded in larger chapters about optimization) but it is quite eye opening.
The bad: he is an IT user talking about modern IT
Like everything with light, there are shadows. In Robert Pardo’s “The Evaluation and Optimization of Trading Strategies” – it is IT. Computers. Robert Pardo is a user. He knows his way around TradeStation (the platform he is using), but he is no IT specialist, and it shows. It is never good to get specific in a book like this – he talks how computers have a problem doing optimizations due to too much work, and I see that Intel releases an 18 core chip this year, dual socket, hyper-threading, so a computer with 2 of them can run 72 parallel threads – we do full optimizations with less computing power (and yes, I think of getting those). He talks about storing optimization and backtest results by exporting them (into text files) and keeping them for later reuse. I see that next week we upgrade our database server to 3tb so we can keep all the results in the database for online analysis. Too specific and obviously a little out of his league.
Yet this vast proliferation of trading strategy development
applications, add-ins, and advanced technological trading applications
is a bit of an illusion. Yes, one can purchase and use all of these
different products if one is so inclined. But try to tie it all
together into a seamless and functional trading application and one
sees wherein the problem lies. It is nearly impossible to do so.
That said, this book is not about specific IT technology, and as such this negative is not really relevant for the content of the book. I have a rule to always write of at least one bad thing, and that was the best I could come up with. Go figure – In Robert Pardo’s “The Evaluation and Optimization of Trading Strategies” is a good book.
The good: He is a survivor and talks about approache
Robert Pardo trades investor funds, and that for many years. He does so automated. This is a significant credential. “The Evaluation and Optimization of Trading Strategies” is a book about his approach to develop and quantify trading strategies, and it shows that he has experience.
Strategy development and testing done incorrectly will lead to real-
time trading losses. Make no mistake about this consequence. As the
famous computer saying goes, “Garbage in, garbage out.” Consequently,
because of the inevitable results that follow error, poor procedure,
and shoddy craftsmanship, computer testing is best done properly or
not done at all.
It is not about in depth statistics (you better know what a standard deviation is before starting to read it). It is not about programming. It is not about markets. It is about not falling into the usual traps of over-optimization. Robert Pardo’s “The Evaluation and Optimization of Trading Strategies” is a book about how to read a backtest result, how to assess how good it is, how to make sure that what works in the historical sense has a chance to survive in the markets.
This book is about a structured approach
Robert Pardo starts at the beginning, at the fundamentals, defining the basics, what an edge is, and takes the reader through a complete development approach. He never goes too deep, but always is deep enough that even people with years of experience can pick up one of the other gem.
Because of ignorance and the difﬁculties of performing optimization
and back-testing correctly, some still believe that testing and
optimization are little more than an exercise in curve-ﬁtting. For
those of you who are unfamiliar with these terms, don’t worry, they
are all formally deﬁned in the appropriate chapters.
The titles of the 14 chapters give a good overview:
- On Trading Strategies
- The Systematic Trading Edge
- The Trading Strategy Development Process
- The Strategy Development Platform
- The Elements of Strategy Design
- The Historical Simulation
- Formulation and Specification
- Preliminary Testing
- Search and Judgment
- Walk-Forward Analysis
- The Evaluation of Performance
- The Many Faces of Overfitting
-Trading the Strategy
A total of 317 pages. Thin enough to be read on a weekend. And, as I can only repeat, Robert Pardo’s “The Evaluation and Optimization of Trading Strategies” is a thorough book.
The Pardo Profit and Loss Projection
In Robert Pardo’s “The Evaluation and Optimization of Trading Strategies” has another interesting gem, which you find starting at page 127. Robert Pardo assumes that the Profit of a backtest is “optimistic” and suggests correcting profit and loss (separately, the loss higher, the profit lower) to come up with a more conservative expectation. This is a very interesting approach and the concepts behind it are valid – as is the case that no software incorporates it. Having your own framework is good – Reflexo will show this calculations in a week or two.
What is it about? Static is unreliable. Robert Pardo assumes that the Profit s one of possible and that for example a small number of losing trades in a backrest means the loss number is unreliable. This is an important thing – and often ignored.
The approach is to take not the profit and loss that is generated in a backtest, but to correct it by one standard error. The standard error is (without all the explanations in the book:
StandardError = StDev(average trade)/SqRt(number of trades)
As this is done to profits and losses separately, and includes the number of trades, a small sample size (i.e. for example a strategy that is having a few loosing trades) will take a larger hit. While this may sound unfair it is a good expectation – because if I have 200 winning but only 15 loosing trades, I am not really sure how representative these loosing trades really are. Which means to rather error on the careful side.
Portfolio optimization, which means using multiple strategies / instruments for a more consistent equity curve by for example combining a trend with a counter trend strategy, is explained in some detail. Capitalization requirements based on strategy parameters, including formulas and their justification.
At the end – a terrific read
Robert Pardo’s “The Evaluation and Optimization of Trading Strategies” is a book well worth the price asked for it. Many strategy developers may be shocked to see how badly they approached the systematic side. How they have put themselves into a hole because of ignorance. Developping a stable strategy is about more than doing some programming and optimization. It needs knowledge that is rare – about a systematic approach – and not a lot of books contain this knowledge. Robert Pardo’s “The Evaluation and Optimization of Trading Strategies” does. I read it in one run. And started planning to incorporate it immediately. Which is rare. My tip: Read it.
Robert Pardo’s “The Evaluation and Optimization of Trading Strategies” is available through all major book retailers. It is a little hard to get (out of print?) and prices go up to 100 USD. You can find a PDF online easily if you insist on breaking the law. Make sure to get the 2nd edition. I hope there is a third one time, or at least a reprint / print on demand.