Compare to the market over the same stretch

Comparing to the market only means something over the same stretch of time the strategy ran. Otherwise you can get any number you like.

Where the catch is

Once a rule has a result, you want to compare it to something — usually to buying and holding. And that's where it's easy to plug in the wrong number.

The most common case: take a well-known figure for how much the market returns over a long period and hold it up against a rule that ran for a short one. Different periods, different market moods, and the comparison means nothing.

Worse is when the comparison number comes from memory. It's almost always optimistic: good years are the ones you remember.

How to do it

Count both sides over the very same stretch, from the same date to the same date. If the rule ran for a year, count the market over that same year — even if the market fell that year and the comparison comes out flattering.

Second: compare more than the money. Compare the depth of the holes in the account too. A strategy that landed in the same place with half the drawdown is a different result, not the same one.

Third: if you're comparing against a simple benchmark like "buy when price is above its average of the last so-many days", check the benchmark itself. Those markers sound solid and often turn out worse than just buying.

What usually comes out

That the market over your chosen stretch behaves nothing like the one everybody remembers. The stretch you tested on can sit entirely inside a decline, and then any rule with a plus in front of it looks like a miracle — though no miracle happened.

And that popular benchmarks don't survive the check: count them over the same stretch and they turn out to be the worst of the options, not the standard.

What this does not tell you

A correct comparison doesn't make a strategy worth running. You can beat a falling market by doing nothing at all.