A backtest tells you about the past. The interesting question is which past 

Tick data, spread modelling, survivorship and the number of times you pressed optimise. Four reasons a strategy tester flatters you, and what to do about each. 

What is the tester actually simulating? 

The tester runs a sequence of prices past your rules and then fills in a long set of assumptions about what would have happened when the two met. The prices are usually real. The assumptions almost never are. 

Every backtest is a model of execution as much as a model of strategy, and the execution half gets far less attention than it deserves. 

MetaTrader 5’s tester is a substantial improvement here, with real tick data support, multi-currency testing and multi-threaded optimisation that MT4 cannot match. Better machinery does not remove the problem. It removes the excuse for some of it. 

Why does tick data quality matter? 

Because a strategy’s entry can depend on what happened inside a bar, and a test using one-minute open, high, low and close data has to guess the order those four prices arrived in. Guess wrongly and a trade that would have been stopped out shows as a winner, or the reverse. The effect is not random. It systematically favours strategies with tight stops and targets, because those are the ones most sensitive to intrabar sequence. So the strategies that look best in a low-quality backtest are disproportionately the ones the backtest is least equipped to evaluate. 

Six standard backtest assumptions. Every one of them errs in the same direction, which is why almost all backtests look better than the live result. 

Why do the errors all point the same way? 

That is the part worth sitting with. If simulation errors were random you would expect roughly half of strategies to underperform their backtest and half to overperform. They do not. The assumptions that are easiest to code are also the ones that remove friction, and removing friction always helps the strategy. 

So the gap between backtest and live is not noise, it is bias, and it has a predictable sign. A sensible working assumption is that live performance will be worse than the backtest, and the only real question is by how much. 

How much optimisation is too much? 

Any amount you do not account for. Each pass through the optimiser is a chance to fit noise, and MT5’s multi-threaded tester makes it trivially easy to run thousands of combinations. Faster optimisation does not produce better strategies. It produces overfitted ones sooner, with more conviction. 

Holding data back is the whole discipline 

The discipline that helps is holding data back. Optimise on one period, test on a period the optimiser never saw, and treat the second result as the real one. If performance collapses out of sample, what you found was a pattern in the sample rather than an edge in the market. This is unglamorous and it is the single most valuable habit in systematic trading. 

What does walk-forward add? 

It repeats that hold-out process rolling forward through time, re-optimising on each window and testing on the next, which approximates what you would actually have done if you had been running the strategy live and periodically retuning it. It is more honest than a single in-sample and out-of-sample split and considerably more work. 

It also usually produces a much less impressive number, which is the point. A strategy that survives walk-forward with a modest edge is a better prospect than one with a spectacular single backtest, and most people prefer the spectacular one. 

Does the broker’s data affect the result? 

Yes, and it is a genuine complication. Backtests run on the broker’s historical feed, and feeds differ. Test the same strategy on two brokers’ data and you get two results, sometimes materially different, before any question of execution arises. There is no canonical retail forex price series to test against. 

It follows that the platform and broker are part of the experiment. The Investors Centre funds live accounts with its own money to test UK trading platforms rather than ranking them from published fee schedules, and its comparison sets the FCA-regulated MT5 options side by side on the costs that actually matter. 

Read that kind of data for what it is: a sample from one broker, one account type and one period, which constrains a backtest’s assumptions rather than supplying them. It narrows the range you are guessing inside. It does not close it. 

What should a credible test report contain? 

Four things that are almost never present together. 

The full trade list rather than a summary, so the distribution can be inspected instead of the average. The out-of-sample period stated explicitly, with the date the optimisation stopped. The cost assumptions written down, including spread model, commission and swap. And the number of parameter combinations tested, which is the single best proxy for overfitting risk. 

That last one is the rarest and the most diagnostic. A strategy selected from twelve candidates is a different proposition from one selected from twelve thousand, even if the final equity curves are identical. Nobody publishes the count, and asking for it tends to end conversations quickly. 

How long does forward testing need to run? 

Long enough to include conditions the backtest could not have contained, which is a question about market regimes rather than about weeks. A trend-following rule that has only ever run in a trending market has not been tested, whatever the elapsed time. In practice that usually means several months at minimum, spanning at least one period where the strategy should struggle. If it has not yet had a bad stretch, what you are looking at is its drawdown so far. That is a different quantity from its drawdown, and only one of the two is any use for sizing. 

What is a realistic expectation? 

A strategy showing a small positive edge in a properly conducted walk-forward test, with real spread modelling and commission included, might be genuinely viable. Anything showing a large edge has probably been fitted. 

And the base rate any backtest argues against is a hard one. The largest study of the question followed 450,000 Taiwanese traders across fifteen years. The share who earned predictably positive returns came in below one per cent. 

What the tester is actually good for 

None of which makes systematic trading pointless. The tester is a tool for rejecting bad ideas rather than for confirming good ones, and it is far better at the first job than the second. Used that way it saves you money. Used the other way round it costs you some, slowly, with a chart attached to justify each instalment.