Why most backtests lie
An EA showing 200% annual return at 10% drawdown in backtesting sounds like the Holy Grail. In live trading it collapses within three months. The broker is rarely to blame — the culprit is almost always curve-fitting: the EA was unknowingly optimised on the test history, not for the future.
Rigorous testing means systematically filtering out this bias. The steps below apply to both MT4 and MT5; the underlying principles hold across platforms.
Step 1: Choose your data source
The quality of a test is only as good as its tick data.
MT5 downloads tick data directly from the broker — quality varies by provider. For a reliable baseline, import from an independent source such as Dukascopy, which provides gapless 1-tick history.
MT4 works internally with M1 bars and interpolates ticks. The "Every tick (based on real ticks)" model improves quality somewhat, but is not a true tick import. For latency- or spread-sensitive EAs, use MT5 or a dedicated backtesting solution (Soft4FX, TickStory).
Before every test run, check:
- Are all bars in the test period present (no data gaps)?
- Is the modelling quality above 90%?
Step 2: Model realistic costs
A backtest without accurate costs is worthless.
- Spread: use the real average spread of your target broker — not the minimum. Spreads widen significantly around news events.
- Commission: ECN brokers typically charge USD 3–7 per lot (round-turn). These costs must be entered explicitly in the tester or accounted for in the EA's code.
- Slippage: MT4/MT5 allow a fixed slippage value in points. Set it to a realistic figure — for a scalping strategy on a standard broker, 2–5 points per order is realistic; on a latency-optimised VPS with an ECN broker, somewhat less.
Step 3: Separate in-sample from out-of-sample
In-sample (IS): the data on which you optimise.
Out-of-sample (OOS): the untouched remainder on which you validate.
A practical rule of thumb: 70% IS, 30% OOS. The critical point is to set the OOS period aside before optimising, then test on it exactly once. If you run multiple optimisation passes on the OOS window you effectively turn it into a second IS set.
Step 4: Walk-forward analysis
The walk-forward test generalises the IS/OOS principle across rolling windows:
- Optimise on window 1 (e.g. 6 months).
- Test the best parameters on window 2 (e.g. 3 months).
- Slide the window forward and repeat — typically 5–10 cycles.
If the EA stays consistently profitable across all OOS windows (even if absolute returns are lower), that is a considerably stronger signal than a single polished backtest. Tools like the MT5 walk-forward optimiser or third-party solutions such as Adaptrade make this process manageable.
Step 5: Robustness checks — beyond optimisation
Beyond walk-forward, additional robustness tests are worth running:
- Monte Carlo analysis: randomly shuffling or dropping trades shows how sensitive the equity curve is to the exact trade sequence.
- Parameter sensitivity: vary each parameter by ±10–20% around the optimal value. Robust EAs show a broad "plateau"; fragile ones show a sharp peak.
- Multi-symbol / multi-timeframe test: does the strategy work on related pairs or timeframes with similar parameters? If it only fires on one exact setting, treat that as a warning sign.
Step 6: Forward-test in live conditions
No backtest replaces the real market. After passing the OOS test, run the EA on:
- Demo account — free, but without real execution quality.
- Live micro-lot (0.01 lot) — low-cost real-market test; slippage and true spread become visible.
A valid forward test needs at least 100–300 trades or 3–6 months, whichever comes first. Only then are drawdown and edge claims statistically meaningful.
For a reliable forward test, stable uptime is essential — that is what a VPS provides. For which brokers offer the best conditions for realistic testing (tight spreads, honest execution), see our broker reviews.
The most common mistakes
| Mistake | Consequence |
|---|---|
| Only "Every tick" on broker data | Unrealistic ticks, results too good |
| Spread fixed at minimum | Underestimated costs, overfitted strategy |
| Multiple optimisation passes on OOS | OOS effectively becomes IS |
| Too many parameters | High overfitting probability |
| Forward test on demo only | No real execution feedback |
Conclusion
A sound EA test rests on three pillars: genuine tick data with realistic costs, a clean OOS window, and a live forward test on a micro-lot account. The walk-forward test is the strongest tool against overfitting. Anyone who maintains this discipline protects themselves against the most expensive mistake in algo trading: putting real money into a curve-fitted system.