Skip to main content

Research runs: walk-forward validation

A single backtest is in-sample. A research run is a validation job instead of one backtest: it optimises the parameters you would otherwise sweep by hand on training spans, then tests what it chose on the data right after, which the optimiser never saw. Its verdict is judged on those out-of-sample test folds. One research job counts as one try for the overfit check.

What you send​

The same backtest spec, plus a research block:

{"protocol": "walk_forward",
 "plan": {"plan": "rolling", "train_days": 180, "test_days": 30, "step_days": 30},
 "space": {"fast": {"low": 5, "high": 50, "step": 5}},
 "n_trials": 40, "objective": "sharpe"}
  • protocol: optimize, walk_forward, nested_walk_forward, multi_config, purged_cv, cpcv, masters4 or monte_carlo.
  • plan (days): rolling or anchored, with train_days, test_days and step_days (step_days defaults to test_days). The default plan is rolling 180 / 60. purged_cv and cpcv take their own plans with purge and embargo days.
  • space: per parameter a range {"low", "high"} (with step or log) or {"choices": [...]}. "sampler": "grid" tries every configuration of a finite space once; the default samples n_trials of them.
  • objective: sharpe, return, calmar or profit_factor.

Validate it first, for free: any problem in the research block is a finding on its field, and the answer quotes the cost. Research is charged by run time; more configurations and more folds mean more run time.

In a chat app with the OptAlgo connector, the research tool does this in one call from a space of parameters, and can keep the last months out as a holdout (holdout_months).

Reading the result​

The verdict is judged on the out-of-sample folds: robust, inconclusive or likely_overfit. Only robust is a pass; inconclusive is not.

  • oos_return is the chained out-of-sample return; the stitched out-of-sample equity curve is in the result.
  • Each fold shows the parameters it chose, its train and test scores, its trades and its windows. The fold scores are the objective (per-bar Sharpe for "objective": "sharpe"), not returns.
  • n_trials is the number of distinct configurations evaluated (what the deflated Sharpe corrects for); evaluations is configurations × folds.
  • A research run keeps its report and spec, not trades or candles, so it has no trade chart.

Validate fixed parameters​

Already have parameters and no search to run? Use a walk-forward with one configuration: every fold tests the same parameters on data after its training span, and it counts as 1 trial.

{"protocol": "walk_forward", "plan": {"plan": "rolling", "train_days": 365, "test_days": 90, "step_days": 90},
 "space": {"lb1": {"choices": [42]}, "lb4": {"choices": [336]}}, "sampler": "grid", "n_trials": 1, "objective": "sharpe"}

Or run the parameters once, as a plain backtest, on a window you never tuned on, and report that run.

Several symbols​

With several symbols and a one-instrument strategy, every trial runs the whole basket: the same parameters on every symbol, the objective on the summed basket equity, and folds that split time for all symbols together. Each fold shows which coins carried or hurt it.

Why it matters for deploying​

A backtest with the search verdict likely_overfit can only run on OptAlgo once a research run of the same code, with a protocol that tests on unseen data and the verdict robust, covers every symbol of that backtest. It never lifts a no_edge block. See Overfit check and the honesty grade.

Cost and limits​

A research run is charged in backtest units on its total run time. The number of trials per run depends on your plan: 40 on Free, 100 on Plus, 500 on Pro (Plans and limits). You can also start one from the Backtest page.

Full reference: Research runs in the LLM reference.