BCK_EG1: Tick-Level Backtest Engine

A tick-driven backtesting engine, and a downstream use case for the tick-data and economic-news data pipelines. It replays real tick data through a realistic execution model and reports performance with walk-forward splitting, rolling-window validation, and Monte-Carlo robustness analysis, in both single-symbol and shared-account (portfolio) modes.

This repository ships the engine only. It contains no market data and no trading strategy. The folder strategies/example_signal_strategy/ is a documented template that shows how a strategy plugs in.

📺 Channel: youtube.com/@BDB5905

About this repo

This is a side project I built and decided to share as part of showing my work. It is not a maintained product, and it is not really meant to be cloned and run as-is. If you do want to run it, or you want the full explanation of how the pieces fit together, get in touch through the channel above.

Where things live

The engine and all of its code are in the BCK_EG1/ folder. Run every command shown below from inside it:

cd BCK_EG1

Problem statement

The upstream projects solved data collection. Raw broker ticks and economic-news data were gathered, cleaned, partitioned, and curated into query-ready tables in the data lakehouse. Clean data on its own does not tell you whether a trading strategy works.

What was missing was a way to run tests, simulations, and validations against that data: to take a strategy's rules and measure, quantitatively, how they would have performed. The returns, the drawdowns, the risk-adjusted numbers, and how much of the result is a real edge rather than luck.

BCK_EG1 is that missing piece. It replays the curated tick data through a realistic execution model and reports strategy performance with proper out-of-sample discipline (walk-forward and rolling-window validation) plus Monte-Carlo robustness checks, so a strategy can be judged on evidence rather than a single cherry-picked run.

Why tick-level

Bar-based backtests hide the two things that decide whether an intraday edge survives contact with a broker: when your order actually fills, and at what price. BCK_EG1 works from raw ticks so it can model:

• entry and exit timing to the tick, not the bar close
• spread, slippage, and commission per fill
• partial fills and book-walk pricing for large orders (market-depth model)
• iceberg order slicing
• margin and stop-out behaviour
• news and holiday blackout windows

Architecture

Upstream pipelines collect market data and land it in the AWS Data Lakehouse, where AWS Glue ETL curates it into Apache Iceberg Gold tables. Gold tables (and the signal model's output) are exported into the local folders that config.json points at. The engine only ever reads from those local folders. It has no cloud dependency of its own.

Repo	Role	Feeds
Big-Data-Project-Jan-2026-Tick-Data-	Collect raw broker tick data	Lakehouse
MQL5-Economic-News-Data-Pipeline-2025	Scrape MQL5 economic calendar and market holidays	Lakehouse
AWS-Data-Lakehouse	S3 raw, then Glue ETL, then Iceberg Gold tables, queried via Athena / Trino / Spark	data/ticks, data/specs, data/news, data/holidays
ML signal model (external)	Consume Gold datasets, produce daily signal files	data/signals

The three pipeline repos above are earlier projects and are illustrative. The current design simply routes all data through the lakehouse.

How a run works

1. config.json is loaded by core/config/config_loader.py. It holds the data paths, starting capital, execution quality, validation ratios, and the multi-asset list.
2. The strategy package (strategies/<name>/) provides trade signals. The example strategy reads pre-computed daily signal files; your own strategy can generate them however it likes.
3. backtest.py applies an optional hard train/test split. rolling_validation.py builds rolling in-sample and out-of-sample windows instead.
4. core/simulation replays ticks day by day, opening and closing positions through the core/execution model and honouring core/news blackout rules.
5. core/metrics computes performance stats. core/reporting writes CSVs and equity-curve charts to results/.
6. monte_carlo_backtest.py re-samples the completed trades.csv (bootstrap and order-shuffle) to separate luck from a real edge.

Repository layout

All paths below are inside BCK_EG1/.

Path	Purpose
core/	The engine: data loading, execution modelling, simulation, metrics, reporting. Not edited per strategy.
core/data/	Tick and symbol-spec loading (ISO-week-partitioned parquet).
core/execution/	Fills, commission, slippage, market depth, iceberg orders, margin.
core/simulation/	The replay engine, intraday session logic, portfolio simulator.
core/news/	Economic-news and market-holiday blackout filters.
core/metrics/, core/reporting/	Performance stats, CSVs, plots.
core/rolling/, core/analysis/	Rolling-window validation, Monte-Carlo analysis.
core/verify/	Self-check that recomputes key numbers by hand (python -m core.verify.verify_calculations).
strategies/<name>/	A pluggable strategy package. The engine imports one by name.
config.json	Global settings (see below).
backtest.py, rolling_validation.py, monte_carlo_backtest.py	Single-symbol entry points.
*_multi_symbol.py	Portfolio variants: all symbols on one shared account.

Install

pip install -r requirements.txt

Requires Python 3.10 or newer. Dependencies: numpy, pandas, pyarrow, matplotlib.

Configure

Edit BCK_EG1/config.json. Paths are resolved relative to BCK_EG1/ (where config.json lives); absolute paths also work.

Section	Key settings
paths	tick_root, specs_root, signal_root, output_root, news_root, holiday_root
trading	initial_capital, leverage, commission_per_lot_usd, execution_quality (good / moderate / bad), max_drawdown_stop, large_lot_threshold
walk_forward	enabled, train_ratio. The hard split used by backtest.py
validation	backtest_ratio, walk_forward_ratio, target_windows, cores. Rolling validation
multi_asset	symbols, exclude_symbols, max_total_signals_per_day, use_priority
monte_carlo	n_sims, seed
news, holidays	enabled, blackout windows, per-symbol currency calendars

Data you provide

Point config.json paths at folders shaped like this:

data/
  ticks/<SYMBOL>/<iso_year>/week_<iso_year>_W<week>.parquet   # tick data
  specs/<SYMBOL>/specs.csv                                    # MT5-style symbol specs
  signals/<SYMBOL>/<SYMBOL>.parquet                           # per-strategy signal files
  news/<CURRENCY>/<CURRENCY>.parquet                          # optional: economic events
  holidays/<CURRENCY>/<CURRENCY>.parquet                      # optional: market holidays
results/                                                      # run outputs

Tick partitions are keyed by ISO calendar week, so the folder is the ISO year, not the calendar year (ISO week 1 can contain late-December dates of the previous year). See core/data/data_loader.py for the exact resolver.

Signal file schema

One row per symbol per calendar day:

column	meaning
date	trading date, for example 2024-01-15
signal	BUY, SELL, or FLAT (FLAT and unknown days are skipped)
entry_time	HH:MM[:SS], market entry time on that date
exit_time	HH:MM[:SS], fixed-time close on that date

There is no price level in the file, so entries are market orders at entry_time and positions close at exit_time. Per-symbol risk (lot size, stop-loss, take-profit, and so on) comes from the strategy package's own config.json.

Where signals come from is up to you. The engine does not care how the date / signal / entry_time / exit_time rows were produced. They can be the output of a machine-learning model, a rules engine, technical indicators, a discretionary process exported to a file, or a completely separate repository. The engine just replays whatever is in the signal files through the execution model.

Claude Code / Cursor friendly

Once the data sources are collected, cleaned, and arranged into the layout above, this repo is set up so an AI coding assistant (Claude Code, Cursor) can take it from there. The strategy contract is small and lives in one documented place (strategies/example_signal_strategy/signal_provider.py), so you can point the assistant at your data folders and it can wire up signal loading and start running backtests almost immediately, without needing to understand the whole engine first.

Running

python backtest.py BTCUSD              # hard train/test split (walk_forward in config.json)
python backtest_multi_symbol.py        # all multi_asset.symbols on one shared account
python rolling_validation.py           # rolling in-sample / out-of-sample windows
python rolling_validation_multi_symbol.py
python monte_carlo_backtest.py BTCUSD  # resample a completed backtest's trades
python monte_carlo_multi_symbol.py

With no symbol argument, the single-symbol scripts fall back to the multi-asset list in config.json.

Outputs

Everything lands under results/ (config.json paths.output_root):

File	Contents
trades.csv, trades_out_of_sample.csv	Every trade: entry and exit time and price, lot size, PnL, capital after, exit reason
summary.csv, summary_out_of_sample.csv	Profit factor, total return, Sharpe, win rate, trade count, final capital
*_equity_curve.png	Equity curve chart
rolling_validation/...	Per-window trades, summaries, and combined charts
Monte Carlo Analysis/...	Bootstrap and shuffle distributions plus summary CSVs

Adapting it

If you want to swap in your own strategy:

1. Copy strategies/example_signal_strategy/ to strategies/<your_name>/.
2. Implement your signal loading or generation in signal_provider.py, keeping the public function names the engine imports (load_signals, get_entry_signal_dates, max_trades_per_day, _symbol_trading_cfg, and so on).
3. Repoint the strategies.example_signal_strategy imports in core/ at your package (about a dozen files; grep for the name).
4. Tune per-symbol risk settings in your package's config.json.

Then sanity-check the engine wiring:

python -m core.verify.verify_calculations

Roadmap

• C++ backtest engine. A native C++ port of this engine is planned for release before the end of 2026, aimed at large tick datasets and parameter sweeps where the Python replay loop becomes the bottleneck. It will keep the same config format and strategy contract.
• Additional built-in example strategies.
• Pluggable strategy discovery (drop-in packages, no core/ edits).

Notes

• A video walking through this repo, how it is structured, how the execution model works, and how to run it, is on YouTube: youtube.com/@BDB5905. The channel also covers the upstream data pipelines and the C++ rewrite as it lands.
• This repo is a shared side project, not a maintained library. If you want to run it, or you need the full picture of how the pieces connect, contact me through the channel.

Disclaimer

Research tooling only. Not investment advice. No strategy, tuned parameters, or market data are included in this repository. Past backtested performance does not predict future results.

This repo was meant to showcase and outline the use case of the data pipeline repos we previously created, outlining the fact that the data needs to be seriously partitioned and cleaned as this use case is very technical and has limited room for error.