test(research): characterize timing and cost limitations

docs/research-backtest-contract.md

@@ -0,0 +1,90 @@
+# Research backtest timing and cost contract
+
+Status: **characterization only; model defects remain open**. Related issue:
+[#366](https://github.com/QuantStrategyLab/BinancePlatform/issues/366).
+Baseline inspected: `44a0cd6e54c12cabf8944b7fbed9080840451297`.
+This change adds synthetic tests and documentation only. It does not correct the
+simulator, generate real signals, select a strategy, or change runtime, orders,
+risk decisions, or notifications.
+
+## Current behavior versus required acceptance
+
+| Area | Observed in `research/backtest.py` | Required before treating results as comparable evidence |
+| --- | --- | --- |
+| Daily information boundary | `prepare_*_daily_indicators` includes the final close of day D; `align_symbol_data` joins on D's date, exposing it at D 00:00 and subsequent hours. | Persist bar open/close and availability timestamps; a decision cannot read an unfinished daily bar. Specify the UTC cutoff, boundary inclusion and source finality/latency. |
+| Hourly signal and fill | The hourly close is used to form the snapshot, select weights, and fill buys/sells on that same row. Binance kline `time` is parsed from open time; close time is discarded. | Declare signal availability and execution timestamp/price. A next-bar contract needs an actually available next quote; a same-close model must explicitly declare that idealization and cannot be silently compared with next-open fills. |
+| Missing data | Hourly closes are forward-filled after the first observed quote, with no quote-age flag. Leading gaps stay NaN. Filled prices can open or liquidate positions. | Distinguish mark-only carry-forward from executable quotes, reject stale fills, specify maximum age, outages, delistings and calendar coverage. |
+| Buy costs | Both trend and BTC DCA calculate quantity as `budget * 0.985 / price`, deduct only `quantity * price` and retain the buffer as cash. | Deduct explicit per-side fees and execution friction, with declared rates and provenance. The buffer is not a 150-bps fee. |
+| Sell costs | Trend exits, circuit-breaker liquidation and BTC DCA reductions credit `quantity * price` without costs. | Apply the chosen cost contract on every exit path as well as entry; preserve an auditable cash/quantity/fee ledger. |
+| Drawdown anchor | Curve records only post-processing rows; `cummax()` starts at the first recorded value. Initial capital is used for return but is not a separate pre-fill curve anchor. | Include initial capital before the first execution so first-fill costs/losses count in drawdown. Specify observation frequency and cash-flow treatment. |
+| Cost stress | No fee/slippage inputs, turnover series, or recomputed cost scenarios are returned. | Supply audited turnover and explicit base assumptions before computing 1×/2×/3× scenarios. Do not invent fee tiers or substitute cash buffers. |
+
+Current output must be labelled **未计成本 / costs not included; timing not
+validated**. Corrected net returns, cost-stress results, and causal timing
+validation are **not computed**. Existing headline return/drawdown output is not
+promotion evidence or a valid same-window net-performance comparison. The tests
+below do not change the script's printed labels; downstream use must honor this
+restriction until separately authorized implementation fixes the model.
+
+## Executable characterization
+
+Run offline after installing the repository's pinned dependencies:
+
+```sh
+python -m unittest discover -s tests -p test_research_backtest_contract.py -v
+python -m unittest discover -s tests -v
+PYTHONPATH=. python -m pytest -q tests
+```
+
+`tests/test_research_backtest_contract.py` calls the real alignment, indicator
+preparation and simulation functions. It replaces universe, weight selection and
+budget helpers with deterministic fixtures to isolate accounting and timing.
+No exchange client is created; `build_client` is guarded to fail. It does not
+validate strategy rankings or dependency allocation policy.
+
+The tests deliberately PASS on current limitations:
+
+- Changing only the final daily close changes that day's first-hour SMA20 and
+  BTC MA200; the previous hour is unchanged.
+- A selected first-row buy is filled at that row's close.
+- A missing-hour quote carried forward can execute a new buy.
+- A flat-price trend buy/sell round trip leaves all initial equity intact.
+- A 1,000-unit budget buys 9.85 units at 100, leaving 9,015 cash; selling at 110
+  gives 10,098.5. These are arbitrary fixture units, not investment returns or
+  a real cost assumption.
+- BTC DCA has the same cash-buffer behavior.
+- The curve contains only simulation-row timestamps and no pre-fill capital
+  observation. The zero-cost fixture does not manufacture a first-fill loss.
+
+These assertions are tripwires for explicit contract changes, not acceptance
+criteria for a correct backtester. When a future model repair changes them,
+replace the affected characterization with causal/cost acceptance tests rather
+than preserving the defect. There are no skipped or expected-failure tests
+masquerading as model validation.
+
+## Future acceptance fixtures (not implemented or claimed passing)
+
+1. Daily boundary: changing D's final close must not change decisions before its
+   finalized availability time. Test the instant before, at and after that time,
+   plus the final unfinished daily bar.
+2. Next-fill timing: a close-derived signal cannot fill at that close under a
+   next-bar contract. Test a price jump and missing next quote. Retain separate,
+   explicitly labelled cases if a same-close approximation is chosen.
+3. Stale market: allow separately labelled valuation carry-forward if intended,
+   but no execution on a fabricated quote. Cover both entry and all exit paths.
+4. Cost conservation: with arbitrary test-only fee `f`, a flat-price notional
+   `q * p` round trip loses exactly `2 * q * p * f` when that is the declared
+   per-side fee rule and the budget is sufficient. Test fee units, spread and
+   slippage separately, both trend and BTC, including circuit-breaker exits.
+   This algebra is a proposed test oracle, not today's model or an account rate.
+5. Initial loss: starting equity 100 and a first-fill cost 1 imply first observed
+   equity 99 and at least -1% drawdown relative to the initial anchor, even if
+   all later observations stay at 99. Current zero-cost fills cannot exercise
+   this acceptance case; it is not marked fixed by the curve-shape test.
+6. Cost scenarios and comparability: verify turnover units and per-side cost
+   accounting before recomputing 1×/2×/3× stress. Require matching data, period,
+   universe, availability, execution, valuation, initial capital and costs.
+
+Implementation of these fixtures against a corrected model is separate work.
+Issue #366 remains open; this document does not authorize a redesign or live
+configuration changes.

tests/test_research_backtest_contract.py

@@ -0,0 +1,159 @@
+"""Offline characterization of known research limitations, not model acceptance.
+
+See docs/research-backtest-contract.md and issue #366. These passing tests
+intentionally describe today's behavior; a model fix must revise the assertions.
+"""
+
+import unittest
+from unittest.mock import patch
+
+import numpy as np
+import pandas as pd
+
+from research import backtest
+
+
+class ResearchBacktestContractTests(unittest.TestCase):
+    def setUp(self):
+        # Any accidental data loading must fail rather than reach Binance.
+        self.client_patch = patch.object(
+            backtest, "build_client", side_effect=AssertionError("offline fixtures only")
+        )
+        self.client_patch.start()
+        self.addCleanup(self.client_patch.stop)
+        self.times = pd.date_range("2025-01-01", periods=3, freq="h", tz="UTC")
+
+    @staticmethod
+    def daily_bars():
+        times = pd.date_range("2024-01-01", periods=201, freq="D", tz="UTC")
+        return pd.DataFrame({
+            "time": times, "open": 100.0, "high": 100.0,
+            "low": 100.0, "close": 100.0, "vol": 10.0,
+        })
+
+    def test_final_daily_close_changes_same_day_first_hour_indicator(self):
+        for prepare, field, expected in (
+            (backtest.prepare_trend_daily_indicators, "sma20", 110.0),
+            (backtest.prepare_btc_daily_indicators, "ma200", 101.0),
+        ):
+            with self.subTest(field=field):
+                daily = self.daily_bars()
+                day = daily.time.iloc[-1]
+                timeline = pd.date_range(day - pd.Timedelta(hours=1), periods=3, freq="h")
+                hourly = pd.DataFrame({"time": timeline, "close": 100.0})
+                baseline = backtest.align_symbol_data(hourly, prepare(daily), timeline)
+                # Only the future final daily close changes; hourly history is identical.
+                daily.loc[daily.index[-1], "close"] = 300.0
+                changed = backtest.align_symbol_data(hourly, prepare(daily), timeline)
+                self.assertEqual(changed[field].iloc[0], baseline[field].iloc[0])
+                self.assertEqual(baseline[field].iloc[1], 100.0)
+                self.assertEqual(changed[field].iloc[1], expected)
+                self.assertEqual(changed[field].iloc[2], expected)
+                # Known lookahead: D's final close is visible at D 00:00.
+
+    def test_missing_hour_close_is_forward_filled_but_leading_gap_is_not(self):
+        hourly = pd.DataFrame({"time": [self.times[1]], "close": [100.0]})
+        aligned = backtest.align_symbol_data(hourly, pd.DataFrame(), self.times)
+        self.assertTrue(pd.isna(aligned.close.iloc[0]))
+        self.assertEqual(aligned.close.iloc[1:].tolist(), [100.0, 100.0])
+
+    def market(self, prices):
+        data = {}
+        for symbol in ("BTCUSDT", "ETHUSDT"):
+            data[symbol] = pd.DataFrame({
+                "time": self.times[:len(prices)],
+                "close": prices if symbol == "ETHUSDT" else 100.0,
+                "sma20": 50.0, "sma60": 50.0, "sma200": 50.0,
+                "atr14": 1000.0, "ma200": 50.0, "ma200_slope": 1.0,
+                "geom200": np.nan, "zscore": np.nan, "sell_trigger": np.nan,
+            })
+        return data
+
+    def run_fixture(self, data, weights, *, dca=False):
+        def allocation(total, cash, trend, btc):
+            return {"trend_usdt_pool": 0.0 if dca else 1000.0,
+                    "trend_layer_equity": trend,
+                    "dca_usdt_pool": 1000.0 if dca else 0.0}
+
+        with (
+            patch.object(backtest, "ALL_SYMBOLS", ["BTCUSDT", "ETHUSDT"]),
+            patch.object(backtest, "FIXED_POOL", ["ETHUSDT"]),
+            patch.object(backtest, "select_rotation_weights", side_effect=weights),
+            patch.object(backtest, "compute_allocation_budgets", side_effect=allocation),
+            patch.object(backtest, "allocate_trend_buy_budget",
+                         side_effect=lambda active, eligible, budget: {
+                             symbol: budget for symbol in eligible
+                         }),
+            patch.object(backtest, "get_dynamic_btc_base_order", return_value=1000.0),
+        ):
+            return backtest.simulate_window(
+                {"name": "synthetic", "label": "synthetic", "pool_mode": "fixed"},
+                "synthetic", data["BTCUSDT"].time.iloc[0],
+                data["BTCUSDT"].time.iloc[-1], data, capture_details=True,
+            )
+
+    def test_same_row_signal_buys_at_that_rows_close(self):
+        data = self.market([100.0, 110.0])
+        observed = []
+
+        def weights(strategy, rows, gate, ts, state):
+            observed.append((ts, rows["ETHUSDT"]["close"]))
+            return {"ETHUSDT": 1.0}
+
+        result = self.run_fixture(data, weights)
+        self.assertEqual(observed[0], (self.times[0], 100.0))
+        self.assertEqual(result["trades"], 1)
+        # 985 / first-row close = 9.85 units; a next-row fill gives no gain here.
+        self.assertEqual(result["equity_series"].tolist(), [10000.0, 10098.5])
+
+    def test_flat_trend_round_trip_deducts_no_cost(self):
+        result = self.run_fixture(self.market([100.0, 100.0]), [{"ETHUSDT": 1.0}, {}])
+        self.assertEqual(result["trades"], 2)
+        self.assertEqual(result["final_equity"], backtest.INITIAL_CAPITAL)
+        self.assertEqual(result["total_return"], 0.0)
+        # The 1.5% buy budget buffer stays cash; it is not a fee.
+
+    def test_trend_round_trip_retains_budget_buffer_as_cash(self):
+        result = self.run_fixture(self.market([100.0, 110.0]), [{"ETHUSDT": 1.0}, {}])
+        self.assertEqual(result["trades"], 2)
+        self.assertEqual(result["final_equity"], 9015.0 + 9.85 * 110.0)
+
+    def test_dca_budget_buffer_is_also_unspent_cash(self):
+        data = self.market([100.0, 100.0])
+        data["BTCUSDT"]["close"] = [100.0, 110.0]
+        data["BTCUSDT"]["geom200"] = 100.0
+        result = self.run_fixture(data, [{}, {}], dca=True)
+        self.assertEqual(result["trades"], 1)
+        self.assertEqual(result["equity_series"].tolist(), [10000.0, 10098.5])
+
+    def test_flat_dca_buy_and_partial_sell_deduct_no_cost(self):
+        data = self.market([100.0, 100.0])
+        data["BTCUSDT"]["geom200"] = 100.0
+        data["BTCUSDT"]["zscore"] = [0.0, 3.0]
+        data["BTCUSDT"]["sell_trigger"] = 2.5
+        result = self.run_fixture(data, [{}, {}], dca=True)
+        self.assertEqual(result["trades"], 2)
+        self.assertEqual(result["equity_series"].tolist(), [10000.0, 10000.0])
+
+    def test_forward_filled_quote_can_be_used_for_a_new_trade(self):
+        data = self.market([100.0, 100.0, 110.0])
+        raw = pd.DataFrame({"time": self.times[[0, 2]], "close": [100.0, 110.0]})
+        aligned = backtest.align_symbol_data(raw, pd.DataFrame(), self.times)
+        data["ETHUSDT"]["close"] = aligned.close
+        self.assertNotIn(self.times[1], raw.time.tolist())
+        result = self.run_fixture(data, [{}, {"ETHUSDT": 1.0}, {"ETHUSDT": 1.0}])
+        self.assertEqual(result["trades"], 1)
+        self.assertEqual(result["equity_series"].tolist(), [10000.0, 10000.0, 10098.5])
+
+    def test_equity_curve_has_only_post_processing_rows_no_initial_anchor(self):
+        result = self.run_fixture(self.market([100.0, 90.0]), [{"ETHUSDT": 1.0}] * 2)
+        curve = result["equity_series"]
+        self.assertEqual(curve.index.tolist(), self.times[:2].tolist())
+        self.assertEqual(len(curve), 2)
+        self.assertAlmostEqual(result["max_drawdown"], 9901.5 / 10000.0 - 1.0)
+        # This zero-cost fixture cannot expose a first-fill loss. The missing
+        # pre-fill capital anchor remains a future cost-model acceptance case.
+
+
+if __name__ == "__main__":
+    unittest.main()