Skip to content

Document the fill-model hazard when evaluating an externally-sourced strategy #529

Description

@eaitbrahim

From the 2026-08-23 Jesse-comparison PRD (docs/superpowers/specs/2026-08-23-strategy-api-expressiveness-prd.md §4.2).

The hazard

Strategies published for other frameworks routinely name their own entry price and assume the fill. Jesse's idiom is literally:

def go_long(self):
    entry = self.price
    stop  = entry - self.atr * self.hp['stop_loss_atr_rate']
    qty   = utils.risk_to_qty(self.balance, 2, entry, stop)
    self.buy = qty, entry          # <- a resting order at a chosen price
    self.stop_loss = qty, stop

keel does not work that way, and has measured what the difference costs.

Since #258, backtest() fills an entry at the next bar's open plus slippage, because that is what execution/executor.py actually places live (order_type="market", limit_price=None). It does not rest an order and wait for a level.

Correcting pullback_continuation to that fill model dropped its gross profit factor from 0.92 to 0.77 and doubled its trade count — a market fill takes the trades the strategy meant to decline, because its entry = signal_bar.high + buffer was demanding follow-through it no longer gets.

So: any externally-sourced strategy ported naively inherits an optimism keel has already priced. Its published numbers are not wrong for its own framework; they are unreachable in this one.

This compounds with the cost picture. Round-trip friction here is ~2.5% of notional. A strategy whose edge depends on getting a chosen price, evaluated in a framework that assumes it does, is optimistic on both axes at once — and neither optimism is visible in the headline return.

Why documentation rather than code

There is nothing to enforce. The fill model is already correct; the hazard is in how a human reads someone else's results. The failure mode is a reasonable person seeing a good backtest elsewhere and not knowing which two questions to ask.

Proposal

A short section — most naturally in docs/experiments/ alongside the cost-restatement documents, or as a checklist in the research docs — stating what to establish before a foreign strategy's numbers mean anything here:

  1. What fill model produced them? A resting limit/stop entry, or a market order? If resting, its edge may not survive keel's next-open fill, and pullback_continuation's 0.92 → 0.77 is the worked example of how much.
  2. What cost regime? Fees and slippage assumed. A result developed at 5bp does not transfer to a 1.2% taker venue, and nothing in the abstract will say so.
  3. What sample size and window? A headline profit factor on a small sample is a lottery ticket — rsi_meanrev showed 1.1631 at median n=38 and 0.8396 across 82 cells at n≥100.
  4. Does it need a capability keel does not have? Pyramiding, partial exits, and conditional entries are Broker port has no bracket/OCO order kind — live trailing stops, BE-rolls and scale_out are blocked on it #502/Route a rule's conditional entry as a genuine resting order (limit/stop) — deferred until a price-conditional rule earns it #333 territory; a strategy depending on them is not evaluable here yet, and pretending otherwise mis-models it.

Acceptance criteria

  • The four questions live somewhere a person evaluating an external strategy will actually meet them.
  • The pullback_continuation 0.92 → 0.77 example is cited concretely — the number is what makes the point land.
  • Cross-referenced from wherever strategy candidates get proposed, so it is read before effort is spent rather than after.

Notes

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

Relationships

None yet

Development

No branches or pull requests

Issue actions