Improve fanout webhook delivery timeouts and retry budget

[migmartri]

Problem

The fanout dispatcher's retry budget was 10 seconds total (MaxElapsedTime), with no per-attempt timeout — a single hung HTTP request could consume the entire budget in one attempt. The webhook plugin's http.Client also had no timeout set, and context.TODO() was passed through to plugin.Execute, so there was no hard cutoff other than TCP/OS defaults.

Additionally, the retry loop did not respect parent context cancellation — even if the parent context was cancelled, backoff.RetryNotify would keep retrying until MaxElapsedTime expired because the backoff was not wrapped with backoff.WithContext.

This is a partial fix for the reliability gaps described in #39 — specifically the timeout/retry improvements from Option 1 (improve the current in-process system).

Solution

• Bump MaxElapsedTime from 10s to 5m: with the existing exponential backoff (500ms initial, ×1.5 multiplier, 60s max interval), this yields ~12-15 attempts — enough to ride out a transient blip, short enough not to leak goroutines in the fire-and-forget dispatch model.
• Add per-attempt timeout (10s): each Execute call now runs under context.WithTimeout(ctx, sdk.PerAttemptTimeout) so a single hung request cannot eat the whole retry budget.
• Set http.Client.Timeout on the webhook plugin: the webhook http.Client now has Timeout: sdk.PerAttemptTimeout (10s), matching the dispatcher's per-attempt deadline. The cutoff is enforced at whichever layer sees the stall first.
• Wrap backoff with parent context: backoff.WithContext(b, ctx) ensures the retry loop stops promptly when the parent context is cancelled, rather than waiting for the full 5m budget.
• Export sdk.PerAttemptTimeout: shared constant in the plugin SDK so the dispatcher and plugins can't silently drift on the timeout value.
• Bump webhook plugin version 1.1 → 1.2: reflects the new per-attempt timeout behavior.
• Replace deprecated io/ioutil with io in the webhook plugin.

Limitations

This is the in-process improvement (Option 1). The dispatch is still fire-and-forget in goroutines — not durable across restarts, no DLQ, delivery failures still only logged. A durable queue solution (NATS/River) is tracked separately in #39.

Refs #39

🤖 Posted by Maximus bot (OpenCode) on behalf of @migmartri