Allow wrapped C++ methods to detect request cancellations

[xyzconstant]

Currently, a server method runs to completion regardless of what the client does, even if the client disconnects. If the client wants to stop waiting there is no way to tell the server. Downstream works around this by coupling each blocking method with an interruptX() method designed solely to wake it up. For example, bitcoin/bitcoin#33676 introduced BlockTemplate::interruptWait() specifically to wake a BlockTemplate::waitNext() that is in progress.

A non-C++ client would expect the server to stop when it drops the call, since that is how cancellation normally works in Cap'n Proto but the method keeps running to completion anyway. What's missing is a way for wrapped C++ methods to detect the cancellation.

This PR implements approach 4 suggested in bitcoin/bitcoin#33575. It introduces a new Cancel proxy parameter that maps to a CancelToken C++ parameter, so methods can stop long-running jobs as soon as the promise executing the request is abandoned. For instance, the following capnp schema method:

waitForValue @0 (context :Proxy.Context, minValue :Int32, cancel :Proxy.Cancel) -> (result :Int32);

Could be defined in C++ this way, blocking in a std::condition_variable::wait() until either the job finishes or the request is canceled:

int FooImplementation::waitForValue(int min_value, CancelToken cancel)
{
    // Construct the callback before locking the mutex it uses
    const OnCancel wake{cancel, [this] {
        const std::lock_guard<std::mutex> lock{m_wait_mutex};
        m_wait_cv.notify_all();
    }};
    std::unique_lock<std::mutex> lock{m_wait_mutex};
    m_wait_cv.wait(lock, [&] { return m_value >= min_value || cancel.canceled(); });
    return m_value;
}

Notice that Cancel must be declared after a Context parameter. Otherwise, the method will run on the event loop thread, where cancellations are delivered, and will run until completion with a token that will never fire. This rule is enforced by mpgen.

Some implementation details:

• ProxyServerBase now overrides dispatchCall, the capnp-generated entry point the RPC system uses to route incoming requests. This override inspects each method's parameters and allows Cap'n Proto to cancel calls to methods that declare Cancel. This is equivalent to $Cxx.allowCancellation but is derived from the schema rather than an annotation. Older capnp versions (<1.0) also benefit from this (see proxy-io.h).
• 3 additional classes are added: CancelState to extend CancelMonitor's single m_on_cancel callback, CancelToken to report cancellations to wrapped C++ methods, and OnCancel to install callbacks via RAII. Together, they mirror std::stop_source/std::stop_token/std::stop_callback.

Please note, this work is server-side only. When you call a ProxyClient's method, it will still block at clientInvoke and wait for the promise to resolve. If needed, we could consider making clientInvoke cancellable in the future.

[DrahtBot]

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

Reviews

See the guideline and AI policy for information on the review process.
A summary of reviews will appear here.

[xyzconstant]

CI failures seem unrelated

[ryanofsky]

Nice work! I've had a number of ideas about this feature over the years so it's really interesting to see it implemented. I will just give some quick thoughts for now so I don't get nerdsniped and wind up spending all day or more on this. Thoughts:

• I don't think the proxy.capnp Cancel struct should exist. The idea of allowing a CancelToken / interfaces::Cancel argument from RFC: Cancelling waitNext calls in the IPC mining interface bitcoin/bitcoin#33575 is just to provide a way for libmultiprocess C++ clients to cancel capnp::Response promises, and for libmultiprocess C++ servers to detect whether the promises they are fulfilling have been cancelled. Rust and python and non-libmultiprocess C++ clients and servers already have a way to do these things, so they should not be affected by implemntation of this feature, and not see any differences in .capnp schema files if a C++ method supports it or doesn't support it.

• Relatedly, the implementation should be orthogonal to the $Cxx.allowCancellation(true); annotation. The annotation controls how capnproto sends cancellations, while CancelToken is way of letting libmultiprocess C++ classes send and receive them. CancelToken will probably be most useful combined with $Cxx.allowCancellation(true); annotations, but it could be useful without them too, for example to interrupt calls on unclean disconnects, but not interrupt calls when clients drop capnp::Response promises because they don't need the results.

• In order for this feature to be useful in Bitcoin Core, it shouldn't require C++ interfaces to directly use the mp::CancelToken type. Bitcoin Core C++ interfaces in src/interfaces/ are intended to be used by node/wallet/gui code and compile without any dependency on libmultiprocess. So libmultiprocess should provide CustomBuildCancel and CustomReadCancel overloads analagous to CustomBuildField and CustomReadField overloads that applications can override to work with custom cancellation arguments. Probably the simplest cancellation argument type would look like:

  using CancelFn = std::function<void()>; // Called when a request is cancelled
  using CancelArg = std::function<void(CancelFn)>; // Called to set a CancelFn that is called when a request is cancelled.

  and support overloads like

  template <typename LocalType, typename Value>
  void CustomBuildCancel(TypeList<CancelArg>, InvokeContext& invoke_context, Value&& value)
  {
      // If client provied a CancelArg argument, call it to give them a CancelFn
      // callback they can use to interrupt this request.
      if (value) value([&invoke_context] { invoke_context.cancel_request(); } };
  }
  
  template <typename LocalType, typename ReadDest>
  decltype(auto) CustomReadCancel(TypeList<CancelArg>, InvokeContext& invoke_context, ReadDest&& read_dest)
  {
      // Return a CancelArg for servers call to register a CancelFn and be
      // notified if the current request is cancelled.
      return read_dest.construct([&invoke_context](CancelFn cancel_fn) invoke_context.on_request_cancelled(std::move(cancel_fn)); });
  }
  ``
  
  Having CustomBuildCancel/CustomReadCancel hooks would let libmultiprocess be agnostic to whatever cancellation interfaces C++ applications want to use, and just give clients a way to send cancellations and servers a way to receive them.
  

• Alternately instead of adding CustomBuildCancel/CustomReadCancel hooks, we could use existing CustomBuildField/CustomReadField hooks with empty input/output arguments. I think to do this would need to add a new hook called something like CustomFieldExists() that is constexpr and returns true by default but false if no capnproto field corresponding to the C++ type will exist. This would let clientInvoke/serverInvoke code handle C++ arguments that don't have corresponding capnproto fields, and be be similar to existing CustomHasField() / CustomHasValue() overloads but be constexpr and reflect whether the field exists at all, not whether it is has a value set.

• It looks like this implementation as of 0986a13 only allows libmultiprocess servers to detect cancellations, but doesn't allow libmultiprocess clients to request cancellations. This is ok, but it probably makes sense to support both because a single argument can support both, and so the feature can be more naturally tested end-to-end.

• Commit 3f6b324 is an interesting way to provide compatibility with older versions of capnproto that don't support $Cxx.allowCancellation annotations. I think in practice it's probably fine to drop support for these older versions though, and if we did want to support them we should probably just provide an $Cxx.allowCancellation annotation for them to use and call context.allowCancellation() when it's present.

• The CI failures in https://github.com/bitcoin-core/libmultiprocess/actions/runs/31636595747/job/94248350765?pr=342 are caused by this PR but they should be easy to fix. The -Wc++23-lambda-attributes errors are garbage from IWYU output but if you look below you will see real IWYU errors that need to be fixed. Once the IWYU errors are fixed all the IWYU output should disappear and CI should be green.