Consider implementing publish flow control

[hannahrogers-google]

When a streaming connection to the pub/sub lite service is started, an InitialRequest is sent to establish the stream. Typically, we wait until an initial response is received before the stream enters the "running" state.

However, if the InitialRequest fails due to a retryable error, the connection is retried and the stream enters the "running" state before an initial response has been received.

[dpcollins-google]

This is WAI from my perspective. The stall before the InitialResponse could be removed entirely for parity if you'd like, but because the client library automatically manages retries of retryable errors in the ready state without user intervention, there's nothing inherently wrong with being in the ready state and having received retryable errors.

Has this caused confusing behavior in any way? This can be changed, but I don't necessarily see the motivation or the end state that would be better.

[dpcollins-google]

After offline discussion:

The underlying issue is that the stream will retry forever, buffering a potentially unbounded backlog of user messages that are being retried while the backend serves UNAVAILABLE continuously. Unlike Pub/Sub, a subset of publishes cannot be failed without losing ordering, so the alternative would be to fail the client if that backlog gets too high.

In the short term, this will be 'resolved' by making it the user's responsibility. The user can easily track the number of outstanding publishes using the returned futures, and can take action if that number grows too large.

If there is an expressed desire for a publish flow control mechanism, we can investigate implementing that in the client in the future, but this mechanism would have to fail the client once it was reached to maintain order.

[johnboy14]

Is it no possible to block the user or atleast allow the user to configure blocking like they do with the pubsub flow control mechanisms today? I'm having an issue where a high rate of messages just fills up the ArrayDeque and the app throws an OOM. I can count outstanding futures but I'd prefer the api to tell me to slow down or force me.