fix(ttl-cache): concurrent calls during the first fetch get undefined (@boergeson)

[boergeson]

Description

Concurrent calls to a cacheWithTTL function while the first fetch is still running each got undefined back, since the cache was marked fresh before the data arrived. Now the first call starts the fetch and later calls wait on the same promise, so there is only one fetch and everyone gets the value.

A failed fetch still marks the cache fresh, so the stale value keeps being served until the ttl passes. That part is unchanged on purpose, the cache also throttles the database.

Closes #8363

[fehmer]

hi @boergeson ,

thank you for your contribution.

Fetching only once if expired and waiting on the fetch on concurrent calls is fine.

I don't think the behavior in case of a fetch error is correct. The cache also acts like a throttel and if there is a problem with the database now each request will re-try and add additional load on the database.

Currently the cache is only used for PSA and the plan was to use it for leaderboard as well, both are non-critical and I think it is better to serve stale data in case of an error instead of an error or no data. Therefor I would keep the current behavior.

@Miodec what is your take on this?

[boergeson]

Makes sense, didnt think of it as a throttle. Reworked it: a failed fetch still marks the cache fresh and the stale value is served until the ttl passes, only the in-flight dedup is left. Updated the test to check the stale value is kept after a failure.