Netopeer2 becomes unresponsive during concurrent edit-config requests when the system clock jumps

[albeliakov]

Summary

Netopeer2 becomes unresponsive when the system clock changes abruptly while multiple NETCONF sessions are sending edit-config requests.

The failure starts with an Invalid argument error in a sysrepo condition-variable wait. Subsequent RPC requests time out, and a process restart is required to restore service.

In our production environment, switching the time synchronization source can cause system clock jumps. This issue therefore results in loss of NETCONF management access during normal operating conditions.

Environment

• sysrepo: 3.3.10
• NETCONF server: Netopeer2
• Concurrent NETCONF sessions in the reproducing test: 20
• Configuration changes are handled by sysrepo change subscribers.

Reproduction scenario

1. Start Netopeer2 and the applications subscribed to configuration changes.
2. Open 20 NETCONF sessions.
3. Continuously send edit-config requests that trigger change subscribers.
4. Abruptly adjust the system clock while requests are being processed.
5. Check whether subsequent NETCONF requests complete.

The failure is timing-dependent. The following results were observed in our test runs:

| NETCONF sessions | Active edit-config requests | System clock jump | Result                      |
| ---------------- | --------------------------- | ----------------- | --------------------------- |
| 20               | Yes                         | No                | No failures observed        |
| 20               | No                          | Yes               | No failures observed        |
| 1                | Yes                         | Yes               | No failures observed        |
| 20               | Yes                         | Yes               | Server becomes unresponsive |

These results suggest that concurrent requests increase the probability of triggering the failure. They do not establish that multiple sessions are strictly required.

Expected behavior

A system clock jump should not leave the server permanently unresponsive.

An affected request may fail with a timeout or another appropriate error, but transaction cleanup should complete, and subsequent requests should remain functional.

Actual behavior

An error occurs while waiting for subscription processing. Subsequent RPC requests time out, and the server does not recover without a process restart.

Relevant logs

Processing subscription events failed (Internal error).
Internal error (/builds/sysrepo/src/src/common.c:3892).
Waiting on a conditional variable failed (_sr_shmsub_notify_wait_wr: Invalid argument).
Failed to send an RPC (Internal error).
EV ORIGIN: SHM event "rpc" ID 1468 processing timed out.

Suspected cause

The failure appears to originate in sysrepo's futex-based condition-variable implementation:

[sr_cond_futex.c in sysrepo v3.3.10](https://github.com/sysrepo/sysrepo/blob/v3.3.10/src/sr_cond/sr_cond_futex.c), function sr_cond_wait_.

Our working hypothesis is:

1. A request is waiting for a subscriber to finish processing an event.
2. A system clock jump affects timeout calculation around this wait.
3. The wait fails with EINVAL.
4. The error path leaves shared synchronization state inconsistent, possibly leaving a lock unreleased.
5. Subsequent operations cannot make progress.

The exact relationship between the clock jump, the arguments passed to futex, and the persistent blocking still needs confirmation. The logs alone do not establish which lock remains held.

Questions

• Is this a known issue, and is there an existing fix or recommended backport for sysrepo 3.3.10?
• Could you help confirm whether the relevant wait and error-handling paths can leave shared synchronization state inconsistent?
• Should this issue be transferred to the sysrepo repository? We are reporting it here because the externally visible failure is loss of Netopeer2 availability.

[michalvasko]

Are you able to collect more information about the error, which function exactly (I assume syscall(SYS_futex, ...)) generates the error and what are its parameters? It is difficult to reproduce and I think simply ignoring this error will do but I was not able to find any information about the cause of EINVAL because of a clock jump. Normally, these functions work fine and even if you use a timeout that has been elapsed, the behavior is similar to a non-blocking operation.