Skip to content

macOS: durable tmux socket location and safe handling of unreachable long-lived servers #4

Closed

Description

@er0826-dex

Problem

A long-lived tmux server became unreachable while its process and agent panes were still alive. AgentDesk showed all sessions as inactive / “No tmux session running” and could not create new sessions.

Previously recorded incident

  • The server had been running for approximately 10 days and still owned 19 agent panes.
  • tmux ls returned server exited unexpectedly.
  • Approximately 75 leftover tmux -C attach-session client processes were reported during recovery.
  • The socket location was /private/tmp/tmux-<uid>/default.
  • Recovery was performed manually; the agents were subsequently resumed from persisted conversation state.

The earlier diagnosis attributed this to temporary-directory cleanup affecting the socket. That is a suspected mechanism, not independently proven by the retained evidence. Verify relevant macOS cleanup behavior and tmux failure modes before claiming a root cause. Killing the server is not established as the only recovery option.

Proposed direction

In the inspected checkout, AgentDesk generally invokes tmux without an explicit socket path or a default TMUX_TMPDIR. Consider a durable, application-owned socket parent such as $HOME/.agent-desk/tmux, with restrictive permissions, while respecting an explicitly configured TMUX_TMPDIR.

Relevant paths:

  • cmd/agent-desk/main.go: initialization before subcommand dispatch.
  • internal/tmux/tmux.go: tmux command execution.
  • internal/tmux/controlpipe.go and pipemanager.go: failed-client cleanup and reconnect lifecycle.

Central initialization may avoid editing individual call sites, but verify all command paths, environment overrides and inherited TMUX behavior.

Safety requirements

  • Changing socket selection can make existing sessions appear missing by directing clients to another server. Define an explicit compatibility/migration strategy; do not silently strand live sessions.
  • Do not automatically kill servers, delete socket directories, or recreate live sessions.
  • Preserve explicit operator configuration and account for launches inside an existing tmux session.
  • Surface directory/permission/environment initialization errors; do not silently fall back while implying the durable location is active.
  • Keep defensive reconnect/client-cleanup changes separately scoped if they require a distinct fix.

Acceptance criteria

  • Reproduce socket-loss/unreachable-server behavior using an isolated tmux server, not real agent sessions.
  • Verify durable socket selection, restrictive permissions, explicit override handling, initialization failure reporting and consistent behavior across TUI/subcommands.
  • Provide a safe transition for users with an already-running server at the old location, with documented rollback.
  • Verify failed connection/reconnection attempts do not accumulate unreaped clients; file a focused follow-up if needed.
  • Run relevant tests and builds and document installation verification against the actual executable on PATH.

This is separate from the fourth-concurrent-TUI freeze report; neither incident establishes the cause of the other.

Activity

  1. millwright-software commented on Nov 11, 2026

    @millwright-software
    Owner

    Fixed in v1.0.28, together with #3. The two incidents have the same cause: the tmux server ran out of open files, and the socket was fine. The full evidence is in documentation/worklog/2026-11-11-tmux-file-limit.md.

    Reproduced on an isolated server. Capped at 64 files, the 20th control client tipped it:

    • tmux list-sessions returned server exited unexpectedly.
    • 18 clients and every pane stayed up.

    That matches this incident. A server that agent-desk started had a limit of 256, because Go restores the original soft limit in exec'd children. At 3 files per control client, 3–4 windows on 19 sessions fills that.

    The socket was not removed. macOS's /usr/libexec/tmp_cleaner deletes only regular files (-type f) and empty directories under /tmp, and a socket is neither. A tmux upgrade under a running server can produce the same error; that wasn't checked on the affected machine.

    The 75 clients fit the expected fan-out, 57–76 for 3–4 windows on 19 sessions. A window killed outright takes its clients with it (tested: 3 → 0 within 1.5 s). Real leak paths in the pipe code are fixed:

    • watchPipe dropped abandoned pipes without Close(). A client whose reader had died stayed attached, holding 3 server files.
    • watchPipe marked the session as reconnecting and then called Connect, which returned early. Every reconnect was a silent no-op until the 20 s sweep ran. TestWatchPipeReconnects fails on the old logic and passes on the new.
    • A client that died in the handshake was never reaped.

    The durable socket move was not done. The evidence doesn't support it, and it would hide sessions on a server already running at the old path. It's worth revisiting if this recurs on a server whose limit is known to be high.

    Recovery. Killing the server is not required: closing a window frees its files, and the server answers again at once. The tmux, session and ui test suites now run against a private tmux server instead of the user's default one.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions