imaplib.Idler raises IndexError when another command's tagged response arrives during IDLE

Bug report

imaplib.IMAP4.idle() iteration raises an unhandled IndexError: pop from empty list when a tagged response arrives while the client is iterating over an IDLE session.

In Idler._pop() (Lib/imaplib.py, line 2001), after calling _get_response(timeout) the method unconditionally executes imap._idle_responses.pop(0). However, _get_response() handles a tagged response by storing it in self.tagged_commands only — it appends nothing to the _idle_responses queue. The queue is therefore still empty and pop(0) raises IndexError, which escapes into user code iterating the Idler.

This happens when a tagged reply for an earlier pipelined command arrives after the IDLE continuation (+) has been consumed and before the next untagged response. RFC 3501 section 5.5 permits command pipelining for IMAP4rev1. The tagged response below belongs to NOOP; the IDLE command's tagged completion still follows DONE as required by RFC 2177 section 3.

Reproducer (run with a CPython build from this checkout, e.g. PCbuild/amd64/python_d.exe repro.py):

import socketserver
import threading
import traceback

import imaplib


class IdleTaggedHandler(socketserver.StreamRequestHandler):
    def _send_line(self, line):
        if isinstance(line, str):
            line = line.encode('ascii')
        self.wfile.write(line + b'\r\n')

    def _send_tagged(self, tag, code, text):
        self._send_line(tag + b' ' + code.encode() + b' ' + text.encode())

    def handle(self):
        self._send_line('* OK [CAPABILITY IMAP4rev1 IDLE] ready')
        pending_noop = None
        while True:
            line = self.rfile.readline()
            if not line:
                return
            tag, _, rest = line.partition(b' ')
            cmd = rest.strip().split(b' ')[0].upper()
            if cmd == b'LOGIN':
                self._send_line(b'* CAPABILITY IMAP4rev1 IDLE')
                self._send_tagged(tag, 'OK', 'LOGIN completed')
            elif cmd == b'NOOP':
                pending_noop = tag
            elif cmd == b'CAPABILITY':
                self._send_line(b'* CAPABILITY IMAP4rev1 IDLE')
                self._send_tagged(tag, 'OK', 'CAPABILITY completed')
            elif cmd == b'IDLE':
                self._send_line(b'+ idling')
                self._send_tagged(pending_noop, 'OK', 'NOOP completed')
                self._send_line(b'* 1 EXISTS')
                line = self.rfile.readline()            # wait for DONE
                if line == b'DONE\r\n':
                    self._send_tagged(tag, 'OK', 'Idle completed')
                else:
                    self._send_tagged(tag, 'BAD', 'Expected DONE')
            else:
                self._send_tagged(tag, 'OK', 'done')


server = socketserver.TCPServer(('127.0.0.1', 0), IdleTaggedHandler)
threading.Thread(target=server.serve_forever, daemon=True).start()

client = imaplib.IMAP4(*server.server_address)
client.login('user', 'pass')
# Leave NOOP's tagged completion pending while starting IDLE.
noop_tag = client._command('NOOP')
try:
    with client.idle() as idler:
        print(next(idler))           # <-- IndexError raised here
    print(client._get_tagged_response(noop_tag))
except Exception:
    traceback.print_exc()

Output on current main (d3ada45):

Traceback (most recent call last):
  File "repro.py", line 56, in <module>
    print(next(idler))
  File "Lib/imaplib.py", line 2014, in __next__
    typ, data = self._pop(timeout)
  File "Lib/imaplib.py", line 2001, in _pop
    resp = imap._idle_responses.pop(0)
IndexError: pop from empty list

Expected behavior: _get_response() should retain the tagged reply in tagged_commands, then IDLE iteration should continue until it can return the next untagged response. Any timeout must continue to use the original deadline.

Reproduced on CPython 3.16.0a1+dev, revision d3ada4509ea, on Windows x64. The reproduction uses the private _command() method to leave a tagged command response pending; the regular public command methods wait for their tagged response before returning.

CPython versions tested on:

3.16 (CPython 3.16.0a1+dev, revision d3ada4509ea)

Operating systems tested on:

Windows (x64)

Linked PRs

gh-159583: Fix IndexError in imaplib IDLE iteration on a tagged response #159586

@foresto

cc @foresto I think you implemented the idler

This happens when a tagged reply for an earlier pipelined command arrives after the IDLE continuation (+) has been consumed and before the next untagged response.

What server sends a tagged response to some unrelated earlier command during the window after it has sent a command continuation request (+) but before it gets the requested continuation?

RFC 3501 and 9051 contain some language in section 2.2.1 (5th paragraph) that might be relevant, but that language could be interpreted at least two different ways. Is there a use case that would justify this server behavior? I haven't seen it from the IMAP servers that I use, and I'm not sure it's supposed to be allowed.

Edit: As I explain in a comment below, I am now fairly sure it is not allowed.

Thanks for asking. I encountered this IndexError in an internal application. The investigation suggested that a delayed server-side SEARCH completion might be involved, so I reproduced the scenario with a mock server that delays the SEARCH response.

The client uses only public imaplib APIs (IMAP4 with a socket timeout, login(), select(), search(), and idle()). The corresponding client/server exchange is:

C: A001 SEARCH ALL
    ... the server is still processing the search ...
[the 2-second socket timeout expires; search() raises TimeoutError]
C: A002 IDLE
S: + idling
S: * SEARCH 1 2 3
S: A001 OK SEARCH completed
C: DONE
S: A002 OK IDLE completed

The mock server delays the SEARCH completion, which is permitted by the RFC. It does not send a command while waiting for DONE, and it sends the tagged completion for IDLE only after DONE. The late A001 OK completes the earlier SEARCH. The client timeout does not cancel that server-side command, and imaplib leaves its tag pending. In the current imaplib implementation, this sequence triggers the IndexError in Idler._pop().

RFC 3501 section 2.2.1 describes this response ordering explicitly:

It is also possible for the server to send a completion response for some other command (if multiple commands are in progress), or untagged data. In either case, the command continuation request is still pending; the client takes the appropriate action for the response, and reads another response from the server.

Section 5.5 permits multiple commands to be in progress when there is no ambiguity. SEARCH does not affect IDLE, and their distinct tags identify the two completions. Here, the earlier SEARCH completion is received while the IDLE continuation is pending; both command lines have already been sent. RFC 2177 requires the tagged completion for IDLE after DONE; this exchange does that, while the other tag belongs to SEARCH.

Other clients also account for tagged or unexpected responses during IDLE. Thunderbird's IMAP parser has a dedicated inIdle branch: it processes the expected IDLE tag normally, and turns a different tag into a controlled fatal error and server alert. Angus Mail's processIdleResponse() dispatches each response and ends IDLE only when the tag matches the IDLE command.

Python clients show both defensive approaches. imapclient's idle_check() passes each received line to an untagged-response parser; a tagged line fails its protocol check and raises the library's ProtocolError (check). aioimaplib routes tagged lines to its pending-command handler, which raises the library's Abort exception for an unexpected tag.

RFC 3501 section 2.2.1 describes this response ordering explicitly:

It is also possible for the server to send a completion response for some other command (if multiple commands are in progress), or untagged data. In either case, the command continuation request is still pending; the client takes the appropriate action for the response, and reads another response from the server.

Pay close attention to the second sentence that you quoted. That paragraph is saying that the server can send a tagged completion response from some other command after receiving IDLE from the client, but BEFORE it sends + idling back to the client. That is, while the command continuation request is still pending.

This is more clear if you understand the protocol as a whole, rather than taking the paragraph above out of context. When the server sends a command continuation request response (which starts with +), it is in the middle of receiving a multi-line command from the client, indicating that it's ready to receive the next part. It would not make sense for it to send unrelated data (like tagged responses to an earlier command) in that state.

Consider also that the IDLE command (which is not the only command that triggers continuation requests from the server) corroborates this in RFC 2177:

as long as an IDLE command is active, the server is now free to send untagged EXISTS, EXPUNGE, and other messages at any time.

Note that tagged responses are not listed among the things the server can send during IDLE. (And in case you missed it, the IDLE command is active as soon as the server sends its continuation request, which starts with +)

So, I do not see a bug in imaplib here. This is why I asked what server behaves as you describe.

I encountered this IndexError in an internal application.

Ah, I see. I suggest you contact the author of that application, and let them know that they have a bug in their IMAP service.

By reading https://www.rfc-editor.org/info/rfc2177 and https://www.rfc-editor.org/info/rfc9051, and the transcript

C: A001 SEARCH ALL
    ... the server is still processing the search ...
[the 2-second socket timeout expires; search() raises TimeoutError]
C: A002 IDLE
S: + idling
S: * SEARCH 1 2 3
S: A001 OK SEARCH completed
C: DONE
S: A002 OK IDLE completed

I can't tell whether the above transcript is correct or not.

Now we have

The IDLE command remains active until the client responds to the continuation, and as long as an IDLE command is active, the server is now free to send untagged EXISTS, EXPUNGE, FETCH, and other responses at any time. If the server chooses to send unsolicited FETCH responses, they MUST include a UID FETCH item.

and (emphasis mine)

The IDLE command is terminated by the receipt of a "DONE" continuation from the client; such response satisfies the server's continuation request. At that point, the server MAY send any remaining queued untagged responses and then MUST immediately send the tagged response to the IDLE command and prepare to process other commands

Now https://www.rfc-editor.org/info/rfc9051/#section-5.5-2 says:

The exception is if an ambiguity would result because of a command that would affect the results of other commands. If the server detects a possible ambiguity, it MUST execute commands to completion in the order given by the client.

I don't know if this counts as such case though but I would advise digging more into the RFCs. I personally think that it would be ok to send unrelated info to the client when in IDLE state, if the client correctly process them, but I don't know if it's really allowed.

Just in case I wasn't clear earlier, I believe this is not a bug in imaplib, but a misunderstanding on the part of the submitter and a mistake in their (in-house) server implementation.

I recommend closing this issue and rejecting PR #159586.

Navigation Menu

Repository navigation

imaplib.Idler raises IndexError when another command's tagged response arrives during IDLE #159583

Bug report

CPython versions tested on:

Operating systems tested on:

Linked PRs

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

imaplib.Idler raises IndexError when another command's tagged response arrives during IDLE #159583

Description

Bug report

CPython versions tested on:

Operating systems tested on:

Linked PRs

Activity

picnixz commented on Oct 16, 2026

foresto commented on Oct 17, 2026

MrGreyfun commented on Oct 18, 2026

foresto commented on Oct 18, 2026

picnixz commented on Oct 18, 2026

foresto commented on Oct 18, 2026

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions