Data race between `BytesIO.__setstate__` and lock-free LOAD_ATTR method

[Naserume]

Bug report

Bug description:

In the free-threaded build, BytesIO.__setstate__() installs the instance __dict__ with a plain (non-atomic) store, which races the lock-free LOAD_ATTR method-cache check that reads the same instance-dict slot with an atomic acquire load.

bytesio_setstate_lock_held() sets self->dict directly,

cpython/Modules/_io/bytesio.c

Lines 1042 to 1059 in a74280e

	dict = PyTuple_GET_ITEM(state, 2);
	if (dict != Py_None) {
	if (!PyDict_Check(dict)) {
	PyErr_Format(PyExc_TypeError,
	"third item of state should be a dict, got a %.200s",
	Py_TYPE(dict)->tp_name);
	return NULL;
	}
	if (self->dict) {
	/* Alternatively, we could replace the internal dictionary
	completely. However, it seems more practical to just update it. */
	if (PyDict_Update(self->dict, dict) < 0)
	return NULL;
	}
	else {
	self->dict = Py_NewRef(dict);
	}
	}

but calling a method on the object (e.g. bio.read(...)) goes through a LOAD_ATTR method specialization that reads the instance-dict slot lock-free with an atomic acquire load.

cpython/Python/generated_cases.c.h

Lines 9055 to 9062 in a74280e

	// _CHECK_ATTR_METHOD_LAZY_DICT
	{
	uint16_t dictoffset = read_u16(&this_instr[4].cache);
	char *ptr = ((char *)PyStackRef_AsPyObjectBorrow(owner)) + MANAGED_DICT_OFFSET + dictoffset;
	PyObject *dict = FT_ATOMIC_LOAD_PTR_ACQUIRE(*(PyObject **)ptr);
	if (dict != NULL) {
	UPDATE_MISS_STATS(LOAD_ATTR);
	assert(_PyOpcode_Deopt[opcode] == (LOAD_ATTR));

__setstate__ holds the BytesIO critical section, but the LOAD_ATTR fast path does not take the object lock, so the plain store races the atomic load on the dict slot.

Reproducer:

import io
from threading import Thread, Barrier

shared = io.BytesIO(b'initial payload')
states = [(b'A' * 64, 0, {}), (b'B' * 128, 32, {}), (b'C' * 256, 128, {})]

N_W, N_R = 1, 8
barrier = Barrier(N_W + N_R)

def setter():
    barrier.wait()
    for i in range(20000):
        try:
            shared.__setstate__(states[i % len(states)])
        except Exception:
            pass

def reader():
    barrier.wait()
    for i in range(20000):
        try:
            shared.read(8)
        except Exception:
            pass

if __name__ == "__main__":
    threads = [Thread(target=setter) for _ in range(N_W)]
    threads += [Thread(target=reader) for _ in range(N_R)]
    for t in threads: t.start()
    for t in threads: t.join()

TSAN Report:

==================
WARNING: ThreadSanitizer: data race (pid=37676)
  Write of size 8 at 0x7fffb66706c8 by thread T2:
    #0 bytesio_setstate_lock_held /cpython/./Modules/_io/bytesio.c:968:24
    #1 bytesio_setstate /cpython/./Modules/_io/bytesio.c:980:11 
    #2 method_vectorcall_O /cpython/Objects/descrobject.c:476:24
    #3 _PyObject_VectorcallTstate /cpython/./Include/internal/pycore_call.h:144:11 
    #4 PyObject_Vectorcall /cpython/Objects/call.c:327:12 
    #5 _Py_VectorCallInstrumentation_StackRefSteal /cpython/Python/ceval.c:768:11 
    #6 _PyEval_EvalFrameDefault /cpython/Python/generated_cases.c.h:1906:35 
...

  Previous atomic read of size 8 at 0x7fffb66706c8 by thread T10:
    #0 _Py_atomic_load_ptr_acquire /cpython/./Include/cpython/pyatomic_gcc.h:557:18
    #1 _PyEval_EvalFrameDefault /cpython/Python/generated_cases.c.h:9149:34 
    #2 _PyEval_EvalFrame /cpython/./Include/internal/pycore_ceval.h:122:16 
...

SUMMARY: ThreadSanitizer: data race /cpython/./Modules/_io/bytesio.c:968:24 in bytesio_setstate_lock_held
==================

CPython versions tested on:

CPython main branch

Operating systems tested on:

Linux

Linked PRs

• gh-153290: Fix data race in BytesIO.__setstate__ installing __dict__ #153376

[bhuvi27]

I'd like to work on this.