Skip to content

libsql-experimental panics under concurrent load #110

Description

@SanketJawali

Bug Report: libsql-experimental panics under concurrent SQLAlchemy usage

Project used to recreate the bug: https://github.com/SanketJawali/libsql-bug-report

Summary

Under concurrent load, a minimal FastAPI + SQLAlchemy application using sqlalchemy-libsql and libsql-experimental consistently triggers Rust panics inside the Python extension.

The panic originates from Connection::rollback() and Connection::cursor() where Option::unwrap() is called on a None value.

This was originally observed in a larger production project, but has since been reproduced in a minimal application with a single table and a simple SELECT endpoint.


Environment

  • Python: 3.13.7
  • FastAPI
  • SQLAlchemy
  • sqlalchemy-libsql 0.1.11
  • libsql-experimental 0.0.55
  • Remote Turso database
  • Linux

(Complete package versions can be provided if needed.)


Minimal application (https://github.com/SanketJawali/libsql-bug-report)

The reproduction consists of:

  • FastAPI
  • SQLAlchemy ORM
  • sqlalchemy-libsql
  • Remote Turso database
  • One table (users)
  • 26 rows
  • One endpoint:
@app.get("/users")
def fetch_users(db: Session = Depends(get_db)):
    return db.execute(select(User).order_by(User.id)).scalars().all()

Each request uses the standard SQLAlchemy session dependency pattern:

def get_db():
    db = SessionLocal()
    try:
        yield db
    finally:
        db.close()

No middleware, authentication, caching, Redis, or application-specific logic is involved.


Reproduction

Using wrk:

wrk -t2 -c5 -d15s http://localhost:8000/users

Even this relatively small amount of concurrency consistently reproduces the issue.


Observed behavior

The server begins returning 500 responses.

(Complete logs can be viewed here: https://github.com/SanketJawali/libsql-bug-report/blob/main/benchmark.md)

Rust panics occur repeatedly:

thread '<unnamed>' panicked at src/lib.rs:220:65:
called `Option::unwrap()` on a `None` value

and

thread '<unnamed>' panicked at src/lib.rs:260:42:
called `Option::unwrap()` on a `None` value

The Python traceback shows these occurring during:

  • Connection.cursor()
  • Connection.rollback()

The panic propagates to Python as:

pyo3_runtime.PanicException:
called `Option::unwrap()` on a `None` value

In other cases SQLAlchemy later reports:

AssertionError
self.dbapi_connection is not None

when attempting to create a cursor, suggesting that the underlying DBAPI connection has become invalid.


Investigation performed

1. Raw libsql client

I implemented another endpoint using the raw libsql client directly (without SQLAlchemy).

Result:

  • No panic observed.
  • Requests completed successfully.

2. SQLAlchemy + NullPool

Configured SQLAlchemy with NullPool.

Result:

  • No Rust panic observed.
  • Requests eventually timed out due to connection creation overhead.
  • The original panic did not occur.

3. pool_reset_on_return=None

Disabled SQLAlchemy's automatic pool reset.

Result:

  • Panics still occur.
  • Therefore the issue does not appear to be caused solely by SQLAlchemy's pool reset behavior.

4. Explicit rollback

Explicitly calling:

db.rollback()

after the query changes the failure mode.

Instead of only panicking during cleanup, panics also occur while creating a cursor for subsequent requests.


5. Connection checkout/checkin logging

Connections are checked out and returned normally until the failure occurs.

The failure does not appear to be isolated to one specific connection object.


Source inspection

Looking at the published Rust source for libsql-experimental, both panic locations correspond to unconditional unwrap() calls on the internal connection:

cursor():

self.conn.borrow().as_ref().unwrap()

rollback():

self_.conn.borrow().as_ref().unwrap()

While investigating, I noticed that the panic locations (src/lib.rs line 220 and 260) correspond to unwrap() calls in cursor() and rollback(). I don't know whether that is the root cause, and can't confirm it myself as i don't know rust or the internal working of the sdk.


Expected behavior: A DBAPI-level exception (e.g. OperationalError) rather than an unhandled Rust panic crossing into Python.


Additional context

This issue was originally discovered while load testing a larger FastAPI application.

To eliminate application-specific factors, I created this minimal reproduction project, which reproduces the same panic with only a simple SELECT endpoint.

I cannot determine whether the root cause is inside libsql-experimental itself or an interaction with sqlalchemy-libsql, but the panic consistently originates inside the Rust extension and appears reproducible with a minimal example.

If it would be helpful, I can also provide the complete minimal reproduction project.


Note

I found this potential bug and have done my best to isolate and find out what the cause is. I have used AI to fill in the gaps i have in my knowledge and isolate this potential bug. I tried my best to understand the nature of this bug, but i didn't understand it 100%. I hope this helped in some way.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions