Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 1 addition & 3 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -54,9 +54,7 @@ modules/<name>/<name>/
**Middleware pipeline** (Starlette `add_middleware` is LIFO — last added runs first). Execution order on a request:
`CorrelationId → RequestLogging → SecurityHeaders → Session → <module middleware> → Tenant (opt-in) → Locale → InertiaLayoutData → app`. When two modules add middleware at the same dependency tier, the module that sorts **later** wraps outermost. Use `depends_on` to express relative order — don't rely on names.

**Database**: per-module `Base` via `create_module_base("<name>")`. Provider auto-detected from `SM_DATABASE_URL`:
- **Postgres** → one schema per module (`orders.<table>`).
- **SQLite** → single schema; `__tablename__` must be prefixed with the module name (`orders_order`).
**Database**: per-module `Base` via `create_module_base("<name>")`. Every module owns its own `MetaData` (so Alembic autogenerate can attribute tables to a module), but all tables live in the host's single schema. `__tablename__` must be prefixed with the module name to avoid collisions (`orders_order`). Postgres and SQLite share the same layout.

Standard mixins in `simple_module_db.mixins`: `AuditMixin`, `SoftDeleteMixin` (bypass with `stmt.execution_options(include_deleted=True)`), `MultiTenantMixin`, `VersionedMixin`. The per-request session (`get_db`) auto-commits **only if** there are pending writes (via `after_flush` listener); otherwise rollback. Service code should **not** call `session.commit()` — flush if you need DB-assigned values.

Expand Down
10 changes: 3 additions & 7 deletions docs/database/models.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,10 +23,7 @@ class Order(Base, AuditMixin, table=True):

### Table naming

- **Postgres.** `create_module_base("orders")` gives the class a per-module schema. The `__tablename__` can be just `order`, and the fully-qualified name is `orders.order`. The prefix `orders_` is redundant but harmless.
- **SQLite.** One schema, so `__tablename__` must be prefixed with the module name to avoid collisions: `orders_order`.

**Convention: always prefix the table name with the module name.** This makes migrations and DB dumps readable in both providers and avoids a "works on my machine" footgun when swapping between them.
All modules share the host's single schema, so `__tablename__` must be prefixed with the module name to avoid collisions: `orders_order`, `users_user`, etc. `create_module_base` does not enforce the prefix — it's a convention the framework relies on.

### Primary keys

Expand All @@ -50,8 +47,7 @@ class OrderLine(Base, table=True):
__tablename__ = "orders_order_line"

id: int | None = Field(default=None, primary_key=True)
order_id: int = Field(foreign_key="orders.order.id") # Postgres
# order_id: int = Field(foreign_key="orders_order.id") # SQLite
order_id: int = Field(foreign_key="orders_order.id")
quantity: int

order: "Order" = Relationship(back_populates="lines")
Expand Down Expand Up @@ -165,7 +161,7 @@ Do not re-enable these rules in module-local configs. Real bugs caused by wrong

## Next steps

- [Per-module Base](/database/per-module-base) — how provider detection and schema isolation works.
- [Per-module Base](/database/per-module-base) — how `create_module_base` and `build_module_metadata` work.
- [Mixins](/database/mixins) — `AuditMixin`, `SoftDeleteMixin`, `MultiTenantMixin`, `VersionedMixin`.
- [Session lifecycle](/database/sessions) — the `get_db` dependency and why you don't call `commit()`.
- [Migrations](/database/migrations) — Alembic autogenerate and per-module branch labels.
29 changes: 8 additions & 21 deletions docs/database/per-module-base.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,25 +13,13 @@ class Order(Base, table=True):
...
```

`create_module_base` returns a SQLModel base that's bound to a private `MetaData` object. This isolation is what lets modules live side-by-side without their tables trampling each other's Alembic autogenerate output.
`create_module_base` returns a SQLModel base bound to a private `MetaData` object. This isolation is what lets Alembic autogenerate attribute each table to a specific module and what makes `build_module_metadata()` able to assemble the combined target metadata.

## Provider auto-detection
## Single shared schema

The function inspects `SM_DATABASE_URL` at import time and picks the right strategy:
All modules — whether on Postgres or SQLite — live in the host's single schema. There is no per-module schema policy. `__tablename__` must be prefixed with the module name to avoid collisions (`orders_order`, `users_user`). `create_module_base` doesn't enforce the prefix; it's a convention the framework relies on.

- **PostgreSQL** → the Base sets `__table_args__ = {"schema": "<name>"}`. Tables live at `orders.<table>`. Each module effectively owns a namespace; one `DROP SCHEMA orders CASCADE` can cleanly uninstall a module.
- **SQLite** → no schema (SQLite has one). The `__tablename__` must still be prefixed with the module name to avoid collisions. `create_module_base` does not enforce the prefix — it's a convention.

You can override detection explicitly for tests or special cases:

```python
from simple_module_db.base import create_module_base
from simple_module_db.types import DatabaseProvider

Base = create_module_base("orders", provider=DatabaseProvider.SQLITE)
```

Production code should let auto-detection do its thing.
The same migrations apply to Postgres and SQLite. There is no provider branching in model metadata.

## The `build_module_metadata()` function

Expand Down Expand Up @@ -59,9 +47,9 @@ If you write a one-off host-level table that autogenerate shouldn't track, exten

## Naming rules

- Module name must match `ModuleMeta.name.lower()`. The framework caches the Base by module name; a mismatch causes silent schema drift.
- Module name must match `ModuleMeta.name.lower()`. The framework caches the Base by module name; a mismatch causes silent metadata drift.
- Module names should be identifiers: `[a-z][a-z0-9_]*`. Hyphens break SQL identifier parsing on some providers.
- Don't rename a module after it ships without migrating data. Both the schema name (Postgres) and the table prefix (SQLite) are durable.
- Don't rename a module after it ships without migrating data. The table prefix is durable across deployments.

## Tables across modules

Expand All @@ -74,13 +62,12 @@ from orders.models import Order
class Invoice(Base, table=True):
__tablename__ = "invoices_invoice"
id: int | None = Field(default=None, primary_key=True)
order_id: int = Field(foreign_key="orders.order.id")
order_id: int = Field(foreign_key="orders_order.id")
```

Caveats:

- Add `depends_on=["Orders"]` in `InvoicesModule.meta` — modules are loaded in topological order; without `depends_on`, `orders.models` might not be imported when `invoices.models` runs.
- Autogenerate handles cross-schema FKs on Postgres natively.
- Uninstalling `Orders` while `Invoices` still references it produces a DB error — cross-module FKs are a commitment.

If you can avoid a hard FK (store `order_id: int` without the constraint), module lifecycles stay more independent. Prefer application-level validation for loose coupling.
Expand All @@ -94,7 +81,7 @@ from simple_module_db.base import build_module_metadata

meta = build_module_metadata()
for t in meta.sorted_tables:
print(t.schema or "(no schema)", t.name)
print(t.name)
```

This is also what the boot-time `SM011` check uses — it compares this set against the Alembic history to detect tables that exist in code but not in any migration.
9 changes: 3 additions & 6 deletions docs/framework-conventions.md
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,7 @@ class OrdersModule(ModuleBase):
)
```

- `name` is unique across the app — it's the schema name for PostgreSQL, the SQLite table prefix, the Inertia component namespace, and the diagnostic reporter name.
- `name` is unique across the app — it's the table-name prefix, the Inertia component namespace, and the diagnostic reporter name.
- `depends_on` expresses a hard ordering requirement; the framework topo-sorts modules and invokes lifecycle hooks in that order.
- Missing or invalid `meta` fails the boot in production (strict discovery) and logs a `SM001` warning in development.

Expand Down Expand Up @@ -159,13 +159,10 @@ class OrderCreate(SQLModel):
```python
from simple_module_db.base import create_module_base

Base = create_module_base("orders") # provider auto-detected from SM_DATABASE_URL
Base = create_module_base("orders")
```

- **PostgreSQL**: the module gets its own schema (`orders`). Tables live at `orders.<table>`.
- **SQLite**: single schema. Prefix `__tablename__` with the module name to avoid collisions (`orders_order`).

You can pin the provider for tests (`provider=DatabaseProvider.SQLITE`), but code that ships should let auto-detection handle it.
`create_module_base` returns a SQLModel base bound to its own `MetaData`, but all modules share the host's single schema (same layout on Postgres and SQLite). Prefix `__tablename__` with the module name to avoid collisions (`orders_order`).

### Mixins

Expand Down
4 changes: 2 additions & 2 deletions docs/guide/first-module.md
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,7 @@ class Order(Base, AuditMixin, SoftDeleteMixin, table=True):

- `AuditMixin` adds `created_at`, `updated_at`, `created_by`, `updated_by` — populated automatically from the request user.
- `SoftDeleteMixin` replaces `DELETE` with `is_deleted=true`. `SELECT` filters them out by default; pass `include_deleted=True` to bypass.
- `__tablename__` must be prefixed with the module name under SQLite. On Postgres, the per-module Base puts tables in an `orders` schema automatically, so the prefix is redundant but harmless.
- `__tablename__` must be prefixed with the module name (`orders_order`) so it doesn't collide with other modules' tables — every module shares the host's single schema on both Postgres and SQLite.

## 3. Update the DTOs

Expand Down Expand Up @@ -75,7 +75,7 @@ uv run alembic revision --autogenerate -m "add orders tables"

Open `migrations/versions/XXXX_add_orders_tables.py` and eyeball it:

- It should create the `orders` schema (Postgres) or the `orders_order` table (SQLite).
- It should create the `orders_order` table.
- Add `branch_labels = ("orders",)` to the revision so you can later `alembic downgrade orders@base` to roll the module back to empty without touching other modules.

Apply:
Expand Down
4 changes: 2 additions & 2 deletions docs/guide/quickstart.md
Original file line number Diff line number Diff line change
Expand Up @@ -76,7 +76,7 @@ uv run alembic revision --autogenerate -m "add orders tables"
make migrate
```

Alembic's autogenerate picks up the new `orders` schema (Postgres) or the `orders_*` tables (SQLite) and writes `migrations/versions/XXXX_add_orders_tables.py`. Add `branch_labels = ("orders",)` to that revision so you can later `alembic downgrade orders@base` to roll the module back in isolation.
Alembic's autogenerate picks up the new `orders_*` tables and writes `migrations/versions/XXXX_add_orders_tables.py`. Add `branch_labels = ("orders",)` to that revision so you can later `alembic downgrade orders@base` to roll the module back in isolation.

## 7. Hit the module

Expand Down Expand Up @@ -107,7 +107,7 @@ uv run pytest modules/orders/tests/test_orders.py -v
- **Routes** — `register_routes(api_router, view_router)` attached the `orders` routers at `/api/orders` and `/orders`.
- **Menu** — `register_menu_items` pushed an entry onto `MenuRegistry`; the Inertia shared-props middleware serialized it into `menus.sidebar` for every authenticated request.
- **Frontend** — `modules.generated.ts` (rebuilt by `make gen-pages`) maps `"Orders/Browse"` to `modules/orders/orders/pages/Browse.tsx`. Vite resolves and HMR-watches that file.
- **Database** — `create_module_base("orders")` namespaced the `Order` table under a Postgres `orders` schema (or the `orders_order` table name under SQLite).
- **Database** — `create_module_base("orders")` gave the module its own SQLModel `MetaData` so Alembic can attribute the `orders_order` table to it.

## Next steps

Expand Down
2 changes: 1 addition & 1 deletion framework/db/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ pip install simple_module_db

## What it provides

- `create_module_base("<module_name>")` — a module-scoped declarative `Base`. PostgreSQL maps it to its own schema; SQLite namespaces via table-name prefix.
- `create_module_base("<module_name>")` — a module-scoped declarative `Base` with its own `MetaData`. All modules share the host's single schema, so `__tablename__` should be prefixed with the module name (e.g. `users_user`) to avoid collisions.
- Per-request async session (`get_db`) with an auto-commit-on-flush hook — `after_flush` commits if there are pending writes, rolls back otherwise.
- Mixins in `simple_module_db.mixins`: `AuditMixin` (created_at/updated_at), `SoftDeleteMixin` (auto-filtered unless `stmt.execution_options(include_deleted=True)`), `MultiTenantMixin`, `VersionedMixin`.
- `DatabaseState` container used by the framework to avoid global mutable state.
Expand Down
4 changes: 4 additions & 0 deletions framework/db/pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,10 @@ dependencies = [
"aiosqlite>=0.20",
"alembic>=1.14",
"asyncpg>=0.30",
# psycopg2 is the *sync* Postgres driver Alembic uses for migrations.
# The async app code goes through asyncpg; migration env.py rewrites the
# URL to +psycopg2 because Alembic's online migrations are sync-only.
"psycopg2-binary>=2.9",
"simple_module_core==0.0.17",
"sqlalchemy[asyncio]>=2.0",
"sqlmodel>=0.0.22",
Expand Down
2 changes: 1 addition & 1 deletion framework/db/simple_module_db/__init__.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
"""SimpleModule DB - SQLAlchemy async support with per-module schema isolation."""
"""SimpleModule DB — async SQLAlchemy/SQLModel runtime shared by every module."""

from simple_module_db.audit import AuditRecord
from simple_module_db.base import create_module_base
Expand Down
95 changes: 22 additions & 73 deletions framework/db/simple_module_db/base.py
Original file line number Diff line number Diff line change
@@ -1,15 +1,16 @@
"""Per-module SQLModel base with schema isolation."""
"""Per-module SQLModel base.

from __future__ import annotations
Every module owns its own :class:`sqlalchemy.MetaData` so Alembic autogenerate
can attribute tables to a module, but all tables live in the host's single
``public`` schema. Modules prefix ``__tablename__`` with the module name to
avoid collisions (e.g. ``users_user``).
"""

import os
from __future__ import annotations

from simple_module_core.dotenv import env_bool
from sqlalchemy import MetaData
from sqlmodel import SQLModel

from simple_module_db.provider import DatabaseProvider, detect_provider

# Convention-based naming for constraints (helps Alembic)
_naming_convention = {
"ix": "ix_%(column_0_label)s",
Expand All @@ -19,7 +20,6 @@
"pk": "pk_%(table_name)s",
}

# Cache created bases to avoid recreating for the same module
_base_cache: dict[str, type[SQLModel]] = {}

# Track all module bases for Alembic discovery. Module-level *mutable* list
Expand All @@ -30,77 +30,27 @@


def _register_base(base: type[SQLModel]) -> None:
"""Append ``base`` to ``all_module_bases`` iff not already present.

Guards against the list growing under repeated imports (test suites,
reloaders, plugin discovery) without changing the public type.
"""
"""Append ``base`` to ``all_module_bases`` iff not already present."""
if base not in all_module_bases:
all_module_bases.append(base)


def _default_schema_policy() -> DatabaseProvider:
"""Resolve the schema layout to register module tables under.
def create_module_base(module_name: str) -> type[SQLModel]:
"""Create a SQLModel abstract base with its own ``MetaData`` for a module.

The :class:`DatabaseProvider` enum doubles as a *schema-layout*
selector here — ``POSTGRESQL`` means "give every module its own
schema (``orders.<table>``)", ``SQLITE`` means "shared public schema,
name-prefixed tables (``orders_<table>``)". The conflation is
deliberate so existing call sites keep working, but conceptually this
is "schema policy", not "what DB are we connecting to": you can run
Postgres with ``SM_SCHEMA_PER_MODULE=false`` to keep a flat layout.
All modules share the host's single schema. Concrete tables should prefix
``__tablename__`` with ``module_name`` (e.g. ``users_user``) so names don't
collide. The per-module ``MetaData`` is what lets Alembic autogenerate
attribute each table to its module and what makes ``build_module_metadata``
able to assemble the combined target metadata.

Resolution order:
1. ``SM_SCHEMA_PER_MODULE`` (authoritative when set, decoupled from URL).
2. ``SM_DATABASE_URL`` (legacy fallback so deployments that haven't
migrated to the explicit knob keep working).
3. ``SQLITE`` (shared schema, the safe default).
"""
explicit = os.environ.get("SM_SCHEMA_PER_MODULE")
if explicit is not None:
return (
DatabaseProvider.POSTGRESQL
if env_bool("SM_SCHEMA_PER_MODULE")
else DatabaseProvider.SQLITE
)

url = os.environ.get("SM_DATABASE_URL", "")
if url:
return detect_provider(url)
return DatabaseProvider.SQLITE


def create_module_base(
module_name: str,
provider: DatabaseProvider | None = None,
) -> type[SQLModel]:
"""Create a SQLModel abstract base with schema isolation for a module.

- PostgreSQL: uses a dedicated schema (e.g., ``products``)
- SQLite: single schema; modules are expected to prefix ``__tablename__``
with the module name to avoid collisions (e.g., ``products_product``)

The provider defaults to whatever ``SM_DATABASE_URL`` indicates, so
module models work in both dev (SQLite) and prod (PostgreSQL) without
code changes. Pass ``provider=`` explicitly in tests that need to pin it.

Returns a cached base if already created for this module+provider. The
returned class is a ``SQLModel`` subclass with a per-module ``MetaData``;
concrete table classes declare ``table=True`` and inherit from it.
Returns a cached base on repeat calls for the same module name.
"""
if provider is None:
provider = _default_schema_policy()

cache_key = f"{module_name}:{provider}"
if cache_key in _base_cache:
return _base_cache[cache_key]

schema_name = module_name.lower()
module_name = module_name.lower()
if module_name in _base_cache:
return _base_cache[module_name]

if provider == DatabaseProvider.POSTGRESQL:
mod_metadata = MetaData(schema=schema_name, naming_convention=_naming_convention)
else:
mod_metadata = MetaData(naming_convention=_naming_convention)
mod_metadata = MetaData(naming_convention=_naming_convention)

# Use type() to create the class, avoiding class body scoping issues
ModuleBase = type( # noqa: N806
Expand All @@ -112,9 +62,8 @@ def create_module_base(
},
)

# Store module name for reference
ModuleBase.__module_name__ = schema_name # type: ignore[attr-defined]
ModuleBase.__module_name__ = module_name # type: ignore[attr-defined]

_base_cache[cache_key] = ModuleBase
_base_cache[module_name] = ModuleBase
_register_base(ModuleBase)
return ModuleBase
2 changes: 1 addition & 1 deletion framework/db/simple_module_db/migrations.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
"""Helpers for Alembic integration with module-based schemas.
"""Helpers for Alembic integration with module-based metadata.

A host's ``migrations/env.py`` should call :func:`build_module_metadata` to
obtain the combined ``target_metadata`` for autogenerate, and
Expand Down
3 changes: 1 addition & 2 deletions framework/db/tests/_audit_models.py
Original file line number Diff line number Diff line change
Expand Up @@ -10,10 +10,9 @@

from simple_module_db.base import create_module_base
from simple_module_db.mixins import AuditMixin, SoftDeleteMixin
from simple_module_db.provider import DatabaseProvider
from sqlmodel import Field

AuditBase = create_module_base("test_audit", provider=DatabaseProvider.SQLITE)
AuditBase = create_module_base("test_audit")


class AuditTestItem(AuditBase, AuditMixin, table=True): # type: ignore[call-arg] # ty: ignore[unsupported-base]
Expand Down
3 changes: 1 addition & 2 deletions framework/db/tests/_models.py
Original file line number Diff line number Diff line change
Expand Up @@ -10,10 +10,9 @@

from simple_module_db.base import create_module_base
from simple_module_db.mixins import MultiTenantMixin, SoftDeleteMixin
from simple_module_db.provider import DatabaseProvider
from sqlmodel import Field

_TenantBase = create_module_base("mt_test", provider=DatabaseProvider.SQLITE)
_TenantBase = create_module_base("mt_test")


class _TenantItem(_TenantBase, MultiTenantMixin, table=True): # ty: ignore[unsupported-base]
Expand Down
Loading
Loading