From c893b1c82b53b048667c92f5ba26af6938a82d85 Mon Sep 17 00:00:00 2001 From: Dimitri Yatsenko Date: Mon, 20 Jul 2026 10:05:59 -0500 Subject: [PATCH] docs: make table-tier definitions consistent (Manual = direct insert, not 'by hand') MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Aligns tutorial and quick-reference tier tables with the authoritative explanation-layer wording (relational-workflow-model.md, autopopulate.md): - Manual: data inserted *directly* from outside the pipeline — by people (forms/GUIs) OR by automated ingestion scripts/instruments — not 'user-entered'/'entered by hand'. The defining property is direct insertion vs. production by make(). - Imported: populated by make() from an external *source* (files, instruments, databases), not only 'files'. - Computed: populated by make() from other tables. Touches quick-ref tables in define-tables.md and table-declaration.md and the tier descriptions in the sql-comparison, calcium-imaging, blob-detection, and 02-schema-design tutorial notebooks. --- src/how-to/define-tables.md | 6 +++--- src/reference/specs/table-declaration.md | 4 ++-- src/tutorials/advanced/sql-comparison.ipynb | 4 ++-- src/tutorials/basics/02-schema-design.ipynb | 8 ++++---- .../domain/calcium-imaging/calcium-imaging.ipynb | 2 +- src/tutorials/examples/blob-detection.ipynb | 4 ++-- 6 files changed, 14 insertions(+), 14 deletions(-) diff --git a/src/how-to/define-tables.md b/src/how-to/define-tables.md index 6348020f..2e53b15b 100644 --- a/src/how-to/define-tables.md +++ b/src/how-to/define-tables.md @@ -30,10 +30,10 @@ class MyTable(dj.Manual): | Type | Base Class | Purpose | |------|------------|---------| -| Manual | `dj.Manual` | Data entered at runtime (users, forms, instruments, imports) | +| Manual | `dj.Manual` | Data inserted directly from outside the pipeline (forms, instruments, ingestion scripts) | | Lookup | `dj.Lookup` | Reference data defined in the schema via `contents` | -| Imported | `dj.Imported` | Data from external sources | -| Computed | `dj.Computed` | Derived data | +| Imported | `dj.Imported` | Populated by `make()` from an external source | +| Computed | `dj.Computed` | Populated by `make()` from other tables | | Part | `dj.Part` | Child of master table | ## Primary Key (Above `---`) diff --git a/src/reference/specs/table-declaration.md b/src/reference/specs/table-declaration.md index ff0d1181..fd93bda0 100644 --- a/src/reference/specs/table-declaration.md +++ b/src/reference/specs/table-declaration.md @@ -23,9 +23,9 @@ class TableName(dj.Manual): | Tier | Base Class | Table Prefix | Purpose | |------|------------|--------------|---------| -| Manual | `dj.Manual` | (none) | Data entered at runtime (users, instruments, imports) | +| Manual | `dj.Manual` | (none) | Data inserted directly from outside the pipeline (users, instruments, ingestion scripts) | | Lookup | `dj.Lookup` | `#` | Reference data defined in the schema via `contents` | -| Imported | `dj.Imported` | `_` | Data from external sources | +| Imported | `dj.Imported` | `_` | Populated by `make()` from an external source | | Computed | `dj.Computed` | `__` | Derived from other tables | | Part | `dj.Part` | `master__` | Detail records of master table | diff --git a/src/tutorials/advanced/sql-comparison.ipynb b/src/tutorials/advanced/sql-comparison.ipynb index d3721d01..8a7240f7 100644 --- a/src/tutorials/advanced/sql-comparison.ipynb +++ b/src/tutorials/advanced/sql-comparison.ipynb @@ -2067,8 +2067,8 @@ "| Tier | Purpose | SQL Equivalent |\n", "|------|---------|----------------|\n", "| `Lookup` | Reference data, parameters | Regular table |\n", - "| `Manual` | User-entered data | Regular table |\n", - "| `Imported` | Data from external files | Regular table + trigger |\n", + "| `Manual` | Data inserted directly from outside the pipeline (people or ingestion scripts) | Regular table |\n", + "| `Imported` | Data from external sources (files, instruments, databases), populated by `make()` | Regular table + trigger |\n", "| `Computed` | Derived results | Materialized view + trigger |\n", "\n", "### 2. Automatic Computation\n", diff --git a/src/tutorials/basics/02-schema-design.ipynb b/src/tutorials/basics/02-schema-design.ipynb index 487159b5..3c699133 100644 --- a/src/tutorials/basics/02-schema-design.ipynb +++ b/src/tutorials/basics/02-schema-design.ipynb @@ -52,9 +52,9 @@ "\n", "| Tier | Class | Purpose | Data Entry |\n", "|------|-------|---------|------------|\n", - "| **Manual** | `dj.Manual` | Core experimental data | Inserted by operators or instruments |\n", + "| **Manual** | `dj.Manual` | Core experimental data | Inserted directly by operators, instruments, or ingestion scripts |\n", "| **Lookup** | `dj.Lookup` | Reference/configuration data | Pre-populated, rarely changes |\n", - "| **Imported** | `dj.Imported` | Data from external files | Auto-populated via `make()` |\n", + "| **Imported** | `dj.Imported` | Data from external sources (files, instruments, databases) | Auto-populated via `make()` |\n", "| **Computed** | `dj.Computed` | Derived/processed data | Auto-populated via `make()` |\n", "\n", "**Manual** tables are not necessarily populated by hand—they contain data entered into the pipeline by operators, instruments, or ingestion scripts using `insert` commands. In contrast, **Imported** and **Computed** tables are auto-populated by calling the `.populate()` method, which invokes the `make()` callback for each missing entry.\n", @@ -1383,9 +1383,9 @@ "- Keep keys minimal but sufficient for uniqueness\n", "\n", "### 2. Use Appropriate Table Tiers\n", - "- **Manual**: Data entered by operators or instruments\n", + "- **Manual**: Data inserted directly — by operators, instruments, or ingestion scripts\n", "- **Lookup**: Configuration, parameters, reference data\n", - "- **Imported**: Data read from files (recordings, images)\n", + "- **Imported**: Data read from external sources (recordings, images, instruments)\n", "- **Computed**: Derived analyses and summaries\n", "\n", "### 3. Normalize Your Data\n", diff --git a/src/tutorials/domain/calcium-imaging/calcium-imaging.ipynb b/src/tutorials/domain/calcium-imaging/calcium-imaging.ipynb index 432263b8..2cbccc37 100644 --- a/src/tutorials/domain/calcium-imaging/calcium-imaging.ipynb +++ b/src/tutorials/domain/calcium-imaging/calcium-imaging.ipynb @@ -1476,7 +1476,7 @@ "metadata": {}, "source": [ "**Legend:**\n", - "- **Green rectangles**: Manual tables (user-entered data)\n", + "- **Green rectangles**: Manual tables (data inserted directly — manually or by ingestion scripts)\n", "- **Gray rectangles**: Lookup tables (parameters)\n", "- **Blue ovals**: Imported tables (data from files)\n", "- **Red ovals**: Computed tables (derived from other tables)\n", diff --git a/src/tutorials/examples/blob-detection.ipynb b/src/tutorials/examples/blob-detection.ipynb index 59f84b3c..ede54011 100644 --- a/src/tutorials/examples/blob-detection.ipynb +++ b/src/tutorials/examples/blob-detection.ipynb @@ -600,7 +600,7 @@ "metadata": {}, "source": [ "The diagram shows:\n", - "- **Green** = Manual tables (user-entered data)\n", + "- **Green** = Manual tables (data inserted directly, manually or via ingestion)\n", "- **Gray** = Lookup tables (reference data)\n", "- **Red** = Computed tables (derived data)\n", "- **Edges** = Dependencies (foreign keys), always flow top-to-bottom" @@ -1398,7 +1398,7 @@ "| Concept | What It Does | Example |\n", "|---------|--------------|--------|\n", "| **Schema** | Groups related tables | `schema = dj.Schema('tutorial_blobs')` |\n", - "| **Manual Table** | Stores user-entered data | `Image`, `SelectedDetection` |\n", + "| **Manual Table** | Stores data inserted directly from outside the pipeline | `Image`, `SelectedDetection` |\n", "| **Lookup Table** | Stores reference/config data | `DetectionParams` |\n", "| **Computed Table** | Derives data automatically | `Detection` |\n", "| **Part Table** | Stores detailed results with master | `Detection.Blob` |\n",