Skip to content

feat!: remove the dlt and ingestr integrations - #1554

Merged
chio-labs-delivery[bot] merged 6 commits into
mainfrom
feat/remove-dlt-ingestr
Nov 17, 2026
Merged

chio-labs-delivery[bot] merged 6 commits into
mainfrom
feat/remove-dlt-ingestr

Conversation

@kvlonge

@kvlonge kvlonge commented Nov 17, 2026

Copy link
Copy Markdown
Contributor

Why

The dlt (dlt_sources) and ingestr (a source's ingestr block) integrations ran loaders outside SQLBuild's warehouse connections, with their own connection mode, checks and plan fields. A Python @loader covers the same need on a SQLBuild connection.

Changes

  • Removed the dlt and ingestr packages, their generated loaders and the external loader connection mode.
  • Removed the dlt and ingestr extras and the plan's integration_kind field. Nothing emits the external_source_load and ingestr_command operations any more, but stored run history that names them still decodes.
  • A sources file that still declares dlt_sources, or a source that still declares ingestr, fails compile with an error that names the file or source and points to @loader.
  • Docs: removed the dlt and ingestr pages, screenshots and mentions; the old URLs redirect to the loader docs. Skill references regenerated.
  • Compiler failure corpus: one case each for a leftover dlt_sources key and ingestr block. The project assembly goldens are regenerated for the removed fields.

BREAKING CHANGE: a project that declares dlt_sources or a source ingestr block no longer compiles. Declare the source with managed: true and load it with a Python @loader named after the source. The dlt and ingestr extras no longer exist.

The next integration pull request builds on this one.

Verification

  • cargo test for the discovery and core crates; a full extension and CLI build.
  • Targeted unit, integration and DuckDB e2e suites for loads, sources YAML discovery, Python node inventory, plan text, history decoding, setting help, project assembly goldens and sqb load.
  • make test: the only local failures were launcher tests that need an installed environment; they pass in CI on the branch built on this one.
  • fensu check, rustfmt and make check-ci on the head.
  • One independent review of the full diff; its findings are fixed.

Remove the declarative dlt (`dlt_sources`) and ingestr (a source's `ingestr`
block) integrations, the loaders SQLBuild generated for them, and the external
loader connection mode they used: loaders that ran before SQLBuild opened its
warehouse connections, with their own scheduling checks. Every loader now runs
on a SQLBuild connection. The `dlt` and `ingestr` extras, the
`external_source_load` and `ingestr_command` lifecycle operations and the
plan's `integration_kind` field are gone.

A sources file that still declares `dlt_sources`, or a source that still
declares `ingestr`, fails compile with an error that points to `@loader`.

BREAKING CHANGE: the dlt and ingestr integrations are removed. A project that
declares `dlt_sources` or a source `ingestr` block no longer compiles; declare
the source with `managed: true` and load it with a Python `@loader` named after
the source. The `dlt` and `ingestr` package extras no longer exist.
Drop the dlt and ingestr integration pages, their sidebar entries and screenshots, and every mention in the loader, source, adapter, configuration and feature comparison pages, and regenerate the skill references.
…ir pages

History written while dlt or ingestr loads ran still names their operations,
so the operation catalogue keeps both names for decoding. The removed docs
pages redirect to the loaders page, the failure corpus covers both
removed-integration errors, and the project assembly goldens record the
digests without integration loaders.
@chio-labs-delivery
chio-labs-delivery Bot added this pull request to the merge queue Nov 17, 2026
Merged via the queue into main with commit 75d213e Nov 17, 2026
60 of 62 checks passed
@chio-labs-delivery
chio-labs-delivery Bot deleted the feat/remove-dlt-ingestr branch November 17, 2026 16:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant