Skip to content

fix(fabric): include default warehouse on listed objects - #6042

Open
cmgoffena13 wants to merge 5 commits into
SQLMesh:mainfrom
cmgoffena13:fix/fabric-default-cache
Open

cmgoffena13 wants to merge 5 commits into
SQLMesh:mainfrom
cmgoffena13:fix/fabric-default-cache

Conversation

@cmgoffena13

@cmgoffena13 cmgoffena13 commented Sep 10, 2026 •

Copy link
Copy Markdown
Collaborator

Description

Closes #5970 -- second part. Aligns the objects list to standard with other engines. Fixes shared test.

  • Fabric treats the default warehouse as None so we skip reconnects. get_data_objects therefore returned catalog=None. Other engines return a real warehouse name.
  • Cache still uses schema.table (super().get_data_objects()). The warehouse name is filled in on the returned objects only.
  • Catalog name on the object -> catalog from the requested schema if present, otherwise current target / connected warehouse / configured default.

Also fixes a bug. A lazy restore to None would still have the connection to the last connection (planning). So any list of objects after that with no catalog would actually return objects for planning rather than the default catalog.

Test plan

  • test_get_data_objects on Fabric (query + df)
  • Unit: that list twice with safe_to_cache=True is one fetch
  • Unit: after a lazy restore to None, get_data_objects("dbo") is labeled planning correctly, not the default warehouse. get_data_objects("ci_abc.dbo") is still labeled ci_abc.
  • Unit: drop of a ci_abc object — two-part SQL, execute-time catalog is ci_abc (no reconnect if already default; switch and restore if on planning)

Checklist

  • I have run make style and fixed any issues
  • I have added tests for my changes (if applicable)
  • All existing tests pass (make fast-test)
  • My commits are signed off (git commit -s) per the DCO

@nicksspirit

Copy link
Copy Markdown
Collaborator

Looking into this one

@nicksspirit

nicksspirit commented Oct 12, 2026 •

Copy link
Copy Markdown
Collaborator

Thanks for picking up the Fabric CI failure in #5970. I went through the change.

The new get_data_objects path looks like it covers the list test. That test already passes a schema that includes the warehouse name, and the new code copies that name onto each listed object:

def get_data_objects(
self,
schema_name: t.Union[str, exp.Table],
object_names: t.Optional[t.Set[str]] = None,
safe_to_cache: bool = False,
) -> t.List[DataObject]:
# Fabric uses None as "default catalog" so we skip reconnects. Other engines
# return a real warehouse name here.
# Cache on schema.table due to @set_catalog stripping the catalog. Then put a
# warehouse name on the returned objects so listing matches other engines.
objects = super().get_data_objects(schema_name, object_names, safe_to_cache)
catalog = to_schema(schema_name).catalog or self._resolved_catalog()
if not catalog:
return objects
return [
DataObject(
catalog=obj.catalog or catalog,
schema=obj.schema_name,
name=obj.name,
type=obj.type,
clustering_key=obj.clustering_key,
)
for obj in objects

The other failure in the issue looks like it is still there. This is existing behavior on main, not something this PR introduces. test_catalog_operations asks for the current catalog and lowercases it before it lists anything:

current_catalog = ctx.engine_adapter.get_current_catalog().lower()
ctx.engine_adapter.set_current_catalog(catalog_name)
assert ctx.engine_adapter.get_current_catalog().lower() == catalog_name
ctx.engine_adapter.set_current_catalog(current_catalog)
assert ctx.engine_adapter.get_current_catalog().lower() == current_catalog

On the default warehouse for fabric, get_current_catalog() still returns None:

def get_current_catalog(self) -> t.Optional[str]:
"""Return the explicit Fabric catalog target for the current thread."""
return self._normalize_catalog(self._target_catalog)

_normalize_catalog turns an empty value, and the default warehouse name, into None:

def _normalize_catalog(self, catalog_name: t.Optional[str]) -> t.Optional[str]:
if not catalog_name:
return None
default_catalog = self._default_catalog or self._extra_config.get("database")
if default_catalog and catalog_name == default_catalog:
return None

None has no .lower(), so that test crashes. This PR does not change get_current_catalog(), and the unit test still expects None:

def test_get_current_catalog_uses_only_explicit_target_catalog(
make_mocked_engine_adapter: t.Callable,
):
adapter = make_mocked_engine_adapter(
FabricEngineAdapter,
database="default_catalog",
)
assert adapter.get_current_catalog() is None

So #5970 is only partly addressed. The reconnect logic can keep the empty value internally, so Fabric can still skip a reconnect. get_current_catalog() needs to return the configured warehouse name when we are on the default catalog. Then the shared test has a name it can lowercase.

Signed-off-by: Cortland Goffena <30168413+cmgoffena13@users.noreply.github.com>
Signed-off-by: Cortland Goffena <30168413+cmgoffena13@users.noreply.github.com>
Signed-off-by: Cortland Goffena <30168413+cmgoffena13@users.noreply.github.com>
Signed-off-by: Cortland Goffena <30168413+cmgoffena13@users.noreply.github.com>
@cmgoffena13
cmgoffena13 force-pushed the fix/fabric-default-cache branch from bc2c724 to c491e42 Compare October 13, 2026 23:50
@cmgoffena13

Copy link
Copy Markdown
Collaborator Author

@nicksspirit -- this has been updated -- good call out on the catalog still being None. This should align the normal output of the fabric adapter to the adapter interface while still keep the fabric shenanigans operational under the hood.

@cmgoffena13

cmgoffena13 commented Oct 14, 2026 •

Copy link
Copy Markdown
Collaborator Author

Databricks CI failure is separate, addressed here: #6142 and the Windows CI failure is separate as well, but it seems like pendulum can't be found. investigating

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

CI Fix - Fabric CI Currently Failing

2 participants