Describe the bug
Expired-environment cleanup can fail for a gateway-managed ClickHouse model
because janitor passes a synthetic-catalog-qualified view name to a ClickHouse
adapter whose virtual catalog has not been injected.
The environment record is invalidated, but its logical view remains. A scoped
janitor run reports:
Cleanup failed!
Janitor completed with failures:
Failed to drop the expired environment view
'"__clickhouse_gw__".my_db.connection_test__dev':
clickhouse does not support catalogs and a catalog was provided:
__clickhouse_gw__
To reproduce
- Configure a multi-gateway project with a catalog-aware default gateway, a
ClickHouse secondary gateway, and gateway_managed_virtual_layer enabled.
- Create and promote a ClickHouse model into a development environment.
- Run
sqlmesh invalidate dev --sync.
- If the logical view remains, run
sqlmesh janitor --environment dev.
In SQLMesh 0.236.0, the separate CREATE VIEW virtual-catalog issue must first
be worked around to reach this cleanup path.
Expected behavior
Invalidation or scoped janitor cleanup should select the model's ClickHouse
gateway adapter, restore the same virtual-catalog configuration used during
planning and promotion, strip that catalog from the target, and execute a
two-part DROP VIEW on ClickHouse.
Likely cause
sqlmesh/core/janitor.py correctly selects the snapshot's model gateway when
the environment is gateway-managed. However, virtual-catalog injection occurs
through scheduler default-catalog discovery and is not guaranteed to run for a
cleanup-only command. The selected ClickHouse adapter therefore still reports
CatalogSupport.UNSUPPORTED, and the inherited @set_catalog() wrapper rejects
the stored three-level view name instead of stripping it.
A local cleanup override that removed only the expected synthetic gateway
catalog allowed scoped janitor cleanup to remove the expired view across the
cluster without affecting another environment or the shared physical object.
Impact
Invalidated ClickHouse development environments can leave orphaned logical
views. Cleanup requires manual DDL or an adapter workaround even though the
environment state tracks the model gateway.
Environment
- SQLMesh: 0.236.0
- ClickHouse: 25.8
- ClickHouse adapter in cluster mode
Related work
Describe the bug
Expired-environment cleanup can fail for a gateway-managed ClickHouse model
because janitor passes a synthetic-catalog-qualified view name to a ClickHouse
adapter whose virtual catalog has not been injected.
The environment record is invalidated, but its logical view remains. A scoped
janitor run reports:
To reproduce
ClickHouse secondary gateway, and
gateway_managed_virtual_layerenabled.sqlmesh invalidate dev --sync.sqlmesh janitor --environment dev.In SQLMesh 0.236.0, the separate
CREATE VIEWvirtual-catalog issue must firstbe worked around to reach this cleanup path.
Expected behavior
Invalidation or scoped janitor cleanup should select the model's ClickHouse
gateway adapter, restore the same virtual-catalog configuration used during
planning and promotion, strip that catalog from the target, and execute a
two-part
DROP VIEWon ClickHouse.Likely cause
sqlmesh/core/janitor.pycorrectly selects the snapshot's model gateway whenthe environment is gateway-managed. However, virtual-catalog injection occurs
through scheduler default-catalog discovery and is not guaranteed to run for a
cleanup-only command. The selected ClickHouse adapter therefore still reports
CatalogSupport.UNSUPPORTED, and the inherited@set_catalog()wrapper rejectsthe stored three-level view name instead of stripping it.
A local cleanup override that removed only the expected synthetic gateway
catalog allowed scoped janitor cleanup to remove the expired view across the
cluster without affecting another environment or the shared physical object.
Impact
Invalidated ClickHouse development environments can leave orphaned logical
views. Cleanup requires manual DDL or an adapter workaround even though the
environment state tracks the model gateway.
Environment
Related work