Describe the bug
In a multi-gateway project using ClickHouse's virtual-catalog support from
#5826, SQLMesh strips the synthetic catalog from the target of a virtual-layer
CREATE VIEW, but leaves it on the physical view referenced by the SELECT.
ClickHouse receives an invalid three-part source name:
CREATE OR REPLACE VIEW "my_db"."connection_test__dev" ON CLUSTER "my_cluster"
AS SELECT *
FROM "__clickhouse_gw__"."my_db"."my_db__connection_test__1234567890"
and rejects it because ClickHouse supports only database.table names.
To reproduce
- Configure a project with a catalog-aware default gateway and a ClickHouse
secondary gateway.
- Enable
gateway_managed_virtual_layer.
- Define a ClickHouse
VIEW model using gateway clickhouse_gw.
- Run
sqlmesh plan dev --auto-apply.
The physical view is created successfully. Virtual-layer promotion then fails
with ClickHouse error code 62 when SQLMesh executes the three-part source
reference shown above.
Expected behavior
SQLMesh should remove the configured synthetic virtual catalog from every
ClickHouse table reference before sending view DDL to ClickHouse. The generated
query should be equivalent to:
CREATE OR REPLACE VIEW "my_db"."connection_test__dev" ON CLUSTER "my_cluster"
AS SELECT *
FROM "my_db"."my_db__connection_test__1234567890"
Likely cause
The @set_catalog() handling on the inherited create_view() method normalizes
the view_name argument, but not tables nested inside the query_or_df
expression. The snapshot evaluator supplies the physical source using its
three-level virtual FQN, so the synthetic catalog reaches ClickHouse.
A local adapter override that copied the query expression and removed only the
adapter's configured virtual catalog from nested exp.Table nodes allowed the
same plan to promote successfully. An unexpected catalog was still rejected.
Impact
Gateway-managed virtual-layer promotion cannot complete for ClickHouse models
in affected multi-gateway projects. Physical objects may be created before the
promotion failure, leaving a partially applied plan to clean up.
Environment
- SQLMesh: 0.236.0
- ClickHouse: 25.8
- ClickHouse adapter in cluster mode
Related work
Describe the bug
In a multi-gateway project using ClickHouse's virtual-catalog support from
#5826, SQLMesh strips the synthetic catalog from the target of a virtual-layer
CREATE VIEW, but leaves it on the physical view referenced by theSELECT.ClickHouse receives an invalid three-part source name:
and rejects it because ClickHouse supports only
database.tablenames.To reproduce
secondary gateway.
gateway_managed_virtual_layer.VIEWmodel usinggateway clickhouse_gw.sqlmesh plan dev --auto-apply.The physical view is created successfully. Virtual-layer promotion then fails
with ClickHouse error code 62 when SQLMesh executes the three-part source
reference shown above.
Expected behavior
SQLMesh should remove the configured synthetic virtual catalog from every
ClickHouse table reference before sending view DDL to ClickHouse. The generated
query should be equivalent to:
Likely cause
The
@set_catalog()handling on the inheritedcreate_view()method normalizesthe
view_nameargument, but not tables nested inside thequery_or_dfexpression. The snapshot evaluator supplies the physical source using its
three-level virtual FQN, so the synthetic catalog reaches ClickHouse.
A local adapter override that copied the query expression and removed only the
adapter's configured virtual catalog from nested
exp.Tablenodes allowed thesame plan to promote successfully. An unexpected catalog was still rejected.
Impact
Gateway-managed virtual-layer promotion cannot complete for ClickHouse models
in affected multi-gateway projects. Physical objects may be created before the
promotion failure, leaving a partially applied plan to clean up.
Environment
Related work