Feat: Configure virtual layer catalogs per gateway

[josephfinlayson]

Description

This pull request is stacked on #5962. The child-specific implementation begins after commit 0963049b; until #5962 merges, GitHub shows both layers in the diff against main.

SQLMesh can already route a model's execution and gateway-managed virtual layer through its effective gateway. However, the gateway connection's default catalog also qualifies the model, so it determines both the physical snapshot catalog and, normally, the published-view catalog. environment_catalog_mapping can move published views, but it applies one environment-wide catalog override. A shared environment therefore cannot preserve separate physical catalogs while publishing each gateway's models into a different stable catalog.

This adds an optional virtual_layer_catalog to GatewayConfig:

gateways:
  project_a:
    connection:
      type: databricks
      catalog: project_a_physical
      server_hostname: <server_hostname>
      http_path: <http_path>
      access_token: <access_token>
    virtual_layer_catalog: project_a_published

model_defaults:
  dialect: databricks
  gateway: project_a

gateway_managed_virtual_layer: true

With this configuration, versioned snapshot tables remain in project_a_physical, while stable environment views are published in project_a_published. The route is configured once per gateway rather than repeated on individual models.

Resolution and precedence

• SQLMesh resolves the route from the model's effective gateway: explicit model gateway, then model_defaults.gateway, then the selected gateway.
• The connection/default catalog continues to determine the physical model and snapshot location.
• virtual_layer_catalog determines the base catalog of the model's published environment view.
• An existing matching environment_catalog_mapping retains precedence for backward compatibility.
• Catalog-suffixed development environments append the environment suffix to the configured virtual-layer catalog.
• Per-model and model-default virtual_layer_catalog properties are rejected; this is intentionally a gateway-level policy.
• Managed SQLMesh models, seeds, Python models, and dbt-derived managed models and seeds use the route. External/source definitions do not inherit it.

State and lifecycle behavior

The resolved route is stored with model and snapshot metadata, including SnapshotTableInfo. This lets a partial multi-project plan preserve the published locations of remote snapshots loaded from shared state even when their project configuration is not loaded.

Changing only virtual_layer_catalog is a metadata-only change. Applying that plan publishes the existing snapshot through the new catalog and demotes the previous view without rebuilding the physical table. Janitor cleanup also uses the persisted route.

Composed project configurations must agree when they define the same gateway name. Conflicting virtual_layer_catalog values raise a configuration error instead of making route resolution depend on project load order.

Compatibility

• Omitting virtual_layer_catalog preserves existing model metadata hashes and publishing behavior.
• The new serialized fields are optional and use backward-safe reads for older cached models and snapshots.
• The loader cache key includes the complete resolved gateway route map, so a route supplied by another composed project invalidates stale model cache entries.
• Existing environment_catalog_mapping behavior and precedence are unchanged.

Prior art and design choice

This builds on SQLMesh's model-level multi-engine routing (#3394), gateway-specific model defaults (#3888), gateway-managed virtual layers (#4101), and environment catalog naming (#4742).

It deliberately avoids a per-model catalog override. #3645 raised the same API-proliferation concern for physical schema routing; a gateway-level policy keeps routing centralized while explicit model gateways continue to provide the existing model-level escape hatch. #4684 tracks the broader, separate problem of generic physical catalog/schema mapping.

dbt provides related but not identical precedents: hierarchical project/directory relation configuration, logical catalog mappings in catalogs.yml, and stable latest-version pointers. This change keeps SQLMesh's existing snapshot and environment semantics while applying the same separation of physical storage from a stable published name.

Test plan

• Gateway config parsing, empty-value validation, and composed-config conflict detection.
• Effective-gateway routing for explicit, model-default, selected, hyphenated, blueprint, SQL, Python, seed, and dbt-derived models.
• Regression coverage proving external/source definitions remain unrouted.
• Snapshot serialization and backward-compatible metadata hashing.
• Existing environment catalog mapping precedence and catalog-suffixed development naming.
• Cross-project loader-cache invalidation when only another project's gateway route changes.
• Multi-project integration coverage for initial deployment, partial-project planning, remote route retention, metadata-only route moves, old-view demotion, and unchanged remote snapshots.
• Janitor cleanup through the persisted route.
• make style
• make fast-test (2,787 passed across the fast, isolated, registry-isolation, and dialect-isolation phases; 4 skipped)

Checklist

• I have run make style and fixed any issues.
• I have added tests for the change.
• Existing fast tests pass locally.
• Every commit is signed off per the DCO.