Skip to content

feat(sidebar): copy objects and duplicate a database across connections - #2567

Merged
datlechin merged 2 commits into
mainfrom
feat/copy-objects-across-connections
Aug 29, 2026
Merged

feat(sidebar): copy objects and duplicate a database across connections#2567
datlechin merged 2 commits into
mainfrom
feat/copy-objects-across-connections

Conversation

@datlechin

Copy link
Copy Markdown
Member

Draft for #2487. Do not merge yet: two independent Codex reviews returned no-ship verdicts, most
of what they found is fixed, and the list of what is not is at the bottom.

Stacked on #2566, which carries the shared DatabaseEndpoint and create-database-form refactors
this needs. Review that one first.

What it does

Right-click a table, a view, a schema or a database in the sidebar and choose Copy To…, or
Duplicate Database… on a database row. Both are also in the Database menu. Choose structure,
data or both, read the script, then copy.

Rows stream: the source's streamRows feeds batches straight into multi-row parameterized INSERTs
sized against the engine's bind-parameter ceiling, so the write side of a ten-million-row table
costs the same as a ten-row one.

Design

New TablePro/Core/ObjectCopy/, reusing rather than duplicating:

Reused For
DatabaseEndpoint, DatabaseEndpointPicker connection → database → schema target picking
CompareMetadataService reading the source's structure, views, routines and triggers
SchemaSyncScriptBuilder, SourceObjectSyncBuilder DDL in the target's dialect
ForeignKeyTopologicalSort parents before children
SQLStatementGenerator the batched INSERT, the writer CSV and JSON import already use
ExecutionGate one authorization for the whole run

The piece both reviews said was missing is ObjectCopyNamespace: the name an engine qualifies its
objects with. It is the schema where the engine has schemas and the database where it does not,
because MySQL reports the database in information_schema's schema column, so its foreign keys and
routines come back qualified by it. Getting that wrong is what silently dropped every dependency
edge on MySQL and PostgreSQL and what made the same-namespace test that guards definition copying
compare nil against nil across two different databases.

The run is: create the database, tear down children-first and build parents-first in one pass, empty
what a data-only replace will append to, copy rows, then install triggers and materialized views.

Fixed after review 1 (30 findings, 21 P1)

The three destructive ones:

  • Data-only Replace deleted the target's rows for good. The TRUNCATE ran in the DDL phase,
    before the per-table transaction, so a failure rolled back only the inserts. It is now inside the
    transaction, and it is a DELETE whenever rollback is promised, because TRUNCATE commits
    implicitly on engines without transactional DDL.
  • Replace could DROP the source's own view or trigger. Definitions now copy only within one
    namespace.
  • Preselection fell back to the whole catalog when its name-only lookup matched nothing.

Plus: FK dependency edges, cleanup ordering, triggers after data, Add-rows no longer replacing
definitions, foreign keys repointed at the copy, one-engine-only copies, routine signatures and
trigger owners in object identity, target column and table spellings, Duplicate gated on the
driver's create-database form, Oracle's one-row batches, rolled-back rows excluded from counts,
cancellation read as cancellation, the injected DatabaseManager used on both sides.

Fixed after review 2 (29 findings, 14 P1)

  • Duplicate Database was unusable on every schema-aware engine. My own review-1 fix refused any
    database-level copy that named no schema, and the sidebar passes none. A database copy now
    enumerates every schema and reads, orders and writes each in its own scope.
  • Neither review-1 fix worked on MySQL, the issue's own engine. ObjectCopyNamespace is the
    answer.
  • Replace generated no DROP on any core driver. dropObjectStatement returns nil on the raw
    plugin for MySQL, PostgreSQL, SQL Server, SQLite, Oracle, DuckDB and Trino, so the CREATE ran
    against the table still there. There is a quoted fallback now.
  • Cleanup and creation run in one scoped call, wrapped in a transaction where DDL is transactional
    and with foreign key checks relaxed where it is not, so a stop between them cannot leave objects
    dropped with nothing put back.
  • Data-only Replace clears every participating table child-first before any is filled.
  • A driver that answers with a view's SELECT rather than its CREATE has that view left out
    rather than executed as a read and reported as copied.
  • Generated and identity columns are read from the driver's own PluginColumnInfo, which the
    snapshot conversion drops.
  • Identifier matching prefers the exact spelling and folds case only when it is unambiguous.
  • Materialized views build after the rows, cancellation is checked before the final flush, committed
    counts survive a non-transactional stop, selectAll unions rather than replaces, Duplicate in the
    menu bar keys on database switching rather than container switching, Copy To reaches hierarchical
    schema rows, materialized views map to the right kind, structure-only plans skip row preparation.

Still open

These are why this is a draft:

  • PostgreSQL SERIAL sequences are not copied. fetchDependentSequences exists and is not
    called, so a copied table's default names a sequence the target does not have.
  • Dameng and Teradata inherit the non-streaming streamRows default, so a large table is
    materialised before the first batch. Gating properly needs a capability all 32 plugins declare.
    Export has the same property today.
  • Routine and trigger DROPs carry no signature or ON <table>, so replacing an overloaded
    PostgreSQL routine fails.
  • A partial catalog read is silent. A routine or trigger listing that fails on permissions
    becomes an empty list, and the copy reports success having carried only tables.
  • Selecting one routine still reads every routine's DDL in the schema before filtering.
  • No screenshot. The ones I captured showed the local username in the title bar, so I removed
    them rather than publish them.
  • CopyObjectsUITests has never run here. Both attempts died on Timed out while enabling automation mode, an unanswered macOS automation prompt. CI has to be the first real run.

What I built and tested

Step Verdict
verify.sh build PASS
verify.sh test (12 suites) PASS, 171/171
verify.sh lint TablePro TableProTests TableProUITests 0 violations
docs/scripts/check-writing-style.sh PASS
docs/scripts/check-docs-against-source.py PASS
verify.sh uitest CopyObjectsUITests INCONCLUSIVE, 0 cases, automation prompt

verify.sh plugins and verify.sh abi were not run and are not needed: nothing under Plugins/
changed.

Base automatically changed from refactor/share-database-endpoint-and-create-database-form to main August 28, 2026 15:59
@datlechin
datlechin force-pushed the feat/copy-objects-across-connections branch from 1fd2ef9 to 37e67d1 Compare August 29, 2026 02:24
@mintlify

mintlify Bot commented Aug 29, 2026

Copy link
Copy Markdown

Preview deployment for your docs. Learn more about Mintlify Previews.

Project Status Preview Updated (UTC)
TablePro 🟢 Ready View Preview Aug 29, 2026, 2:25 AM

💡 Tip: Enable Workflows to automatically generate PRs for you.

@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.

@datlechin
datlechin merged commit 1f0d357 into main Aug 29, 2026
9 checks passed
@datlechin
datlechin deleted the feat/copy-objects-across-connections branch August 29, 2026 02:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant