docs(lwdid): tutorial 31 - rolling-transformation DiD (numbers-locked)

Replaces the tutorial withdrawn from PR #588, authored via the
numbers-locked workflow: every cited number prototyped and locked in
scripts first, the notebook assembled and executed once per revision
(per-worktree kernel; committed kernelspec python3).

Content: the TWFE failure mode via Goodman-Bacon decomposition on a
synthetic staggered panel (static TWFE 2.00 vs true 2.87, 21%
forbidden-comparison weight); demeaning with exact T_{N-2} inference on
a 12-unit panel (full classical-model conditions stated); detrending
under treatment-related heterogeneous trends; Prop 99 replicating LW
2026 Table 3 exactly (demean -0.4222/0.1208, detrend -0.2270/0.0941,
exact p 0.021, df 37) with the per-period trajectory and RI as a test
of Fisher's sharp null under the documented package convention (0.054
at 9999 reps; as-printed 0.020 discrepancy noted); Walmart fitted as
the honest staggered design with real first-entry cohorts (WATT(1)
0.0322/0.0051 vs Appendix Table A4, near-lead placebos < 0.006 with
the single-cohort far-lead 0.033 read against its support); cluster +
wild-bootstrap replay on a DGP whose region-post shock survives
demeaning (naive 0.131 vs CR1 0.237); transformation diagnostics
framed as descriptive sensitivity information.

Methodology accuracy: estimand labels
match the default not-yet-treated pool (cell-mass aggregate, with the
complete tau_omega composite gate stated); identification stacks
(NAC/CPTC/OVLC common timing, CNAS/CHT/OVLS staggered) and cross-unit
independence stated for exact inference; t_{G-1} CR1 reference; ETWFE
equivalence scoped to the impact cell (calendar t=g / event r=0);
LWDiD-vs-CS efficiency framed as the paper's complementarity; replay
helpers scoped to common-timing reg fits; paper titles per the
registry artifacts.

Guards: tests/test_t31_lwdid_drift.py (22 tests) pins the rendered
surface (markdown AND outputs) for every load-bearing number, hash-pins
all 13 code cells (stale-output contract with a negative control),
re-derives all four synthetic DGPs against the library, and pins the
corrected methodological wording. REGISTRY staggered-replication note
amended (castle NT-only vs Walmart default-pool goldens); tutorial
registered in docs/tutorials/index.rst + doc-deps (six source entries);
TODO row retired; CHANGELOG entry added.

Author: igerber

CHANGELOG.md

@@ -8,6 +8,16 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 ## [Unreleased]
 
 ### Added
+- **LWDiD tutorial notebook** (`docs/tutorials/31_lwdid.ipynb`): the replacement for
+  the tutorial withdrawn from PR #588, authored via the numbers-locked workflow (every
+  cited number prototyped in scripts first, notebook assembled and executed once).
+  Builds the method from the TWFE failure mode (Goodman-Bacon decomposition on a
+  synthetic staggered panel) through demeaning/detrending and exact small-sample
+  inference, then replicates both papers' applications on the authors' SSC data: Prop
+  99 (LW 2026 Table 3: demean -0.422, detrend -0.227, exact p 0.021; RI under the
+  package's Phipson-Smyth convention) and Walmart entry fitted as the honest
+  staggered design with real first-entry cohorts (WATT(1) = 0.032 vs Appendix Table
+  A4), plus cluster/wild-bootstrap inference and transformation diagnostics.
 - **ImputationDiD `aux_partition` variants now anchored against Stata `did_imputation
   avgeffectsby()` (no library behavior change beyond a docstring wording fix).** The
   coarser `aux_partition="cohort"` / `"horizon"` groupings — previously the

TODO.md

@@ -69,7 +69,6 @@ generic sparse-FE, QR+SVD rank-detection redundancy, `check_finite` bypass — m
 
 | Issue | Location | Origin | Effort | Priority |
 |-------|----------|--------|--------|----------|
-| Author a fresh LWDiD tutorial notebook (the contribution's tutorial 27 was WITHDRAWN from PR #588 during CI review: its Walmart empirical narrative was built on a fabricated common onset with jobs translations that did not match the staggered estimate, and per-sentence repair failed across 5 review rounds). Build via the numbers-locked workflow: prototype in scripts, lock the numbers, then assemble and execute once; teach the staggered fit as the causal specification and keep any pooled contrast explicitly diagnostic | `docs/tutorials/` | #588 | Medium |
 | LWDiD staggered fit recomputes cohort-wide work across surfaces: each cohort's rolling transformation is built once per `fit_staggered` cohort loop, but unit-level post summaries, control-eligibility sets, and the composite path's `ydot_by_cohort` are re-derived independently by the cell, aggregate, and `_composite_regression_aggregation` paths — runtime grows ~cohorts × panel size. Cache the per-cohort transformation + post summaries in one pass and reuse; add a many-cohort unbalanced-panel benchmark | `diff_diff/lwdid_staggered.py`, `diff_diff/lwdid.py` | #588 | Mid | Low |
 | Reuse the demeaner's factorized codes in `absorbed_fe_rank`/`absorbed_fe_cr1_k_increment` instead of re-factorizing: at 186k rows the rank helper adds ~1.9 ms per absorbed fit (7.7% of the fastest Rust-served TWFE fit) and the K_reference increment ~3.2 ms per clustered-hc1 absorbed fit (~13%; see `docs/performance-plan.md` "Component-aware absorbed-FE rank"), and the helpers and `demean_by_groups` factorize the same group columns. Threading the codes through the call sites halves the factorize work; the `connected_components` call itself is ~1.1 ms. Deliberately not done in the correctness PRs. | `diff_diff/utils.py` | #variance-inventory | Quick | Low |
 | `EfficientDiD` conditional path: the largest remaining O(n) stage is the sieve/nuisance construction outside the tiled pass (~9s at 10k). (The `_ridge_solve_weights` Python-prep shave landed 2026-07-07 — the `omega_stack[rest]` fancy-index copy and tail scatter are skipped when no row is zero-masked, byte-identical outputs; the `zero_mask` abs scan itself remains, needed for correctness.) | `efficient_did_covariates.py` | CS-scaling | Mid | Low |

docs/doc-deps.yaml

@@ -867,12 +867,16 @@ sources:
     docs:
       - path: docs/api/lwdid.rst
         type: api_reference
+      - path: docs/tutorials/31_lwdid.ipynb
+        type: tutorial
 
   diff_diff/lwdid_randomization.py:
     drift_risk: medium
     docs:
       - path: docs/api/lwdid.rst
         type: api_reference
+      - path: docs/tutorials/31_lwdid.ipynb
+        type: tutorial
 
   diff_diff/lwdid_sensitivity.py:
     drift_risk: medium
@@ -885,6 +889,8 @@ sources:
     docs:
       - path: docs/api/lwdid.rst
         type: api_reference
+      - path: docs/tutorials/31_lwdid.ipynb
+        type: tutorial
 
   diff_diff/lwdid_staggered.py:
     drift_risk: medium
@@ -894,6 +900,8 @@ sources:
       - path: docs/methodology/REGISTRY.md
         section: "LWDiD"
         type: methodology
+      - path: docs/tutorials/31_lwdid.ipynb
+        type: tutorial
 
   diff_diff/lwdid.py:
     drift_risk: medium
@@ -910,6 +918,8 @@ sources:
         type: user_guide
       - path: docs/choosing_estimator.rst
         type: user_guide
+      - path: docs/tutorials/31_lwdid.ipynb
+        type: tutorial
 
   # ── TROP (trop group) ──────────────────────────────────────────────
 
@@ -1043,6 +1053,9 @@ sources:
       - path: docs/tutorials/02_staggered_did.ipynb
         type: tutorial
         note: "Bacon decomposition diagnostics for staggered DiD"
+      - path: docs/tutorials/31_lwdid.ipynb
+        type: tutorial
+        note: "TWFE failure-mode demonstration"
       - path: README.md
         section: "Estimators (one-line catalog entry)"
         type: user_guide
@@ -1381,6 +1394,9 @@ sources:
         type: api_reference
       - path: docs/tutorials/09_real_world_examples.ipynb
         type: tutorial
+      - path: docs/tutorials/31_lwdid.ipynb
+        type: tutorial
+        note: "Prop 99 / Walmart replication (provenance-asserted)"
       - path: docs/methodology/REGISTRY.md
         section: "Castle Doctrine treatment coding"
         type: methodology

docs/methodology/REGISTRY.md

@@ -2551,7 +2551,7 @@ Event-study/placebo transformations over ALL periods (Appendix D): demeaning (D.
 - IPWRA (workhorse): logit propensity score per cell + WLS with weights `w = D + (1-D) p/(1-p)`; IPW = special case without the outcome-regression component.
 - **Note:** API vocabulary (as specified for the PR #588 implementation, in review): `estimation_method="reg"` = the paper's RA, `"ipw"` = IPW, `"dr"` = IPWRA (house CallawaySantAnna values; the doubly-robust option), `"psm"` = propensity-score matching, an LWDiD-only extra with no CS analog.
 - Control pool at (g, r): `A_{r+1} = 1` (never-treated + not-yet-treated) by default; NT-only optional. Pre-treatment placebo cells use the Appendix D.3 rule `A_{g,t} = {G = g} ∪ {G = 0} ∪ {G > max(g,t)}`.
-- **Note (replicating the paper's staggered numbers):** the implementation's default is `control_group='not_yet_treated'`, matching OVLS (eq. (4.10)) as stated in the text. The paper's *printed* staggered results, however, are computed against the never-treated pool only, so **reproducing them requires passing `control_group='never_treated'` explicitly**. This is not a discrepancy in either direction — it is a sample-definition choice that the text leaves to the analyst while the applications fix it to NT-only. Every staggered replication golden in `tests/test_methodology_lwdid.py` (castle `tau_omega`, and the composite-regression reference) therefore passes `control_group="never_treated"`; a default-pool fit yields different, equally valid estimates because the (g,t) cells draw on a strictly larger control sample.
+- **Note (replicating the paper's staggered numbers):** the implementation's default is `control_group='not_yet_treated'`, matching OVLS (eq. (4.10)) as stated in the text. The paper's *printed* staggered applications split by regime (amended in the tutorial-31 authoring pass; previously stated as NT-only across the board): the small-N composite applications (castle `tau_omega`) are never-treated-based by construction — the Stata small-N mode has no control-pool option — so **reproducing them requires passing `control_group='never_treated'` explicitly**, and the castle/composite goldens in `tests/test_methodology_lwdid.py` do; the large-N event-study applications (Walmart Tables A4/A5) use the package's default pool (never-treated + not-yet-treated, matching the library default `control_group='not_yet_treated'`), and the Walmart event-study goldens fit with the default accordingly (see the `control_pool` provenance block in `benchmarks/data/lwdid_stata_golden.json`). Neither is a discrepancy — the pool is a sample-definition choice; a different-pool fit yields different, equally valid estimates because the (g,t) cells draw on a different control sample.
 
 *Aggregation:*
 - Event-study: `WATT(r) = sum_{g in G_r} omega_{g,r} ATT(g, g+r)` with `omega_{g,r}` = (treated units of cohort g contributing at event time r) / (total treated units contributing at event time r) - the operative definition per LW 2025 Appendix E.1, required under unbalanced panels where a cohort's contributing count at r can differ from `N_g`. In balanced panels this simplifies to `N_g / N_{G_r}` (Sec. 6.2/D.3). Aggregated influence function `IF_{i,r} = sum_g omega_{g,r} IF_{i,g,g+r}`.