Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 18 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,24 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
remain outside the Stata anchor and are documented as such (REGISTRY `## LPDiD`
Deviation 4; `DEFERRED.md` non-absorbing row narrowed to the RA +
placebo/pooled-pre residuals).
- **LWDiD guide coverage + scope-limitation docs + Rust parity tests** (pre-release
audit). The estimator was missing from the full, practitioner, and autonomous
wheel-shipped AI guides (only the `llms.txt` catalog line existed): `llms-full.txt`
gains an `LWDiD` section (introspection-pinned constructor/fit signatures, estimand-
by-timing-mode caveats, a Choosing-an-Estimator row) and an `LWDiDResults` entry;
`llms-practitioner.txt`'s few-treated branch and `llms-autonomous.txt`'s support
matrix / §4.3 / §4.8 now route to LWDiD's exact small-N inference niche. Scope
limitations are now documented on every relevant surface (panel-only, no sampling
weights / `survey_design=`, no `anticipation=`, HonestDiD non-admission): `lwdid.rst`
Scope-limitations note, a REGISTRY Edge-cases Note, the survey-support and SE-method
matrices in `choosing_estimator.rst`, `survey-roadmap.md`, and
`practitioner_decision_tree.rst`, with the anticipation offset, HonestDiD admission,
and survey weights recorded as DEFERRED rows (and practitioner-code-surface routing
as a TODO row). New `TestLWDiDBackendParity` (5 Rust-vs-Python estimator-level
equivalence tests: reg/hc1 plain + CR1, post-fit RI/WCB replay, staggered
event-study, dr) and `TestLWDiDGuideSection` guide pins. `doc-deps.yaml` maps
`lwdid.py` to all new surfaces; stale pre-merge wording ("forthcoming", "in
review") retired from `references.rst` and the REGISTRY section.
- **LWDiD tutorial notebook** (`docs/tutorials/31_lwdid.ipynb`): the replacement for
the tutorial withdrawn from PR #588, authored via the numbers-locked workflow (every
cited number prototyped in scripts first, notebook assembled and executed once).
Expand Down
3 changes: 3 additions & 0 deletions DEFERRED.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,8 @@ provenance and AI-review deviation-documentation: a row here (or in
|-------|----------|----|----------|
| LWDiD PSM matching variance: implement the Abadie-Imbens (2006) matching variance (matched-control reuse + first-stage matching uncertainty) so `estimation_method='psm'` can report valid inference instead of the current fail-closed NaN tuple (point retained; naive var(diffs)/n was invalid under with-replacement reuse) | `diff_diff/lwdid.py` | #588 | Low |
| LWDiD cohort-relative sensitivity exclusions: `robustness_pre_periods` / `sensitivity_no_anticipation` currently reject multi-cohort staggered inputs because their exclusion windows are defined relative to the EARLIEST adoption (later cohorts' own pre periods fall inside the global post window and survive every restriction, mislabeling the specification). Supporting staggered inputs needs per-cohort window semantics (exclude the last k periods of each cohort's own `t < g` window before its transformation), which the current row-subset design cannot express — a per-cohort masking derivation + its aggregation contract | `diff_diff/lwdid_sensitivity.py` | #588 | Low |
| LWDiD estimation-time anticipation offset (CS-parity gap): no `anticipation=` parameter exists — no-anticipation is hard-assumed and the only surface is the single-cohort `sensitivity_no_anticipation` refit diagnostic. The LW device for anticipation is the anchor-window shift — drop the last k pre-treatment periods from the transformation/anchor window (`Ybar_{i,S0}` with `S0 < S-1`, per the REGISTRY LWDiD Anticipation edge case; NOT a cohort recode g → g − k) — and exposing an `anticipation=` parameter needs that derivation checked against LW 2025 NAC/CNAS before implementation | `diff_diff/lwdid.py` | LWDiD release audit | Low |
| HonestDiD rejects LWDiD event studies: `LWDiDResults.aggregate('event_study')` emits the unified EventStudyResults surface but the `honest_did.py` allowlist admits CS/Stacked sources only. Admission needs a methodology decision, not an allowlist edit: LWDiD pre-period cells are transformation-based placebos on the Appendix D control pools, and Rambachan-Roth requires those coefficients plus a joint pre/post vcov to stand as valid pre-trend evidence | `diff_diff/honest_did.py`, `diff_diff/lwdid_results.py` | LWDiD release audit | Low |
| HonestDiD non-chronological declared partitions (native `MultiPeriodDiDResults` route): the Rambachan-Roth restriction matrices are built POSITIONALLY over the concatenated declared pre/post lists assuming one chronological boundary, but the native route accepts non-suffix `post_periods` / non-last-pre references and returns bounds whose restriction system does not match the Registry equations (pre-existing; surfaced by the Phase 3(a) calendar-route review, which fails closed instead). Fix = transform the declared partition into boundary form where a valid mapping exists, else reject on the native route too - needs the restriction-geometry derivation. REGISTRY HonestDiD Note records the limitation. | `diff_diff/honest_did.py` | 3(a) | Medium |
| `PlaceboTests` `boundary_gap` — a permutation randomization-inference margin (SE-audit item (b)); NOT computed anywhere in code today, so this is a new feature + result field, not a coverage lock. **User-locked 2026-07-09: defer until a derivation/paper source exists** — do not design or implement from scratch. | `tests/test_methodology_placebo.py`, `diff_diff/diagnostics.py` | SE-audit | Low |
| TwoStageDiD honest/pretrends container admission DEFERRED (decision revised from "widen" during the 2(b) PR-3b plan review): analytical fits carry the joint Gardner-GMM event-study covariance (M-092), but the pre-period coefficients are stage-1 residual MEANS — the reference horizon is dropped from the no-intercept Stage-2 design and the zero anchor row appended mechanically — not contrasts against a reference period, while HonestDiD's Δ^RM/Δ^SD arithmetic hard-codes the `delta_0 = 0` normalization into its boundary/bridge constraints. Admission needs either a Stage-2 re-estimation with the reference horizon in the design or a derived residual-to-reference normalization mapping (+ its variance transform). Both consumers' TypeErrors state the deferral; see the REGISTRY TwoStageDiD Note (d). | `diff_diff/honest_did.py`, `diff_diff/pretrends.py`, `diff_diff/two_stage_aggregation.py` | 2(b) PR-3b | Low |
Expand Down Expand Up @@ -78,6 +80,7 @@ For survey-specific limitations (`NotImplementedError` paths), see the
|-------|----------|----|----------|
| RDDensityTest public bandwidth-selector helper (an `rdbwdensity`-equivalent exposed function) and a Tutorial-28 executed density-test demo cell (Act 4d; needs notebook re-execution + drift-suite updates): the selector ships internal-only and the tutorial carries a prose pointer for now. | `diff_diff/rddensity.py`, `docs/tutorials/28_rdd_scholarship_illusion.ipynb` | rddensity PR-B | Low |
| LW 2025 Sec 4.3 all-eventually-treated mode for LWDiD (drop `D_infinity`, effects relative to the last cohort as reference, last cohort's own effect not estimable). Current behavior (by decision, see the REGISTRY LWDiD Sec 4.3 note): such designs raise ValueError under both control strategies rather than silently truncating | `diff_diff/lwdid_staggered.py` | #588 | Low |
| LWDiD sampling weights / `survey_design=`: `fit()` accepts no weight argument on any path (passing `survey_design=` raises `TypeError`) — the LW papers derive the rolling transformation and the collapsed exact/HC/CR1 inference for unweighted panels, and no weighted counterpart of the transformation or exact-inference layer has been derived. Documented scope exclusion (REGISTRY LWDiD Edge-cases Note, `docs/api/lwdid.rst` Scope limitations note, survey-support matrix row, survey-roadmap Current Limitations row) | `diff_diff/lwdid.py` | LWDiD release audit | Low |
| EfficientDiD/ImputationDiD/TwoStageDiD kit retention has no memory opt-out: EfficientDiD's kit always retains the per-(g,t) EIF dict (M-023), ImputationDiD's references the `_fit_data` panel objects (M-021, pre-existing retention), and TwoStageDiD's retains a column-subset working-frame copy (M-022, the first new panel retention); a `store_kit=False` constructor knob would restore an opt-out at the cost of post-fit aggregation on that result | `diff_diff/efficient_did.py`, `diff_diff/imputation.py`, `diff_diff/two_stage.py` | 2(b) PR-3a | Low |
| MMM interop, StackedDiD totals (remainder of the estimator-owned-totals delivery - `aggregate('total')` shipped for CS/EfficientDiD/ImputationDiD/TwoStageDiD on panel non-survey fits): StackedDiD is STAGED OUT because the total estimand is ill-defined under `weighting="population"`/`"sample_share"` (a weighted estimand for which raw treated exposure is not the right mass), and its results class is view-only (a total would be its first computed level). Needs an estimand decision per weighting variant before any implementation. | `diff_diff/mmm.py`, `diff_diff/stacked.py` | mmm-interop | Low |
| `SyntheticControl` fit-snapshot residency (`_SyntheticControlFitSnapshot`) — **investigated 2026-07-07, parked**: the snapshot ALIASES the fit's own working pivots (zero extra construction cost); the retained residency implements the documented freeze contract (post-fit mutation of estimator inputs must not change `in_space_placebo()` / `leave_one_out()` / conformal output on an already-returned results object, and `__getstate__` already excludes it from pickles). A compact array representation saves only pandas overhead (the float panel dominates); releasing residency needs new API surface (`release`/opt-out flag) or a freeze-contract change. Revisit on user demand for very large donor panels. | `synthetic_control.py`, `synthetic_control_results.py` | follow-up | Low |
Expand Down
1 change: 1 addition & 0 deletions TODO.md
Original file line number Diff line number Diff line change
Expand Up @@ -86,6 +86,7 @@ generic sparse-FE, QR+SVD rank-detection redundancy, `check_finite` bypass — m
| Tighten the mypy suppressions that back the enforced-zero posture: burn down `prep_dgp`'s per-module `[index]` override (needs a None-vs-array restructure that preserves the seeded RNG stream), and evaluate re-enabling the globally disabled codes (`arg-type`, `return-value`, `var-annotated`, `assignment`) one at a time — `assignment` alone hid several real annotation drifts found during the 2026-07 triage. | `pyproject.toml` `[tool.mypy]`, `diff_diff/prep_dgp.py` | lint-CI | Mid | Low |
| Tracking-file contract guard test: reject NEW active deferred-work pointers at `TODO.md` (deferred rows live in `DEFERRED.md`; allowlist for historical/past-tense prose and actionable-row pointers) and assert rows cross-linking a `docs/v4-deprecations.yaml` `M-xxx` id don't restate ledger status. Origin: tracking-split local review R2. | `tests/`, `TODO.md`, `DEFERRED.md` | tracking-split | Quick | Low |
| Real-data CI canary for dataset-backed replication tests: `test_methodology_lwdid.py`'s Prop 99 / Walmart goldens skip (visibly) when loaders fall back to synthetic; add a lane or canary asserting `df.attrs["source"] == "lwdid_ssc_ancillary"` in CI so network regressions cannot silently de-gate the replication tests. Follow-on from the loader-fallback repair (#723), which made provenance explicit but deliberately did not add a network-dependent CI lane. | `tests/test_methodology_lwdid.py`, `.github/workflows/` | LWDiD validation suite | Quick | Low |
| Route LWDiD into the practitioner code surfaces: `practitioner_next_steps` (`diff_diff/practitioner.py`), `agent_workflow.py`, `DiagnosticReport`, and `BusinessReport` never dispatch to or mention LWDiD — the AI guides now cover it, but the code surfaces need contract decisions (when to recommend it, what diagnostics to run) plus tests | `diff_diff/practitioner.py`, `diff_diff/agent_workflow.py`, `diff_diff/diagnostic_report.py`, `diff_diff/business_report.py` | LWDiD release audit | Mid | Low |
| df-provenance completion follow-up (PR C descoping): a Wooldridge public per-row df channel (a per-key dict that can also carry the hc2_bm BM contrast dofs the scalar `_df_analytic_fallback` cannot represent — the unified surface's Wooldridge df column stays NaN until then); ImputationDiD lead-horizon `event_study_df` (needs the `_compute_lead_coefficients`/`_aggregate_event_study` return-path change plus bootstrap-clearing semantics for post rows); and the `inference_df` (DiD/MPD/SA/StackedDiD) vs `df_inference` (CS/dCDH) adapter-naming unification. NOTE (M-024): StackedDiD's `df_survey=None` on analytical fits is DELIBERATE - the container resolver's `df_inference` fallback not matching `inference_df` is load-bearing (normal-theory FLCI; see the REGISTRY StackedDiD M-024 Note and the pin in `tests/test_event_study_consumers.py`) - the unification must preserve that outcome or renegotiate it explicitly. | `diff_diff/wooldridge_results.py`, `diff_diff/imputation.py`, `diff_diff/results_base.py` | #variance-PR-C | Mid | Low |
| `worktree-rm` safety via a tested argv helper: the prose rewrite (ask-before-remove confirmation gate, detached-HEAD reachability/rescue, tip-identity force-delete guard) was reverted to the main version because editing the prose repeatedly reintroduced shell-injection (last: sourcing a state file built from a git-derived branch name). Restore those guards in a `worktree_rm.py` that takes the name via file ingress, invokes git through argv arrays, and has a metacharacter-branch/path injection regression test — the pattern that worked for `pr_prepare.py`/`premerge_scan.py`. | `.claude/commands/worktree-rm.md`, `.claude/scripts/` | skill-audit | Heavy | Medium |
| `premerge_scan.py` should scan the staged blob (`git show :path`) for staged methodology files, not the working-tree copy — a stage-then-revert-working-copy edit currently reads the safe working version and misses the staged violation. Union staged-index findings with unstaged/untracked filesystem findings. | `.claude/scripts/premerge_scan.py` | skill-audit | Mid | Low |
Expand Down
42 changes: 41 additions & 1 deletion diff_diff/guides/llms-autonomous.txt
Original file line number Diff line number Diff line change
Expand Up @@ -348,6 +348,7 @@ supported / out of scope; `warn` supported but with documented caveats;
| `TwoStageDiD` | ✓ | ✓ | ✗ | ✗ | ✗ | ✓ | ✗ | ✗ | ✓ |
| `StackedDiD` | ✓ | ✓ | ✗ | ✗ | ✗ | ✓ | ✗ | ✗ | ✓ |
| `WooldridgeDiD` (ETWFE) | ✓ | ✓ | ✗ | ✗ | ✗ | ✓ | ✗ | ✗ | ✓ |
| `LWDiD` | ✓ | ✓ | ✗ | ✗ | ✓ | warn | partial | ✗ | ✓ |
| `EfficientDiD` | ✓ | ✓ | ✗ | ✗ | partial | ✓ | ✗ | ✗ | ✓ |
| `SyntheticDiD` | ✓ | ✗ | ✗ | ✗ | ✓ | ✓ | ✓ | ✗ | partial |
| `TROP` | ✓ | ✓ | ✗ | ✗ | ✗ | ✗ | ✓ | ✗ | partial |
Expand All @@ -358,6 +359,18 @@ supported / out of scope; `warn` supported but with documented caveats;
| `RegressionDiscontinuity` | ✗ (cross-sectional; sharp: treatment = running >= cutoff; fuzzy: observed take-up via `takeup=`) | ✗ | ✗ | ✗ | ✗ | ✓ (precision only - estimand unchanged, unlike the DiD conditional-PT role; `covariates=`) | ✗ | ✗ | ✗ (follow-up) |

**Footnotes.**
- `LWDiD` + never-treated: requires at least one never-treated unit under
`control_group='not_yet_treated'` and at least two under
`'never_treated'`; all-eventually-treated designs raise under both
control strategies.
- `LWDiD` + covariate adjustment (`warn`): unit-constant covariates only;
time-varying covariate columns raise `ValueError` (the rolling
transformation collapses the panel to one row per unit).
- `LWDiD` + few-treated (`partial`): supports few or even a single treated
unit, but the mechanism is exact small-sample inference on the collapsed
regression (classical exact t / randomization inference, valid at one
treated unit) - NOT synthetic weighting; the column's synthetic-control
reading applies to `SyntheticDiD`/`TROP`.
- `TwoWayFixedEffects` + staggered: fits but mixes positive and negative
cohort-weights that violate the ATT interpretation; consult
`BaconDecomposition` to quantify. Prefer any staggered-robust
Expand Down Expand Up @@ -494,6 +507,12 @@ estimators:
under either `PT-All` or `PT-Post`; use `EfficientDiD.hausman_pretest`
to pick. Requires a balanced panel (`PanelProfile.is_balanced ==
True`); `fit()` raises `ValueError` on unbalanced input.
- `LWDiD` (Lee & Wooldridge 2025, 2026) - unit-specific rolling
demeaning/detrending converts the panel into per-(cohort, period)
cross-sectional problems. Control pool selectable
(`control_group='not_yet_treated'` default, or `'never_treated'`);
covariates must be unit-constant; the only staggered option with an
exact small-sample inference layer (see §4.8).

Diagnostic: `BaconDecomposition().fit(df, ...)` shows the weight allocation of a
TWFE fit to 2×2 comparison types. Forbidden-comparison weight > 10% is a
Expand Down Expand Up @@ -656,11 +675,32 @@ but derivable from `cohort_sizes` + `has_never_treated`):
unit untreated at period `t` as the donor pool (via the absorbing-state
D matrix); supports staggered adoption and more complex factor
structures. No covariate-adjustment surface on `fit()`.
- `LWDiD` - exact small-sample inference on the collapsed cross-sectional
regression: `vcov_type='classical'` (with `n_bootstrap=0`; a positive
`n_bootstrap` replaces the common-timing headline inference with the
unit/cluster bootstrap) gives an exact t reference
distribution valid down to a single treated unit. Causal only under the
LWDiD identifying assumptions (no anticipation + parallel trends - the
heterogeneous-linear-trends variant for `rolling='detrend'` - +
overlap) and exact only under the classical error assumptions; the
canonical contract (which surfaces get the exact t, covariate designs)
is the LWDiD section of `get_llm_guide("full")` / the REGISTRY "LWDiD"
section. Common-timing `estimation_method='reg'` fits offer post-fit
`randomization_test()` (any such fit) and `wild_cluster_bootstrap()`
(additionally requires a `cluster=` fit; unclustered fits raise).
Randomization inference relaxes the classical error assumptions, but
it tests Fisher's SHARP null and is valid only when the observed
treatment assignment is exchangeable under a credible assignment
mechanism (the implementation permutes treatment labels) - in
observational settings without that justification, the permutation
p-value is not valid inference.

Classical DiD estimators will still produce estimates, but inference is
unreliable with very small treated groups; cluster-robust SE relies on
the number of clusters, not the number of treated units. Bootstrap
methods in the library are preferred.
methods in the library are one mitigation; `LWDiD`'s exact t /
randomization inference is the direct small-N answer when its classical
assumptions are acceptable.

### §4.9 Heterogeneous adoption intensity

Expand Down
Loading
Loading