Skip to content

Commit 8f1b06a

Browse files
Make bisect_ppx actually instrument the library — coverage was measuring nothing (#713)
The coverage job runs, passes, and **measures nothing**. ```console $ dune runtest --force --instrument-with bisect_ppx 537 tests OK, exit 0 $ bisect-ppx-report summary Error: no *.coverage files found ``` Zero `.coverage` files are written anywhere in the tree. ## Root cause `lib/dune` declares `(preprocess (pps …))` but **no instrumentation backend**. `--instrument-with bisect_ppx` only instruments libraries that opt in with an `(instrumentation (backend bisect_ppx))` stanza. Without one, dune instruments nothing, the tests run uninstrumented, and there is no coverage data to report. ## Why nobody noticed `.github/workflows/ci.yml:213` sets **`continue-on-error: true`** on the step that calls `bisect-ppx-report`. So the report failed on every run, the failure was swallowed, and the coverage job went green while producing no data — the same hollow-gate shape as a test that passes without running. The `continue-on-error` is **left in place**: it's defensible for a visibility-only artefact step, and with this fix the step now has something to report. Worth revisiting separately if coverage ever becomes gating. ## After the fix | | before | after | |---|---|---| | `.coverage` files | **0** | 2 | | Coverage | *unmeasurable* | **8103/16243 (49.89%)** | That number has never been measurable before, so treat it as a **baseline** — not a regression, and not an improvement. ## What it says needs attention Core paths are reasonably covered: | Module | Coverage | | |---|---|---| | `parser.ml` | 75.64% | 969/1281 | | `quantity.ml` | 79.28% | 241/304 | | `typecheck.ml` | 68.37% | 1042/1524 | | `borrow.ml` | 68.03% | 800/1176 | | `resolve.ml` | 59.21% | 405/684 | | `lexer.ml` | 58.33% | 49/84 | The gaps are concentrated in **backends and tooling**, not the front end: | Module | Coverage | | |---|---|---| | **`wasm_gc.ml`** | **0.00%** | **0/178** ← entirely unexercised | | `wasm.ml` | 0.24% | 1/423 | | `types.ml` | 9.95% | 19/191 | | `lean_codegen.ml` | 10.48% | 13/124 | | `value.ml` | 13.13% | 26/198 | | `c_codegen.ml` | 14.60% | 67/459 | | `lsp_server.ml` | 15.47% | 43/278 | `wasm_gc.ml` at **0/178** is the standout — 178 points, none reached by any of the 537 tests. ## Two things I checked and did *not* change **The 32 `[SKIP]`s in `res-to-affine-walker` are correct behaviour, not a fake gate.** `test_walker.ml:65` skips with an actionable message when the tree-sitter CLI isn't on PATH, and CI *does* install it (ci.yml:59–68) — so those tests run in CI and skip only locally. Good design; left alone. **The suite is otherwise healthy:** 537 tests run, 0 fail. Better than the estate norm. ## Scope One file changed: `lib/dune`. 🤖 Generated with [Claude Code](https://claude.com/claude-code)
2 parents 707fd5f + 0f23e41 commit 8f1b06a

1 file changed

Lines changed: 9 additions & 1 deletion

File tree

lib/dune

Lines changed: 9 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -114,7 +114,15 @@
114114
(flags
115115
(:standard -w -8-9))
116116
(preprocess
117-
(pps ppx_deriving.show ppx_deriving.eq ppx_deriving.ord sedlex.ppx)))
117+
(pps ppx_deriving.show ppx_deriving.eq ppx_deriving.ord sedlex.ppx))
118+
; Without this stanza `dune runtest --instrument-with bisect_ppx` has nothing
119+
; to attach to: dune instruments nothing, the tests run uninstrumented, and no
120+
; .coverage files are written. `bisect-ppx-report summary` then fails with
121+
; "no *.coverage files found" — and because the CI step that calls it carries
122+
; `continue-on-error: true`, that failure was swallowed and the coverage job
123+
; went green while measuring nothing.
124+
(instrumentation
125+
(backend bisect_ppx)))
118126

119127
(menhir
120128
(modules parser)

0 commit comments

Comments
 (0)