ce-compound-refresh's Cross-Doc Conflict Check (SKILL.md, "Look for outright contradictions between docs in scope") only compares documents inside docs/solutions/ with each other. In an agent-oriented repo, that misses the contradictions that actually cause damage.
What happened
An agent opened a PR changing a rendered artifact and handed it over for review without deploying it anywhere the reviewer could look at it. The rule against exactly that was documented, and the agent had read it earlier in the same session — it had quoted commands out of that file and run them successfully three times.
The rule still didn't fire, because two other live sources taught the opposite:
- a repo-local skill — the one an agent loads at the moment it does this kind of work — described the wrong order with no review step at all
- a runbook described the wrong order too, and explicitly labelled the correct behaviour "the classic trap"
The convention document was right. It was outvoted 2-to-1 by the guidance an agent actually reaches for while working.
Worse: the conflict had already been found. A spec seven days earlier named both offending sources, ratified the correct order, and wrote "the merge-first passage is deleted, not softened." The deletion was then listed under "Follow-ups (not in this PR)" and never made.
Why the current check can't catch this
Every one of those documents is internally consistent and none has a broken reference. A path-existence sweep comes back clean. A docs-only conflict check comes back clean, because the contradiction isn't between two learnings — it's between a learning and the instruction layer: skills, runbooks, AGENTS.md/CLAUDE.md.
That layer is not incidental in these repos. It is the guidance an agent loads at the moment of acting, so when it disagrees with a documented convention, the instruction layer wins by default. That makes this failure worse than ordinary staleness: a document that disagrees with the code is out of date, but a document that disagrees with the skill an agent loads is actively steering behaviour the wrong way.
Suggestion
Extend the conflict check past docs/solutions/ to the agent-facing guidance layer — repo-local skills, runbooks, and root instruction files. Even a narrow version would have caught this: for each convention in scope, check whether any skill or runbook covering the same procedure states a different order or a contradictory rule.
Two smaller notes from the same run:
ce-compound already does a weak version of this, and it was the single most valuable output of the run. The Related Docs Finder's stale/contradicted section is what surfaced the whole thing — it found the two contradicting sources and the unshipped ratification. But that's emergent from one subagent's broad remit rather than a named step, so it isn't guaranteed and isn't reproducible. It's worth promoting to a first-class check in both skills.
Consider an optional enforced_by: frontmatter field. In this corpus, 14 of 18 conventions named no automated check. The repo had separately measured its own compliance: rules with a check behind them were followed ~100% of the time, rules living only as prose about 41%. If a document can record what enforces it, "which documents are most likely to have drifted" becomes a query rather than a guess — and it gives the refresh a real prior when deciding between Keep, Replace, and Delete.
Version: verified against HEAD via the contents API before filing.
ce-compound-refresh's Cross-Doc Conflict Check (SKILL.md, "Look for outright contradictions between docs in scope") only compares documents insidedocs/solutions/with each other. In an agent-oriented repo, that misses the contradictions that actually cause damage.What happened
An agent opened a PR changing a rendered artifact and handed it over for review without deploying it anywhere the reviewer could look at it. The rule against exactly that was documented, and the agent had read it earlier in the same session — it had quoted commands out of that file and run them successfully three times.
The rule still didn't fire, because two other live sources taught the opposite:
The convention document was right. It was outvoted 2-to-1 by the guidance an agent actually reaches for while working.
Worse: the conflict had already been found. A spec seven days earlier named both offending sources, ratified the correct order, and wrote "the merge-first passage is deleted, not softened." The deletion was then listed under "Follow-ups (not in this PR)" and never made.
Why the current check can't catch this
Every one of those documents is internally consistent and none has a broken reference. A path-existence sweep comes back clean. A docs-only conflict check comes back clean, because the contradiction isn't between two learnings — it's between a learning and the instruction layer: skills, runbooks,
AGENTS.md/CLAUDE.md.That layer is not incidental in these repos. It is the guidance an agent loads at the moment of acting, so when it disagrees with a documented convention, the instruction layer wins by default. That makes this failure worse than ordinary staleness: a document that disagrees with the code is out of date, but a document that disagrees with the skill an agent loads is actively steering behaviour the wrong way.
Suggestion
Extend the conflict check past
docs/solutions/to the agent-facing guidance layer — repo-local skills, runbooks, and root instruction files. Even a narrow version would have caught this: for each convention in scope, check whether any skill or runbook covering the same procedure states a different order or a contradictory rule.Two smaller notes from the same run:
ce-compoundalready does a weak version of this, and it was the single most valuable output of the run. The Related Docs Finder's stale/contradicted section is what surfaced the whole thing — it found the two contradicting sources and the unshipped ratification. But that's emergent from one subagent's broad remit rather than a named step, so it isn't guaranteed and isn't reproducible. It's worth promoting to a first-class check in both skills.Consider an optional
enforced_by:frontmatter field. In this corpus, 14 of 18 conventions named no automated check. The repo had separately measured its own compliance: rules with a check behind them were followed ~100% of the time, rules living only as prose about 41%. If a document can record what enforces it, "which documents are most likely to have drifted" becomes a query rather than a guess — and it gives the refresh a real prior when deciding between Keep, Replace, and Delete.Version: verified against HEAD via the contents API before filing.