Add Java SDK capability manifest and compatibility matrix - #71151
Open
jason810496 wants to merge 3 commits into
Open
Add Java SDK capability manifest and compatibility matrix#71151jason810496 wants to merge 3 commits into
jason810496 wants to merge 3 commits into
Conversation
The Conformance section of the Language SDK guide defines which TaskInstance states and capabilities an SDK may declare, but nothing lets an SDK state what it actually supports. A hand-maintained table in each SDK's docs drifts from its runtime the moment a capability lands, and readers cannot tell whether a gap is real or just stale documentation. Give every Language SDK one validated schema to declare its support in, and one renderer that turns that declaration into a table, so a published matrix is derived from the SDK rather than transcribed from it. Conditional native-Dag capabilities render as "not applicable" rather than unsupported when the SDK does not author native Dags, matching how the spec gates them.
Readers of the Java SDK docs have no way to tell which parts of the Language SDK conformance spec the runtime implements. The gaps are real and moving — native Dag authoring, deferral, and the state stores are not there yet — so an unqualified "see the conformance spec" overstates what a task can do today. Let the SDK declare its own support in Kotlin next to the code that implements it, and generate the tables in the README and the published Dokka API reference from that declaration. Prek regenerates both and fails on drift, so the matrix cannot go stale without CI noticing, and a capability landing in the runtime is a one-line edit rather than three tables to update by hand.
jason810496
requested review from
amoghrajesh,
ashb,
bugraoz93,
gopidesupavan,
jscheffl,
potiuk and
uranusjr
as code owners
August 5, 2026 05:53
boring-cyborg
Bot
added
area:dev-tools
area:java-sdk
backport-to-v3-3-test
The manifest model sat at default (public) visibility in the stable org.apache.airflow.sdk.conformance package, so a type that exists purely to emit a build-time JSON file shipped in the published artifact and in the Dokka API reference. That would make every later change to the manifest schema a source and binary compatibility question for SDK users, for no benefit — nothing outside the Gradle dump task consumes it. Marking it internal removes it from both surfaces, and the generically named Document becomes CapabilityManifest now that the name is no longer public. The rest hardens the surrounding tooling against ways it could mislead: the sync hook now re-triggers when its own implementation changes, so editing the checker cannot leave the manifest unverified; an unsupported dimension carrying a "since" version is rejected rather than silently rendering as unversioned; the manifest is decoded as the first complete JSON value instead of slicing to the last brace, which mis-parses if Gradle ever frames the output; and the JDK gate fails closed when CI is set but empty. The Dokka column-width rules now only raise a minimum width rather than forbidding wrapping, so they cannot cause overflow in unrelated tables that a Markdown-generated table gives no way to exclude.
phanikumv
reviewed
Aug 5, 2026
| // Keep in sync with airflowSupervisorSchemaVersion in java-sdk/gradle.properties. | ||
| private const val SUPERVISOR_SCHEMA_VERSION = "2026-06-16" | ||
|
|
||
| private fun yes(note: String = ""): CapabilityEntry = CapabilityEntry(supported = true, since = MIN_AIRFLOW_VERSION, note = note) |
Contributor
There was a problem hiding this comment.
Suggested change
| private fun yes(note: String = ""): CapabilityEntry = CapabilityEntry(supported = true, since = MIN_AIRFLOW_VERSION, note = note) | |
| private fun yes(note: String = "", since: String = MIN_AIRFLOW_VERSION): CapabilityEntry = | |
| CapabilityEntry(supported = true, since = since, note = note) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
The Language SDK conformance spec (#70717) defines which TaskInstance states and capabilities a Language SDK may declare, but nothing lets an SDK say what it actually supports.
This PR makes that support machine-readable and generates the docs from it.
How
Both hooks regenerate their target and exit non-zero when it was stale, so a drifted table fails the
build and the fix is to re-stage the regenerated file. A capability landing in the runtime becomes a
one-line edit to the Kotlin constant instead of three tables to update by hand.
Verification
Was generative AI tooling used to co-author this PR?
Generated-by: Claude Code (Opus 4.8) following the guidelines