Dart: extract dartdoc instead of discarding it

[rod-moraes]

The gap

extract_dart() strips comments before any extraction pass runs:

comment_string_pattern = re.compile(... r"|//[^\n]*")

def _comment_replace(match):
    token = match.group(0)
    if token.startswith("/"):   # `///` lands here too
        return ""

/// is a special case of //, so every dartdoc comment in a Dart corpus is deleted before extraction starts. For a language whose convention puts the "what is this and how do you use it" statement in exactly that place, that is the single richest human-authored signal in the file, thrown away in the first ten lines.

Measured on Flutter's own lib/src/material (182 files):

/// doc blocks	14,358
with real prose	13,869 (97%)
{@tool} blocks naming a runnable example	155
{@macro} / {@template}	714 / 224

All of it currently goes to "".

What this PR does

Collect doc blocks from the raw source before stripping (the existing passes keep operating on the comment-free text, unchanged), then bind each block to what it documents and turn the curated parts into edges.

Binding

A block attaches to the first line below it that is not more documentation — skipping blank lines, annotations and plain // comments — at that declaration's own granularity:

Block sits above	Doc binds to
library;	the file node
class / mixin / enum / extension / typedef	that type
a constructor	that constructor
a constructor parameter	the field this.x forwards to, or the parameter itself
anything else	that member (field, method, getter, …)

Constructors and documented parameters get nodes because nothing else mints them: the method pass skips every name starting uppercase, and parameter lists are never walked. They are created only where a doc block points at one, so the cost tracks the documentation rather than every constructor in the corpus.

• Foo() / Foo.named() labels keep an unnamed constructor from colliding with its class.
• Foo._() gets an ID that does not normalize onto the class node — or, for the bare _, onto the file node (Dart private named constructor (Foo._()) collapses to an empty entity in make_id, colliding with the file node and disabling _is_file_node #2738).
• Enclosing type comes from brace depth, the parameter list from paren depth, both counted with string literals and trailing comments blanked — so Padding( inside a build method is read as a call, not a constructor declaration.

Node attribute

doc = the block's full prose, every paragraph, joined by blank lines, with directives removed, inline HTML stripped, [refs] unwrapped and control characters dropped (#2897). Not truncated: a consumer that wants the one-line summary takes doc.split("\n\n")[0], which is dartdoc's own convention for the first paragraph.

Splitting long docs across nodes was the alternative and is worse for this shape of data — the median block is a single paragraph, so it would mint ~14k nodes to hold one paragraph each.

Edges

Source	Relation	Context
class → its documented constructor	contains	dartdoc_constructor
constructor → its documented parameter	references	dartdoc_parameter
See also: entries	references	dartdoc_see_also
** See code in <path> **	references	dartdoc_sample
{@template id}	defines	dartdoc_template
{@macro id}	references	dartdoc_macro

{@template}/{@macro} is dartdoc's own transclusion; linking the two ends makes reused documentation traversable across files and packages.

On FloatingActionButton:

doc (1,467 chars, 4 paragraphs): "A Material Design floating action button.\n\nA floating action button is a circular icon button that hovers over content to promote..."
contains   dartdoc_constructor -> FloatingActionButton()
contains   dartdoc_constructor -> FloatingActionButton.small()
contains   dartdoc_constructor -> FloatingActionButton.large()
contains   dartdoc_constructor -> FloatingActionButton.extended()
references dartdoc_see_also    -> scaffold
references dartdoc_see_also    -> elevatedbutton
references dartdoc_sample      -> examples_api_lib_material_floating_action_button_floating_action_button_0_dart

What it deliberately does not do

Inline [Foo] mentions in prose are parsed but not emitted as edges. On src/material they add ~6,600 edges (+16%) that mostly restate relations the AST passes already found, and they pile onto hub types ([ThemeData] alone appears in 423 blocks). Easy to add behind a flag — the parser already returns them.

Doc-derived edges use the generic references relation, so _GENERIC_RELATIONS in build.py keeps them from ever downgrading a calls/inherits on the same pair, and the dartdoc_* context keeps a doc-stated relation distinguishable from one proven by code — the confusion #2270 is about.

Cost

	before	after
src/material nodes	23,353	25,207 (+7.9%)
src/material edges	41,505	44,393 (+7.0%)
nodes carrying a doc	0	13,042 (1.67 MB of text)
extraction time	3.05s	3.47s

src/widgets (186 files) for a second data point: 737 documented constructors, 1,709 see-also edges, 585 macro links.

Notes / limits

• /// only. Legacy /** */ dartdoc is still stripped — zero occurrences in src/material, so it didn't seem worth the surface. Say the word and I'll add it.
• {@macro} is linked, not expanded. Resolving the text needs a global {@template} index across packages (many Material macros resolve into src/widgets or dart:ui), which is a bigger change than one per-file extractor. Linking the two ends gets the graph the relation without that pass.
• Docs are keyed by the label the node will carry, matching how this extractor already mints IDs. A label declared twice in one file (build in two widget classes) already collapses to one node, so the first block wins. add_node now also fills in a doc on a node an earlier pass created under a differently-normalized label — IDs strip leading underscores, so a private field _field and a parameter field are one node, and without this the second label's doc was silently dropped (hit for real in cloud_firestore/filters.dart).
• Flutter itself never documents constructor parameters (0 occurrences across src/material and src/widgets — it documents the fields instead), so that path is exercised by unit tests and verified against cloud_firestore/filters.dart, which does.
• External reference nodes (source_file=None) never inherit a same-named local symbol's doc.

Testing

• 10 tests in tests/test_dart.py covering: docs on file/class/constructor/parameter/field/method nodes, full multi-paragraph text with no truncation, undocumented declarations staying clean, all six edge kinds, Foo._() not collapsing onto the class, a constructor call not being read as a declaration, // comments between doc and declaration, the normalized-ID collision, the lowercase-See also resolution rule, docs not leaking onto external nodes, directive-only blocks, and HTML stripping vs. <https://…> autolink preservation.
• Full suite: 4890 passed, 11 skipped on 3.10 with --all-extras.
• ruff check, pyright, and python -m tools.skillgen --check all clean.
• End-to-end graphify <dir> --code-only confirms doc and every dartdoc_* edge survive into graph.json.

🤖 Generated with Claude Code

[graphify-labs]

Graphify reviewed this change.

Worth a look — the grounded gate found no coupling regressions or blocking issues, but 1 advisory finding(s) below merit a look before merge.

Formal verification. No changes could be formally verified in this run.

---

Graphify review — findings

Adds dartdoc (///) extraction to the Dart extractor via new _parse_dartdoc and _collect_dartdoc helpers, attaching a bounded lead-paragraph doc summary to file and declaration nodes in extract_dart. Recovers dartdoc from raw source before comment stripping, unwrapping [refs], stripping HTML/directives, and guarding external (source_file=None) nodes from inheriting a local symbol's doc on name collision. Emits curated cross-reference edges (See also:, {@tool} samples, {@template}/{@macro} transclusion) tagged dartdoc_*, while deliberately skipping inline [Foo] mentions.

Worth a look

• Dartdoc is lost when metadata annotation spans multiple lines — graphify/extractors/dart.py:146 · Escalate · medium
  • agreed by 2 of 2 members but NOT verified (no proof, no reproducing execution) — consensus is not a verdict; needs human review

Analysis details — impact, health, verification

Impact & health

Graphify review

Impact — 39 functions depend on the 38 functions this change touches.

Health — this change adds coupling hotspots:

• new: extract_dart() — 11 callers, 8 callees

Verification — 39 functions in the blast radius were not formally verified this run (proofs are advisory here).

Gate & verification

graphify gate

PASS — objectively clean (no health regressions, tests not run — proofs not run this pass (advisory)). Grounded, not self-assessed.

Advisory (not blocking):

• verification_scope: 39 function(s) in the blast radius were not formally verified this run

Formal verification

Could not verify: Could not verify extract\_dart.

The verifier did not have enough to check extract\_dart, so it is saying so rather than guessing. No false assurance is the whole point.

Guarantee: No guarantee either way, this is an honest abstention, not a pass.

Note: Reason: parameter `path` is annotated `Path` — outside the synthesizable primitive/collection set

· 1 grounded finding(s) anchored inline below.

[graphify-labs]

⚠️ Health regression — extract_dart()

fans out to 8 callees (efferent coupling); 11 callers depend on it (afferent coupling).

Grounded coupling-delta finding (deterministic), not an LLM guess.

[graphify-labs]

Graphify reviewed this change.

Worth a look — the grounded gate found no coupling regressions or blocking issues, but 3 advisory finding(s) below merit a look before merge.

Formal verification. No changes could be formally verified in this run.

---

Graphify review — findings

Adds dartdoc (///) extraction to the Dart extractor: new _parse_dartdoc, _dartdoc_structure, _collect_dartdoc, constructor label/key helpers, and doc-node/edge wiring recover comment prose from raw source before the comment-stripping pass runs, binding each block to the file, type, constructor, forwarded field/parameter, or member it documents. Extends extract_dart to attach docs in add_node, emit add_dartdoc_edges, and surface see_also/samples/templates/macros. Covers behavior with new test_dart.py cases for binding, truncation, HTML/ref stripping, directive-only blocks, ID collisions, and constructor-vs-call disambiguation.

Worth a look

• add_node overwrites/misses node_by_id for pre-existing external nodes — graphify/extractors/dart.py · Escalate · medium
  • agreed by 2 of 2 members but NOT verified (no proof, no reproducing execution) — consensus is not a verdict; needs human review
• Function-typed fields bind dartdoc to Function instead of the field — graphify/extractors/dart.py:45 · Escalate · medium
  • agreed by 2 of 2 members but NOT verified (no proof, no reproducing execution) — consensus is not a verdict; needs human review
• Multi-line annotations stop dartdoc from reaching the declaration — graphify/extractors/dart.py:250 · Escalate · medium
  • agreed by 2 of 2 members but NOT verified (no proof, no reproducing execution) — consensus is not a verdict; needs human review

Analysis details — impact, health, verification

Impact & health

Graphify review

Impact — 53 functions depend on the 52 functions this change touches.

Health — this change adds coupling hotspots:

• new: extract_dart() — 15 callers, 9 callees

Verification — 53 functions in the blast radius were not formally verified this run (proofs are advisory here).

Gate & verification

graphify gate

PASS — objectively clean (no health regressions, tests not run — proofs not run this pass (advisory)). Grounded, not self-assessed.

Advisory (not blocking):

• verification_scope: 53 function(s) in the blast radius were not formally verified this run

Formal verification

Could not verify: Could not verify extract\_dart.

The verifier did not have enough to check extract\_dart, so it is saying so rather than guessing. No false assurance is the whole point.

Guarantee: No guarantee either way, this is an honest abstention, not a pass.

Note: Reason: parameter `path` is annotated `Path` — outside the synthesizable primitive/collection set

· 1 grounded finding(s) anchored inline below.

[graphify-labs]

⚠️ Health regression — extract_dart()

fans out to 9 callees (efferent coupling); 15 callers depend on it (afferent coupling).

Grounded coupling-delta finding (deterministic), not an LLM guess.