fix(server): close shortest path edge iterators - #3167
Conversation
Close the original backend iterator on forward, backward, and super-node traversal paths so early returns cannot leave HStore scanners active. Add focused iterator-lifecycle regression coverage. Fixes apache#3166
274433e to
1a08017
Compare
imbajin
left a comment
There was a problem hiding this comment.
Blocking: yes. Summary: The new cleanup covers iterators after acquisition, but multi-label acquisition can still leak already-opened scanners, and the regression tests do not exercise the threshold, wrapper, or exception paths. Evidence: exact-head ShortestPathTraverser/HugeTraverser control flow, TinkerPop CloseableIterator behavior, and the added ShortestPathTraverserTest.
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## master #3167 +/- ##
============================================
- Coverage 41.06% 36.15% -4.92%
+ Complexity 519 426 -93
============================================
Files 771 771
Lines 65962 65987 +25
Branches 8766 8767 +1
============================================
- Hits 27088 23857 -3231
- Misses 36008 39466 +3458
+ Partials 2866 2664 -202 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
Codecov status clarificationThe reported 4.92 percentage-point project coverage drop is caused by incomplete coverage aggregation for the PR head, rather than by this PR removing coverage from thousands of existing lines.
The Codecov steps for Commons, PD, Store, HStore, Server RocksDB, and Server HBase all found coverage files but then logged an empty token and HTTP 429 rate-limit errors. The uploader exits with status 0 by default, so the corresponding GitHub Actions jobs still appear green. Only the Server memory upload was accepted. Relevant logs: Accordingly, This does not remove the need to review exception-path coverage independently. The new Related repository-level JaCoCo aggregation work is tracked in #3160 and #3161. The anonymous-upload 429 failures across the other workflows are additional CI infrastructure evidence, so I am not changing the functional implementation solely in response to this incomplete Codecov report. |
Purpose of the PR
ShortestPathTraversercould return after finding a path without closing the current backend edge iterator. With HStore, the abandoned iterator could leave its remote scanner active and Store threads waiting to deliver pages that the client no longer consumed.The same ownership gap also existed in the backward search and in the iterator used by the super-node probe.
Main Changes
edgesOfVertex()in bothforward()andbackward().finally, including early-return and exception paths. The source iterator is closed directly becauseskipSuperNodeIfNeeded()may consume and replace it.superNode()after counting it.UnitTestSuite.The change does not modify traversal semantics, degree/capacity limits, request timeouts, HStore code, configuration, or public APIs.
Verifying these changes
apache/master, the new test preserves the expected path results and fails all three iterator-close assertions:Tests run: 3, Failures: 3.ShortestPathTraverserTestpasses:OK (3 tests).MAVEN_OPTS='-Xms64m -Xmx256m' mvn clean compile -pl hugegraph-server/hugegraph-test -am -Dmaven.javadoc.skip=truecompletes withBUILD SUCCESSfor all 21 affected reactor modules.mvn editorconfig:formatcompletes withBUILD SUCCESS, andgit diff --checkreports no errors.Additional HStore runtime validation of the same production patch used one PD, one Store, and one Server with the full SNAP Twitter-2010 graph (41,652,230 vertices and 1,468,365,182 directed edges). The bounded shortest-path gate completed 1,516/1,516 warmup and 5,791/5,791 measured requests with zero errors, zero swap, stable service PIDs, and zero blocked
HgChannel.sendframes at all sampled points through 600 seconds.Does this PR potentially affect the following parts?
Documentation Status
Doc - TODODoc - DoneDoc - No Need