forked from apache/spark
-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
[SPARK-47553][SS] Add Java support for transformWithState operator APIs
### What changes were proposed in this pull request? Add Java support for transformWithState operator APIs ### Why are the changes needed? To add support for using transformWithState operator in Java ### Does this PR introduce _any_ user-facing change? Yes ### How was this patch tested? Added unit tests ``` [info] Test test.org.apache.spark.sql.JavaDatasetSuite#testGroupByKey() started [info] Test test.org.apache.spark.sql.JavaDatasetSuite#testCollect() started [info] Test test.org.apache.spark.sql.JavaDatasetSuite#testKryoEncoderErrorMessageForPrivateClass() started [info] Test test.org.apache.spark.sql.JavaDatasetSuite#testJavaBeanEncoder() started [info] Test test.org.apache.spark.sql.JavaDatasetSuite#testTupleEncoder() started [info] Test test.org.apache.spark.sql.JavaDatasetSuite#testPeriodEncoder() started [info] Test test.org.apache.spark.sql.JavaDatasetSuite#testRowEncoder() started [info] Test test.org.apache.spark.sql.JavaDatasetSuite#testNestedTupleEncoder() started [info] Test test.org.apache.spark.sql.JavaDatasetSuite#testTupleEncoderSchema() started [info] Test test.org.apache.spark.sql.JavaDatasetSuite#testMappingFunctionWithTestGroupState() started [info] Test test.org.apache.spark.sql.JavaDatasetSuite#testReduce() started [info] Test test.org.apache.spark.sql.JavaDatasetSuite#testSelect() started [info] Test test.org.apache.spark.sql.JavaDatasetSuite#testInitialStateFlatMapGroupsWithState() started [info] Test test.org.apache.spark.sql.JavaDatasetSuite#testJavaEncoderErrorMessageForPrivateClass() started [info] Test run finished: 0 failed, 0 ignored, 45 total, 14.73s [info] Passed: Total 45, Failed 0, Errors 0, Passed 45 [success] Total time: 20 s, completed Mar 28, 2024, 12:37:30 PM ``` ### Was this patch authored or co-authored using generative AI tooling? No Closes apache#45758 from anishshri-db/task/SPARK-47553. Authored-by: Anish Shrigondekar <anish.shrigondekar@databricks.com> Signed-off-by: Jungtaek Lim <kabhwan.opensource@gmail.com>
- Loading branch information
1 parent
a427a45
commit 1515c56
Showing
7 changed files
with
479 additions
and
20 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.