Skip to content

Commit

Permalink
Update for 0.18.0, move CICD configs to supported Spark versions (#680)
Browse files Browse the repository at this point in the history
  • Loading branch information
srowen authored Apr 10, 2024
1 parent c42d6bc commit ddd1ef5
Show file tree
Hide file tree
Showing 5 changed files with 13 additions and 13 deletions.
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
name: Spark 3.2 / Java 8 / Scala 2.12
name: Spark 3.3 / Java 8 / Scala 2.12
on:
push:
branches: [master]
Expand All @@ -14,4 +14,4 @@ jobs:
with:
java-version: 'adopt@1.8'
- name: Build and test
run: sbt -Dspark.testVersion=3.2.4 ++2.12.15 clean mimaReportBinaryIssues test
run: sbt -Dspark.testVersion=3.3.4 ++2.12.15 clean mimaReportBinaryIssues test
4 changes: 2 additions & 2 deletions .github/workflows/test_spark_3_4_java_11.yml
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
name: Spark 3.4 / Java 11 / Scala 2.13
name: Spark 3.4 / Java 11 / Scala 2.12
on:
push:
branches: [master]
Expand All @@ -14,4 +14,4 @@ jobs:
with:
java-version: 'adopt@1.11'
- name: Build and test
run: sbt -Dspark.testVersion=3.4.1 ++2.13.8 clean scalastyle test:scalastyle mimaReportBinaryIssues test
run: sbt -Dspark.testVersion=3.4.1 ++2.12.15 clean mimaReportBinaryIssues test
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
name: Spark 3.3 / Java 11 / Scala 2.13
name: Spark 3.5 / Java 11 / Scala 2.13
on:
push:
branches: [master]
Expand All @@ -14,4 +14,4 @@ jobs:
with:
java-version: 'adopt@1.11'
- name: Build and test
run: sbt -Dspark.testVersion=3.3.3 ++2.13.8 clean mimaReportBinaryIssues test
run: sbt -Dspark.testVersion=3.5.1 ++2.13.8 clean scalastyle test:scalastyle mimaReportBinaryIssues test
8 changes: 4 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,15 +16,15 @@ You can link against this library in your program at the following coordinates:
```
groupId: com.databricks
artifactId: spark-xml_2.12
version: 0.17.0
version: 0.18.0
```

## Using with Spark shell

This package can be added to Spark using the `--packages` command line option. For example, to include it when starting the spark shell:

```
$SPARK_HOME/bin/spark-shell --packages com.databricks:spark-xml_2.12:0.17.0
$SPARK_HOME/bin/spark-shell --packages com.databricks:spark-xml_2.12:0.18.0
```

## Features
Expand Down Expand Up @@ -410,7 +410,7 @@ Automatically infer schema (data types)
```R
library(SparkR)

sparkR.session("local[4]", sparkPackages = c("com.databricks:spark-xml_2.12:0.17.0"))
sparkR.session("local[4]", sparkPackages = c("com.databricks:spark-xml_2.12:0.18.0"))

df <- read.df("books.xml", source = "xml", rowTag = "book")

Expand All @@ -422,7 +422,7 @@ You can manually specify schema:
```R
library(SparkR)

sparkR.session("local[4]", sparkPackages = c("com.databricks:spark-xml_2.12:0.17.0"))
sparkR.session("local[4]", sparkPackages = c("com.databricks:spark-xml_2.12:0.18.0"))
customSchema <- structType(
structField("_id", "string"),
structField("author", "string"),
Expand Down
6 changes: 3 additions & 3 deletions build.sbt
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ import com.typesafe.tools.mima.core.MissingClassProblem

name := "spark-xml"

version := "0.17.0"
version := "0.18.0"

organization := "com.databricks"

Expand All @@ -12,7 +12,7 @@ crossScalaVersions := Seq("2.12.15", "2.13.8")

scalacOptions := Seq("-unchecked", "-deprecation")

val sparkVersion = sys.props.get("spark.testVersion").getOrElse("3.4.1")
val sparkVersion = sys.props.get("spark.testVersion").getOrElse("3.5.1")

// To avoid packaging it, it's Provided below
autoScalaLibrary := false
Expand Down Expand Up @@ -81,7 +81,7 @@ fork := true
// Prints JUnit tests in output
Test / testOptions := Seq(Tests.Argument(TestFrameworks.JUnit, "-v"))

mimaPreviousArtifacts := Set("com.databricks" %% "spark-xml" % "0.16.0")
mimaPreviousArtifacts := Set("com.databricks" %% "spark-xml" % "0.17.0")

mimaBinaryIssueFilters ++= {
Seq()
Expand Down

0 comments on commit ddd1ef5

Please sign in to comment.