Skip to content

Commit

Permalink
Release prep
Browse files Browse the repository at this point in the history
  • Loading branch information
edgararuiz committed Jun 24, 2024
1 parent f0c3a41 commit 39748e1
Show file tree
Hide file tree
Showing 3 changed files with 25 additions and 26 deletions.
2 changes: 1 addition & 1 deletion DESCRIPTION
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
Package: pysparklyr
Title: Provides a 'PySpark' Back-End for the 'sparklyr' Package
Version: 0.1.4.9004
Version: 0.1.5
Authors@R: c(
person("Edgar", "Ruiz", , "edgar@posit.co", role = c("aut", "cre")),
person(given = "Posit Software, PBC", role = c("cph", "fnd"))
Expand Down
10 changes: 7 additions & 3 deletions NEWS.md
Original file line number Diff line number Diff line change
@@ -1,14 +1,18 @@
# pysparklyr (dev)
# pysparklyr 0.1.5

* Adds support for `I()` in `tbl()`
### Improvements

* Fixes issues with having multiple line functions in `spark_apply()`
* Adds support for `I()` in `tbl()`

* Ensures `arrow` is installed by adding it to Imports (#116)

* If the cluster version is higher than the available Python library, it will
either use, or offer to install the available Python library

### Fixes

* Fixes issues with having multiple line functions in `spark_apply()`

# pysparklyr 0.1.4

### New
Expand Down
39 changes: 17 additions & 22 deletions cran-comments.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,36 +2,31 @@

In this version:

* Adds support for `spark_apply()` via the `rpy2` Python library
* It will not automatically distribute packages, it will assume that the
necessary packages are already installed in each node. This also means that
the `packages` argument is not supported
* As in its original implementation, schema inferring works, and as with the
original implementation, it has a performance cost. Unlike the original, the
Databricks, and Spark, Connect version will return a 'columns' specification
that you can use for the next time you run the call.

* At connection time, it enables Arrow by default. It does this by setting
these two configuration settings to true:
* `spark.sql.execution.arrow.pyspark.enabled`
* `spark.sql.execution.arrow.pyspark.fallback.enabled`
* Adds support for `I()` in `tbl()`

* Ensures `arrow` is installed by adding it to Imports (#116)

* If the cluster version is higher than the available Python library, it will
either use, or offer to install the available Python library

* Fixes issues with having multiple line functions in `spark_apply()`

## Test environments

- Ubuntu 22.04, R 4.3.3, Spark 3.5 (GH Actions)
- Ubuntu 22.04, R 4.3.3, Spark 3.4 (GH Actions)
- Ubuntu 22.04, R 4.4.1, Spark 3.5 (GH Actions)
- Ubuntu 22.04, R 4.4.1, Spark 3.4 (GH Actions)

- Local Mac OS M3 (aarch64-apple-darwin23), R 4.3.3, Spark 3.5 (Local)
- Local Mac OS M3 (aarch64-apple-darwin23), R 4.4.0, Spark 3.5 (Local)

## R CMD check environments

- Mac OS M3 (aarch64-apple-darwin23), R 4.3.3 (Local)
- Mac OS M3 (aarch64-apple-darwin23), R 4.4.0 (Local)

- Mac OS x86_64-apple-darwin20.0 (64-bit), R 4.3.3 (GH Actions)
- Windows x86_64-w64-mingw32 (64-bit), R 4.3.3 (GH Actions)
- Linux x86_64-pc-linux-gnu (64-bit), R 4.3.3 (GH Actions)
- Linux x86_64-pc-linux-gnu (64-bit), R 4.5.0 (dev) (GH Actions)
- Linux x86_64-pc-linux-gnu (64-bit), R 4.2.3 (old release) (GH Actions)
- Mac OS x86_64-apple-darwin20.0 (64-bit), R 4.4.1 (GH Actions)
- Windows x86_64-w64-mingw32 (64-bit), R 4.4.1 (GH Actions)
- Linux x86_64-pc-linux-gnu (64-bit), R dev (GH Actions)
- Linux x86_64-pc-linux-gnu (64-bit), R 4.4.1 (GH Actions)
- Linux x86_64-pc-linux-gnu (64-bit), R 4.3.3 (old release) (GH Actions)


## R CMD check results
Expand Down

0 comments on commit 39748e1

Please sign in to comment.