Can't load R libraries

[roaldarbol]

Checks

• I have checked that this issue has not already been reported.

• I have confirmed this bug exists on the latest version of pixi, using pixi --version.

Reproducible example

When installing R libraries, there's an error when you try to load that library. Effectively it means that R currently cannot be used with pixi unfortunately.

Here's a pixi.toml:

[project]
name = "Test"
version = "0.1.0"
description = "Add a short description here"
authors = ["Mikkel Roald-Arbøl"]
channels = ["conda-forge"]
platforms = ["osx-64"]

[tasks]

[dependencies]
r = "4.3.*"
r-tidyverse = "2.0.0.*"

Start a shell with pixi shell. which R to make sure that it is indeed the pixi R installation being used.
Then open R with

R

Then once R (like ipython) is open, run:

library(tidyverse)

Which returns the error:

Error: package or namespace load failed for ‘tidyverse’ in library.dynam(lib, package, package.lib):
 shared object ‘magrittr.dylib’ not found

Issue description

I've tried with multiple packages, both with R but also trying to run an R script with Rscript - same outcome. I've frequently run R scripts from the command line, so I'm fairly sure it's a pixi issue.

Expected behavior

I would expect pixi to detect the installed packages and simply load the library.

[wolfv]

This is my output on osx-arm64 (Apple Silicon):

> help()
> library(tidyverse)
── Attaching core tidyverse packages ────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────── tidyverse 2.0.0 ──
✔ dplyr     1.1.4     ✔ readr     2.1.4
✔ forcats   1.0.0     ✔ stringr   1.5.1
✔ ggplot2   3.4.4     ✔ tibble    3.2.1
✔ lubridate 1.9.3     ✔ tidyr     1.3.0
✔ purrr     1.0.2
── Conflicts ──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────── tidyverse_conflicts() ──
✖ dplyr::filter() masks stats::filter()
✖ dplyr::lag()    masks stats::lag()
ℹ Use the conflicted package to force all conflicts to become errors
>

[roaldarbol]

I tried to run .libPaths() to see where it fetches the packages, and it still looks first in the global non-pixi installation:

.libPaths()
[1] "/Users/roaldarbol/Library/R/x86_64/4.3/library"         
[2] "/Users/roaldarbol/Research/Test/.pixi/env/lib/R/library"

Do you also have a global non-pixi R installed? If not, can you try that too - and also install tidyverse from R with install.packages('tidyverse')?

EDIT: It seems you need to set .libPaths to the desired library.

> .libPaths()
[1] "/Users/roaldarbol/Library/R/x86_64/4.3/library"         
[2] "/Users/roaldarbol/Research/Test/.pixi/env/lib/R/library"
> .libPaths("/Users/roaldarbol/Research/Test/.pixi/env/lib/R/library")
> .libPaths()
[1] "/Users/roaldarbol/Research/Test/.pixi/env/lib/R/library"
> library(tidyverse)
── Attaching core tidyverse packages ───────────────────────────────────────────────────────────────────── tidyverse 2.0.0 ──
✔ dplyr     1.1.4     ✔ readr     2.1.4
✔ forcats   1.0.0     ✔ stringr   1.5.1
✔ ggplot2   3.4.4     ✔ tibble    3.2.1
✔ lubridate 1.9.3     ✔ tidyr     1.3.0
✔ purrr     1.0.2     
── Conflicts ─────────────────────────────────────────────────────────────────────────────────────── tidyverse_conflicts() ──
✖ dplyr::filter() masks stats::filter()
✖ dplyr::lag()    masks stats::lag()
ℹ Use the conflicted package (<http://conflicted.r-lib.org/>) to force all conflicts to become errors
> 

[roaldarbol]

As an aside, using virtual environments is not common in R-land unfortunately, and currently . If pixi could function as or interact as an R Project that also specifies R installation, that would be huge. Especially since 99% work in RStudio, so would be amazing if RStudio could pick up the R installation from pixi. Surely would have to involve RStudio, but I could easily see them having an interest in it - they collaborate quite a bit with various Jupyter folks.

[roaldarbol]

As a short-term fix, the user can create an .Renviron file, and in it specify the path manually:

R_LIBS_USER="/Users/roaldarbol/Research/Test/.pixi/env/lib/R/library"

.Renviron is detected both by RStudio and R in the command line (unlike .Rprofile seemingly).

[ruben-arts]

Nice find, is there a difference between pixi and conda that you know of in this case?

[roaldarbol]

Haven't tried with conda to be honest. I've long seen conda/mamba as purely Python-land, it's only with pixi I got really excited to try getting everything together in one environment. (BTW happy to chat more about what would make a great R pixi experience on Discord if anyone is interested).😅

In the end the easiest thing ended up using renv for the R dependencies for now. renv also gets picked up if you use command-line R installed with pixi (r-base). And renv does exactly just specify which library to use it seems (and of course creates a new library).

[ruben-arts]

Yes I would love to hear more about it as I (and I think the team) doesn't have a lot of R experience so far.

[notPlancha]

R in conda is supposed to only set libpaths to the cond environment; this seems true for any channel (it seems that's the case for the r and conda-forge channel at least), however R will still source a lot of files outside the conda environment. So @roaldarbol try running pixi run R --vanilla and see if that fixes it.

[roaldarbol]

@notPlancha Just tried, and that doesn't change the library paths unfortunately. I guess the "sourcing files outside the conda environment" is the crux of the issue here: identifying where it does that and ensuring that it doesn't.

[notPlancha]

@roaldarbol How did you install R globally? Asking to see if I can reproduce the problem. Also could you run pixi run type -a R? Pixi could be doing weird shenanigans with the path. Lastly what does pixi run R --vanilla -e ".libPaths()" output?

[roaldarbol]

I've installed R globally with brew.

Running pixi run type -a R gives the following output (I am working in the Test folder):

R is /Users/roaldarbol/Research/Test/.pixi/env/bin/R
R is /usr/local/bin/R
R is /usr/local/bin/R

Running pixi run R --vanilla -e ".libPaths()" strangely gives me an error:

× Unexpected character.
  │   ()
  │   ~

But running pixi run R --vanilla and then .libPaths() produces:

[1] "/Users/roaldarbol/Library/R/x86_64/4.3/library"         
[2] "/Users/roaldarbol/Research/Test/.pixi/env/lib/R/library"

[notPlancha]

My suspicion of what's happening is that the environment variable R_LIBS_USER (or R_LIBS?) is set, and R is prepending that path. If that's not what's happening then I don't know.

I made some googling and my conclusion is that even though r-base in conda-forge already isolates packages (for some reason), it's not officially supported conda-forge/r-base-feedstock#65 (comment), so I think the best solutions right now are:

1. to have a .Rprofile that runs .libPaths(R.home("library")) or
2. to do add conda-ecosystem-user-package-isolation to the project (pixi add conda-ecosystem-user-package-isolation)

The disadvantages of 1 is that if .Rprofile is set site wise, then it can lead to unreproducible results, and if it's set project wise, it'll not load the site wise .Rprofile, which some people find it useful. I haven't tested 2 so I don't know if it actually works, but looking into it it seems it basically just changes R_LIBS_USER (and PYTHONNOUSERSITE), per https://github.com/conda-forge/conda-ecosystem-user-package-isolation-feedstock/blob/a21dabc7466215ac4d0a4b701c912368150d515e/recipe/activate-user-package-isolation.sh.

If the latter actually works, I think it's a great opportunity for pixi to implement this change itself, or at least add an option for default (user-defined) packages on pixi init (similar to conda config --add create_default_packages)

But before that, @roaldarbol please check if either R_LIBS_USER or R_LIBS are set as environment variables