Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Error needing catching in @iload ? #385

Open
ablaom opened this issue May 24, 2021 · 9 comments
Open

Error needing catching in @iload ? #385

ablaom opened this issue May 24, 2021 · 9 comments

Comments

@ablaom
Copy link
Member

ablaom commented May 24, 2021

It seems there is an error that we are not catching (and ignoring) here, as the interactive part following the message works fine:

using MLJ # in a new environment with only MLJ
julia> Clf = @iload EvoTreeClassifier
[ Info: For silent loading, specify `verbosity=0`. 
[ Info: Precompiling EvoTrees [f6006082-12f8-11e9-0c9c-0d5d367ab1e5]
ERROR: LoadError: ArgumentError: Package CUDA [052768ef-5323-5732-b1bb-66c8b64840ba] is required but does not seem to be installed:
 - Run `Pkg.instantiate()` to install all recorded dependencies.

Stacktrace:
 [1] _require(pkg::Base.PkgId)
   @ Base ./loading.jl:990
 [2] require(uuidkey::Base.PkgId)
   @ Base ./loading.jl:914
 [3] require(into::Module, mod::Symbol)
   @ Base ./loading.jl:901
 [4] include
   @ ./Base.jl:386 [inlined]
 [5] include_package_for_output(pkg::Base.PkgId, input::String, depot_path::Vector{String}, 
dl_load_path::Vector{String}, load_path::Vector{String}, concrete_deps::Vector{Pair{Base.PkgId, UInt64}}, source::Nothing)                                                
   @ Base ./loading.jl:1213
 [6] top-level scope
   @ none:1
 [7] eval
   @ ./boot.jl:360 [inlined]
 [8] eval(x::Expr)
   @ Base.MainInclude ./client.jl:446
 [9] top-level scope
   @ none:1
in expression starting at /Users/anthony/Dropbox/Julia7/EvoTrees/src/EvoTrees.jl:1

The package providing an interface to `EvoTreeClassifier` is not in your current environment.
What do you want to do? 

 > Install EvoTrees in current environment.
   Abort.
@ablaom
Copy link
Member Author

ablaom commented May 24, 2021

Maybe related to JuliaAI/MLJ.jl#757

@ablaom
Copy link
Member Author

ablaom commented May 24, 2021

cc @giordano

@giordano
Copy link
Member

Given the issue you're experiencing it's likely there is an exception that isn't being caught, but I can't reproduce your issue in a clean depot:

% JULIA_PKG_SERVER="pkg.julialang.org" JULIA_DEPOT_PATH=$(mktemp -d) julia -q
(@v1.6) pkg> add MLJ#dev
  Installing known registries into `/tmp/tmp.KauxRMUxvt`

       Added registry `General` to `/tmp/tmp.KauxRMUxvt/registries/General`
     Cloning git-repo `https://github.com/alan-turing-institute/MLJ.jl.git`
    Updating git-repo `https://github.com/alan-turing-institute/MLJ.jl.git`
    Updating registry at `/tmp/tmp.KauxRMUxvt/registries/General`
   Resolving package versions...
   Installed ProgressMeter ─────────────── v1.6.2
   Installed StatisticalTraits ─────────── v1.1.0
   Installed IniFile ───────────────────── v0.5.0
   Installed RecipesBase ───────────────── v1.1.1
   Installed Formatting ────────────────── v0.4.2
   Installed FixedPointNumbers ─────────── v0.8.4
   Installed Libiconv_jll ──────────────── v1.16.1+0
   Installed Distances ─────────────────── v0.10.3
   Installed HTTP ──────────────────────── v0.9.9
   Installed MLJBase ───────────────────── v0.18.7
   Installed MLJTuning ─────────────────── v0.6.5
   Installed Compat ────────────────────── v3.30.0
   Installed Parsers ───────────────────── v1.1.0
   Installed MLJOpenML ─────────────────── v1.0.0
   Installed Parameters ────────────────── v0.12.2
   Installed XML2_jll ──────────────────── v2.9.12+0
   Installed Preferences ───────────────── v1.2.2
   Installed Reexport ──────────────────── v1.0.0
   Installed IteratorInterfaceExtensions ─ v1.0.0
   Installed EzXML ─────────────────────── v1.1.0
   Installed MLJSerialization ──────────── v1.1.2
   Installed Requires ──────────────────── v1.1.3
   Installed Memento ───────────────────── v1.1.2
   Installed DataAPI ───────────────────── v1.6.0
   Installed ChainRulesCore ────────────── v0.9.44
   Installed MLJModels ─────────────────── v0.14.6
   Installed Distributions ─────────────── v0.24.18
   Installed JSON ──────────────────────── v0.21.1
   Installed Rmath_jll ─────────────────── v0.3.0+0
   Installed FilePathsBase ─────────────── v0.9.10
   Installed Rmath ─────────────────────── v0.7.0
   Installed LearnBase ─────────────────── v0.4.1
   Installed BSON ──────────────────────── v0.3.3
   Installed DocStringExtensions ───────── v0.8.4
   Installed MLJIteration ──────────────── v0.3.0
   Installed Crayons ───────────────────── v4.0.4
   Installed DataValueInterfaces ───────── v1.0.0
   Installed ColorTypes ────────────────── v0.11.0
   Installed JLSO ──────────────────────── v2.5.0
   Installed QuadGK ────────────────────── v2.4.1
   Installed MLJModelInterface ─────────── v1.1.0
   Installed MLJScientificTypes ────────── v0.4.7
   Installed SortingAlgorithms ─────────── v1.0.0
   Installed URIs ──────────────────────── v1.3.0
   Installed TableTraits ───────────────── v1.0.1
   Installed StatsAPI ──────────────────── v1.0.0
   Installed StatsBase ─────────────────── v0.33.8
   Installed IterationControl ──────────── v0.4.0
   Installed LogExpFunctions ───────────── v0.2.4
   Installed ComputationalResources ────── v0.3.2
   Installed Missings ──────────────────── v1.0.0
   Installed StableRNGs ────────────────── v1.0.0
   Installed UnPack ────────────────────── v1.0.2
   Installed DataStructures ────────────── v0.18.9
   Installed Tables ────────────────────── v1.4.2
   Installed CodecZlib ─────────────────── v0.7.0
   Installed ScientificTypes ───────────── v1.1.2
   Installed SpecialFunctions ──────────── v1.4.1
   Installed EarlyStopping ─────────────── v0.1.8
   Installed Syslogs ───────────────────── v0.3.0
   Installed JLLWrappers ───────────────── v1.3.0
   Installed ExprTools ─────────────────── v0.1.3
   Installed StructTypes ───────────────── v1.7.2
   Installed OpenSpecFun_jll ───────────── v0.5.4+0
   Installed InvertedIndices ───────────── v1.0.0
   Installed Mocking ───────────────────── v0.7.1
   Installed CategoricalArrays ─────────── v0.10.0
   Installed PDMats ────────────────────── v0.11.0
   Installed TranscodingStreams ────────── v0.9.5
   Installed StatsFuns ─────────────────── v0.9.8
   Installed PersistenceDiagramsBase ───── v0.1.1
   Installed TimeZones ─────────────────── v1.5.5
   Installed FillArrays ────────────────── v0.11.7
   Installed OrderedCollections ────────── v1.4.1
   Installed MbedTLS ───────────────────── v1.0.3
   Installed LossFunctions ─────────────── v0.7.0
   Installed PrettyTables ──────────────── v1.0.1
   Installed LatinHypercubeSampling ────── v1.8.0
  Downloaded artifact: Libiconv
  Downloaded artifact: Libiconv
  Downloaded artifact: XML2
  Downloaded artifact: XML2
  Downloaded artifact: Rmath
  Downloaded artifact: OpenSpecFun
    Updating `/tmp/tmp.KauxRMUxvt/environments/v1.6/Project.toml`
  [add582a8] + MLJ v0.16.4 `https://github.com/alan-turing-institute/MLJ.jl.git#dev`
    Updating `/tmp/tmp.KauxRMUxvt/environments/v1.6/Manifest.toml`
  [fbb218c0] + BSON v0.3.3
  [324d7699] + CategoricalArrays v0.10.0
  [d360d2e6] + ChainRulesCore v0.9.44
  [944b1d66] + CodecZlib v0.7.0
  [3da002f7] + ColorTypes v0.11.0
  [34da2185] + Compat v3.30.0
  [ed09eef8] + ComputationalResources v0.3.2
  [a8cc5b0e] + Crayons v4.0.4
  [9a962f9c] + DataAPI v1.6.0
  [864edb3b] + DataStructures v0.18.9
  [e2d170a0] + DataValueInterfaces v1.0.0
  [b4f34e82] + Distances v0.10.3
  [31c24e10] + Distributions v0.24.18
  [ffbed154] + DocStringExtensions v0.8.4
  [792122b4] + EarlyStopping v0.1.8
  [e2ba6199] + ExprTools v0.1.3
  [8f5d6c58] + EzXML v1.1.0
  [48062228] + FilePathsBase v0.9.10
  [1a297f60] + FillArrays v0.11.7
  [53c48c17] + FixedPointNumbers v0.8.4
  [59287772] + Formatting v0.4.2
  [cd3eb016] + HTTP v0.9.9
  [83e8ac13] + IniFile v0.5.0
  [41ab1584] + InvertedIndices v1.0.0
  [b3c1a2ee] + IterationControl v0.4.0
  [82899510] + IteratorInterfaceExtensions v1.0.0
  [692b3bcd] + JLLWrappers v1.3.0
  [9da8a3cd] + JLSO v2.5.0
  [682c06a0] + JSON v0.21.1
  [a5e1c1ea] + LatinHypercubeSampling v1.8.0
  [7f8f8fb0] + LearnBase v0.4.1
  [2ab3a3ac] + LogExpFunctions v0.2.4
  [30fc2ffe] + LossFunctions v0.7.0
  [add582a8] + MLJ v0.16.4 `https://github.com/alan-turing-institute/MLJ.jl.git#dev`
  [a7f614a8] + MLJBase v0.18.7
  [614be32b] + MLJIteration v0.3.0
  [e80e1ace] + MLJModelInterface v1.1.0
  [d491faf4] + MLJModels v0.14.6
  [cbea4545] + MLJOpenML v1.0.0
  [2e2323e0] + MLJScientificTypes v0.4.7
  [17bed46d] + MLJSerialization v1.1.2
  [03970b2e] + MLJTuning v0.6.5
  [739be429] + MbedTLS v1.0.3
  [f28f55f0] + Memento v1.1.2
  [e1d29d7a] + Missings v1.0.0
  [78c3b35d] + Mocking v0.7.1
  [bac558e1] + OrderedCollections v1.4.1
  [90014a1f] + PDMats v0.11.0
  [d96e819e] + Parameters v0.12.2
  [69de0a69] + Parsers v1.1.0
  [b1ad91c1] + PersistenceDiagramsBase v0.1.1
  [21216c6a] + Preferences v1.2.2
  [08abe8d2] + PrettyTables v1.0.1
  [92933f4c] + ProgressMeter v1.6.2
  [1fd47b50] + QuadGK v2.4.1
  [3cdcf5f2] + RecipesBase v1.1.1
  [189a3867] + Reexport v1.0.0
  [ae029012] + Requires v1.1.3
  [79098fc4] + Rmath v0.7.0
  [321657f4] + ScientificTypes v1.1.2
  [a2af1166] + SortingAlgorithms v1.0.0
  [276daf66] + SpecialFunctions v1.4.1
  [860ef19b] + StableRNGs v1.0.0
  [64bff920] + StatisticalTraits v1.1.0
  [82ae8749] + StatsAPI v1.0.0
  [2913bbd2] + StatsBase v0.33.8
  [4c63d2b9] + StatsFuns v0.9.8
  [856f2bd8] + StructTypes v1.7.2
  [cea106d9] + Syslogs v0.3.0
  [3783bdb8] + TableTraits v1.0.1
  [bd369af6] + Tables v1.4.2
  [f269a46b] + TimeZones v1.5.5
  [3bb67fe8] + TranscodingStreams v0.9.5
  [5c2747f8] + URIs v1.3.0
  [3a884ed6] + UnPack v1.0.2
  [94ce4f54] + Libiconv_jll v1.16.1+0
  [efe28fd5] + OpenSpecFun_jll v0.5.4+0
  [f50d1b31] + Rmath_jll v0.3.0+0
  [02c8fc9c] + XML2_jll v2.9.12+0
  [0dad84c5] + ArgTools
  [56f22d72] + Artifacts
  [2a0f44e3] + Base64
  [ade2ca70] + Dates
  [8bb1440f] + DelimitedFiles
  [8ba89e20] + Distributed
  [f43a241f] + Downloads
  [9fa8497b] + Future
  [b77e0a4c] + InteractiveUtils
  [4af54fe1] + LazyArtifacts
  [b27032c2] + LibCURL
  [76f85450] + LibGit2
  [8f399da3] + Libdl
  [37e2e46d] + LinearAlgebra
  [56ddb016] + Logging
  [d6f4376e] + Markdown
  [a63ad114] + Mmap
  [ca575930] + NetworkOptions
  [44cfe95a] + Pkg
  [de0858da] + Printf
  [3fa0cd96] + REPL
  [9a3f8284] + Random
  [ea8e919c] + SHA
  [9e88b42a] + Serialization
  [1a1011a3] + SharedArrays
  [6462fe0b] + Sockets
  [2f01184e] + SparseArrays
  [10745b16] + Statistics
  [4607b0f0] + SuiteSparse
  [fa267f1f] + TOML
  [a4e569a6] + Tar
  [8dfed614] + Test
  [cf7118a7] + UUIDs
  [4ec0a83e] + Unicode
  [e66e0078] + CompilerSupportLibraries_jll
  [deac9b47] + LibCURL_jll
  [29816b5a] + LibSSH2_jll
  [c8ffd9c3] + MbedTLS_jll
  [14a3606d] + MozillaCACerts_jll
  [83775a58] + Zlib_jll
  [8e850ede] + nghttp2_jll
  [3f19e933] + p7zip_jll
    Building TimeZones → `/tmp/tmp.KauxRMUxvt/scratchspaces/44cfe95a-1eb2-52ea-b672-e2afdf69b78f/960099aed321e05ac649c90d583d59c9309faee1/build.log`
Precompiling project...
  80 dependencies successfully precompiled in 39 seconds (3 already precompiled)

(@v1.6) pkg> 

julia> using MLJ

julia> Clf = @iload EvoTreeClassifier
[ Info: For silent loading, specify `verbosity=0`. 

The package providing an interface to `EvoTreeClassifier` is not in your current environment.
What do you want to do? 

 > Install EvoTrees in current environment.
   Abort.

   Resolving package versions...
   Installed CEnum ────────── v0.4.1
   Installed BFloat16s ────── v0.1.0
   Installed EarCut_jll ───── v2.1.5+1
   Installed NetworkLayout ── v0.3.0
   Installed MacroTools ───── v0.5.6
   Installed Scratch ──────── v1.0.3
   Installed RandomNumbers ── v1.4.0
   Installed IterTools ────── v1.3.0
   Installed AbstractFFTs ─── v1.0.1
   Installed GeometryBasics ─ v0.3.12
   Installed StructArrays ─── v0.5.1
   Installed Memoize ──────── v0.4.4
   Installed GPUCompiler ──── v0.11.5
   Installed Adapt ────────── v3.3.0
   Installed Random123 ────── v1.3.1
   Installed LLVM ─────────── v3.7.1
   Installed StaticArrays ─── v1.2.0
   Installed TimerOutputs ─── v0.5.9
   Installed GPUArrays ────── v6.4.1
   Installed EvoTrees ─────── v0.8.0
   Installed CUDA ─────────── v3.2.1
  Downloaded artifact: EarCut
    Updating `/tmp/tmp.KauxRMUxvt/environments/v1.6/Project.toml`
  [f6006082] + EvoTrees v0.8.0
    Updating `/tmp/tmp.KauxRMUxvt/environments/v1.6/Manifest.toml`
  [621f4979] + AbstractFFTs v1.0.1
  [79e6a3ab] + Adapt v3.3.0
  [ab4f0b2a] + BFloat16s v0.1.0
  [fa961155] + CEnum v0.4.1
  [052768ef] + CUDA v3.2.1
  [f6006082] + EvoTrees v0.8.0
  [0c68f7d7] + GPUArrays v6.4.1
  [61eb1bfa] + GPUCompiler v0.11.5
  [5c1252a2] + GeometryBasics v0.3.12
  [c8e1da08] + IterTools v1.3.0
  [929cbde3] + LLVM v3.7.1
  [1914dd2f] + MacroTools v0.5.6
  [c03570c3] + Memoize v0.4.4
  [46757867] + NetworkLayout v0.3.0
  [74087812] + Random123 v1.3.1
  [e6cf234a] + RandomNumbers v1.4.0
  [6c6a2e73] + Scratch v1.0.3
  [90137ffa] + StaticArrays v1.2.0
  [09ab397b] + StructArrays v0.5.1
  [a759f4b9] + TimerOutputs v0.5.9
  [5ae413db] + EarCut_jll v2.1.5+1
    Building Random123 → `/tmp/tmp.KauxRMUxvt/scratchspaces/44cfe95a-1eb2-52ea-b672-e2afdf69b78f/7c6710c8198fd4444b5eb6a3840b7d47bd3593c5/build.log`
Precompiling project...
  21 dependencies successfully precompiled in 45 seconds (83 already precompiled)
import EvoTrees ✔
EvoTrees.EvoTreeClassifier

julia>

@giordano
Copy link
Member

Reading the code, I'm having a hard time figuring out where the exception is being thrown. Can you please show the output of @macroexpand @iload EvoTreeClassifier for you? I get

quote
    #= /tmp/tmp.O88ehAk8Zj/packages/MLJModels/9mQfs/src/loading.jl:129 =#
    begin
        #= /tmp/tmp.O88ehAk8Zj/packages/MLJModels/9mQfs/src/loading.jl:168 =#
        true && begin
            #= logging.jl:350 =#
            let
                #= logging.jl:351 =#
                var"#47#level" = Base.CoreLogging.Info
                #= logging.jl:352 =#
                var"#48#std_level" = Base.CoreLogging.convert(Base.CoreLogging.LogLevel, var"#47#level")
                #= logging.jl:353 =#
                if var"#48#std_level" >= Base.CoreLogging._min_enabled_level[]
                    #= logging.jl:354 =#
                    var"#49#group" = :loading
                    #= logging.jl:355 =#
                    var"#50#_module" = Main
                    #= logging.jl:356 =#
                    var"#51#logger" = Base.CoreLogging.current_logger_for_env(var"#48#std_level", var"#49#group", var"#50#_module")
                    #= logging.jl:357 =#
                    if !(var"#51#logger" === Base.CoreLogging.nothing)
                        #= logging.jl:358 =#
                        var"#52#id" = :Main_75496672
                        #= logging.jl:361 =#
                        if Base.CoreLogging._invoked_shouldlog(var"#51#logger", var"#47#level", var"#50#_module", var"#49#group", var"#52#id")
                            #= logging.jl:362 =#
                            var"#53#file" = "/tmp/tmp.O88ehAk8Zj/packages/MLJModels/9mQfs/src/loading.jl"
                            #= logging.jl:363 =#
                            var"#54#line" = 168
                            #= logging.jl:364 =#
                            local var"#55#msg", var"#56#kwargs"
                            #= logging.jl:365 =#
                            begin
                                #= logging.jl:308 =#
                                var"#55#msg" = "For silent loading, specify `verbosity=0`. "
                                #= logging.jl:309 =#
                                var"#56#kwargs" = (;)
                                #= logging.jl:310 =#
                                true
                            end && Base.CoreLogging.handle_message(var"#51#logger", var"#47#level", var"#55#msg", var"#50#_module", var"#49#group", var"#52#id", var"#53#file", var"#54#line"; var"#56#kwargs"...)
                        end
                    end
                end
                #= logging.jl:371 =#
                Base.CoreLogging.nothing
            end
        end
        #= /tmp/tmp.O88ehAk8Zj/packages/MLJModels/9mQfs/src/loading.jl:169 =#
        var"#37#proxy" = (MLJModels.MLJModels).info("EvoTreeClassifier"; pkg = nothing, interactive = true)
        #= /tmp/tmp.O88ehAk8Zj/packages/MLJModels/9mQfs/src/loading.jl:172 =#
        var"#38#handle" = (MLJModels.MLJModels).Handle((var"#37#proxy").name, (var"#37#proxy").package_name)
        #= /tmp/tmp.O88ehAk8Zj/packages/MLJModels/9mQfs/src/loading.jl:173 =#
        var"#39#dic" = (MLJModels.MLJModels).INFO_GIVEN_HANDLE[var"#38#handle"]
        #= /tmp/tmp.O88ehAk8Zj/packages/MLJModels/9mQfs/src/loading.jl:174 =#
        (var"#40#path_str", var"#41#pkg_str") = (var"#39#dic"[:load_path], var"#39#dic"[:package_name])
        #= /tmp/tmp.O88ehAk8Zj/packages/MLJModels/9mQfs/src/loading.jl:175 =#
        var"#42#path" = var"#40#path_str" |> (MLJModels.Meta).parse
        #= /tmp/tmp.O88ehAk8Zj/packages/MLJModels/9mQfs/src/loading.jl:176 =#
        var"#43#pkg" = MLJModels.Symbol(var"#41#pkg_str")
    end
    begin
        #= /tmp/tmp.O88ehAk8Zj/packages/MLJModels/9mQfs/src/loading.jl:181 =#
        var"#44#path_components" = MLJModels.Symbol.(MLJModels.split(var"#40#path_str", '.'))
        #= /tmp/tmp.O88ehAk8Zj/packages/MLJModels/9mQfs/src/loading.jl:184 =#
        var"#45#api_pkg" = var"#44#path_components"[1]
        #= /tmp/tmp.O88ehAk8Zj/packages/MLJModels/9mQfs/src/loading.jl:185 =#
        var"#46#api_pkg_str" = MLJModels.string(var"#45#api_pkg")
        #= /tmp/tmp.O88ehAk8Zj/packages/MLJModels/9mQfs/src/loading.jl:187 =#
        if false || true
            #= /tmp/tmp.O88ehAk8Zj/packages/MLJModels/9mQfs/src/loading.jl:188 =#
            try
                #= /tmp/tmp.O88ehAk8Zj/packages/MLJModels/9mQfs/src/loading.jl:189 =#
                (MLJModels.MLJModels)._import(Main, var"#45#api_pkg", var"#43#pkg", false)
            catch
                #= /tmp/tmp.O88ehAk8Zj/packages/MLJModels/9mQfs/src/loading.jl:191 =#
                if true
                    #= /tmp/tmp.O88ehAk8Zj/packages/MLJModels/9mQfs/src/loading.jl:192 =#
                    (MLJModels.MLJModels).request("The package providing an interface " * "to `EvoTreeClassifier` " * "is not in your " * "current environment.\n" * "What do you want to do? ", "Install $(var"#46#api_pkg_str") in current environment.", "Abort.") == 1 || MLJModels.throw(MLJModels.InterruptException)
                end
                #= /tmp/tmp.O88ehAk8Zj/packages/MLJModels/9mQfs/src/loading.jl:202 =#
                (MLJModels.MLJModels)._import(Main, :Pkg, :Pkg, false)
                #= /tmp/tmp.O88ehAk8Zj/packages/MLJModels/9mQfs/src/loading.jl:203 =#
                (MLJModels.Pkg).add(var"#46#api_pkg_str")
            end
        end
        #= /tmp/tmp.O88ehAk8Zj/packages/MLJModels/9mQfs/src/loading.jl:206 =#
        (MLJModels.MLJModels)._import(Main, var"#45#api_pkg", var"#43#pkg", true)
        #= /tmp/tmp.O88ehAk8Zj/packages/MLJModels/9mQfs/src/loading.jl:207 =#
        (MLJModels.MLJModels)._eval(Main, var"#42#path")
    end
end

Trying to load the package before the interactive Pkg prompt happens only at line L189
https://github.com/alan-turing-institute/MLJModels.jl/blob/6722abd5546d1cd58ff9547e98c7bdcf60ffcfb5/src/loading.jl#L188-L190
which is protected in a try/catch block and isn't rethrown nor shown

@ablaom
Copy link
Member Author

ablaom commented May 25, 2021

@giordano Thanks for investigating. Naturally this is difficult to debug without being able to reproduce the error. Let me make sure I can also reproduce and get back to you. I actually don't think I was using a fresh environment after all. My apologies for not being more thorough.

@ablaom
Copy link
Member Author

ablaom commented May 26, 2021

Okay, I do get (a different) error in a bare environment.

The error

julia> using MLJModels
julia> @iload EvoTreeRegressor
[ Info: For silent loading, specify `verbosity=0`. 
[ Info: Precompiling EvoTrees [f6006082-12f8-11e9-0c9c-0d5d367ab1e5]
ERROR: LoadError: ArgumentError: Package StaticArrays [90137ffa-7385-5640-81b9-e52037218182] is required but does not seem to be installed:
 - Run `Pkg.instantiate()` to install all recorded dependencies.

Stacktrace:
 [1] _require(pkg::Base.PkgId)
   @ Base ./loading.jl:990
 [2] require(uuidkey::Base.PkgId)
   @ Base ./loading.jl:914
 [3] require(into::Module, mod::Symbol)
   @ Base ./loading.jl:901
 [4] include
   @ ./Base.jl:386 [inlined]
 [5] include_package_for_output(pkg::Base.PkgId, input::String, depot_path::Vector{String}, 
dl_load_path::Vector{String}, load_path::Vector{String}, concrete_deps::Vector{Pair{Base.PkgId, UInt64}}, source::Nothing)                                                
   @ Base ./loading.jl:1213
 [6] top-level scope
   @ none:1
 [7] eval
   @ ./boot.jl:360 [inlined]
 [8] eval(x::Expr)
   @ Base.MainInclude ./client.jl:446
 [9] top-level scope
   @ none:1
in expression starting at /Users/anthony/Dropbox/Julia7/EvoTrees/src/EvoTrees.jl:1

The package providing an interface to `EvoTreeRegressor` is not in your current environment.
What do you want to do? 

   Install EvoTrees in current environment.
 > Abort.

My environment

(@testload) pkg> st
      Status `~/.julia/environments/testload/Project.toml`
  [d491faf4] MLJModels v0.14.6
(@testload) pkg> st -m
      Status `~/.julia/environments/testload/Manifest.toml`
  [324d7699] CategoricalArrays v0.10.0
  [d360d2e6] ChainRulesCore v0.9.44
  [3da002f7] ColorTypes v0.11.0
  [34da2185] Compat v3.30.0
  [ed09eef8] ComputationalResources v0.3.2
  [a8cc5b0e] Crayons v4.0.4
  [9a962f9c] DataAPI v1.6.0
  [864edb3b] DataStructures v0.18.9
  [e2d170a0] DataValueInterfaces v1.0.0
  [b4f34e82] Distances v0.10.3
  [31c24e10] Distributions v0.25.1
  [ffbed154] DocStringExtensions v0.8.4
  [1a297f60] FillArrays v0.11.7
  [53c48c17] FixedPointNumbers v0.8.4
  [59287772] Formatting v0.4.2
  [41ab1584] InvertedIndices v1.0.0
  [82899510] IteratorInterfaceExtensions v1.0.0
  [692b3bcd] JLLWrappers v1.3.0
  [682c06a0] JSON v0.21.1
  [7f8f8fb0] LearnBase v0.4.1
  [2ab3a3ac] LogExpFunctions v0.2.4
  [30fc2ffe] LossFunctions v0.7.0
  [a7f614a8] MLJBase v0.18.7
  [e80e1ace] MLJModelInterface v1.1.0
  [d491faf4] MLJModels v0.14.6
  [2e2323e0] MLJScientificTypes v0.4.7
  [e1d29d7a] Missings v1.0.0
  [bac558e1] OrderedCollections v1.4.1
  [90014a1f] PDMats v0.11.0
  [d96e819e] Parameters v0.12.2
  [69de0a69] Parsers v1.1.0
  [b1ad91c1] PersistenceDiagramsBase v0.1.1
  [21216c6a] Preferences v1.2.2
  [08abe8d2] PrettyTables v1.0.1
  [92933f4c] ProgressMeter v1.6.2
  [1fd47b50] QuadGK v2.4.1
  [3cdcf5f2] RecipesBase v1.1.1
  [189a3867] Reexport v1.0.0
  [ae029012] Requires v1.1.3
  [79098fc4] Rmath v0.7.0
  [321657f4] ScientificTypes v1.1.2
  [a2af1166] SortingAlgorithms v1.0.0
  [276daf66] SpecialFunctions v1.4.1
  [64bff920] StatisticalTraits v1.1.0
  [82ae8749] StatsAPI v1.0.0
  [2913bbd2] StatsBase v0.33.8
  [4c63d2b9] StatsFuns v0.9.8
  [856f2bd8] StructTypes v1.7.2
  [3783bdb8] TableTraits v1.0.1
  [bd369af6] Tables v1.4.2
  [3a884ed6] UnPack v1.0.2
  [efe28fd5] OpenSpecFun_jll v0.5.4+0
  [f50d1b31] Rmath_jll v0.3.0+0
  [0dad84c5] ArgTools
  [56f22d72] Artifacts
  [2a0f44e3] Base64
  [ade2ca70] Dates
  [8bb1440f] DelimitedFiles
  [8ba89e20] Distributed
  [f43a241f] Downloads
  [9fa8497b] Future
  [b77e0a4c] InteractiveUtils
  [b27032c2] LibCURL
  [76f85450] LibGit2
  [8f399da3] Libdl
  [37e2e46d] LinearAlgebra
  [56ddb016] Logging
  [d6f4376e] Markdown
  [a63ad114] Mmap
  [ca575930] NetworkOptions
  [44cfe95a] Pkg
  [de0858da] Printf
  [3fa0cd96] REPL
  [9a3f8284] Random
  [ea8e919c] SHA
  [9e88b42a] Serialization
  [1a1011a3] SharedArrays
  [6462fe0b] Sockets
  [2f01184e] SparseArrays
  [10745b16] Statistics
  [4607b0f0] SuiteSparse
  [fa267f1f] TOML
  [a4e569a6] Tar
  [8dfed614] Test
  [cf7118a7] UUIDs
  [4ec0a83e] Unicode
  [e66e0078] CompilerSupportLibraries_jll
  [deac9b47] LibCURL_jll
  [29816b5a] LibSSH2_jll
  [c8ffd9c3] MbedTLS_jll
  [14a3606d] MozillaCACerts_jll
  [83775a58] Zlib_jll
  [8e850ede] nghttp2_jll
  [3f19e933] p7zip_jll

The macro expansion

julia> @macroexpand @iload EvoTreeRegressor
quote
    #= /Users/anthony/.julia/packages/MLJModels/9mQfs/src/loading.jl:129 =#
    begin
        #= /Users/anthony/.julia/packages/MLJModels/9mQfs/src/loading.jl:168 =#
        true && begin
                #= logging.jl:350 =#
                let
                    #= logging.jl:351 =#
                    var"#104#level" = Base.CoreLogging.Info
                    #= logging.jl:352 =#
                    var"#105#std_level" = Base.CoreLogging.convert(Base.CoreLogging.LogLevel, var"#104#level")
                    #= logging.jl:353 =#
                    if var"#105#std_level" >= Base.CoreLogging._min_enabled_level[]
                        #= logging.jl:354 =#
                        var"#106#group" = :loading
                        #= logging.jl:355 =#
                        var"#107#_module" = Main
                        #= logging.jl:356 =#
                        var"#108#logger" = Base.CoreLogging.current_logger_for_env(var"#105#std_level", var"#106#group", var"#107#_module")
                        #= logging.jl:357 =#
                        if !(var"#108#logger" === Base.CoreLogging.nothing)
                            #= logging.jl:358 =#
                            var"#109#id" = :Main_75496674
                            #= logging.jl:361 =#
                            if Base.CoreLogging._invoked_shouldlog(var"#108#logger", var"#104#level", var"#107#_module", var"#106#group", var"#109#id")
                                #= logging.jl:362 =#
                                var"#110#file" = "/Users/anthony/.julia/packages/MLJModels/9mQfs/src/loading.jl"
                                #= logging.jl:363 =#
                                var"#111#line" = 168
                                #= logging.jl:364 =#
                                local var"#112#msg", var"#113#kwargs"
                                #= logging.jl:365 =#
                                begin
                                        #= logging.jl:308 =#
                                        var"#112#msg" = "For silent loading, specify `verbosity=0`. "
                                        #= logging.jl:309 =#
                                        var"#113#kwargs" = (;)
                                        #= logging.jl:310 =#
                                        true
                                    end && Base.CoreLogging.handle_message(var"#108#logger", var"#104#level", var"#112#msg", var"#107#_module", var"#106#group", var"#109#id", var"#110#file", var"#111#line"; var"#113#kwargs"...)
                            end
                        end
                    end
                    #= logging.jl:371 =#
                    Base.CoreLogging.nothing
                end
            end
        #= /Users/anthony/.julia/packages/MLJModels/9mQfs/src/loading.jl:169 =#
        var"#94#proxy" = (MLJModels.MLJModels).info("EvoTreeRegressor"; pkg = nothing, interactive = true)
        #= /Users/anthony/.julia/packages/MLJModels/9mQfs/src/loading.jl:172 =#
        var"#95#handle" = (MLJModels.MLJModels).Handle((var"#94#proxy").name, (var"#94#proxy").package_name)
        #= /Users/anthony/.julia/packages/MLJModels/9mQfs/src/loading.jl:173 =#
        var"#96#dic" = (MLJModels.MLJModels).INFO_GIVEN_HANDLE[var"#95#handle"]
        #= /Users/anthony/.julia/packages/MLJModels/9mQfs/src/loading.jl:174 =#
        (var"#97#path_str", var"#98#pkg_str") = (var"#96#dic"[:load_path], var"#96#dic"[:package_name])
        #= /Users/anthony/.julia/packages/MLJModels/9mQfs/src/loading.jl:175 =#
        var"#99#path" = var"#97#path_str" |> (MLJModels.Meta).parse
        #= /Users/anthony/.julia/packages/MLJModels/9mQfs/src/loading.jl:176 =#
        var"#100#pkg" = MLJModels.Symbol(var"#98#pkg_str")
    end
    begin
        #= /Users/anthony/.julia/packages/MLJModels/9mQfs/src/loading.jl:181 =#
        var"#101#path_components" = MLJModels.Symbol.(MLJModels.split(var"#97#path_str", '.'))                                                                                       
        #= /Users/anthony/.julia/packages/MLJModels/9mQfs/src/loading.jl:184 =#
        var"#102#api_pkg" = var"#101#path_components"[1]
        #= /Users/anthony/.julia/packages/MLJModels/9mQfs/src/loading.jl:185 =#
        var"#103#api_pkg_str" = MLJModels.string(var"#102#api_pkg")
        #= /Users/anthony/.julia/packages/MLJModels/9mQfs/src/loading.jl:187 =#
        if false || true
            #= /Users/anthony/.julia/packages/MLJModels/9mQfs/src/loading.jl:188 =#
            try
                #= /Users/anthony/.julia/packages/MLJModels/9mQfs/src/loading.jl:189 =#
                (MLJModels.MLJModels)._import(Main, var"#102#api_pkg", var"#100#pkg", false)
            catch
                #= /Users/anthony/.julia/packages/MLJModels/9mQfs/src/loading.jl:191 =#
                if true
                    #= /Users/anthony/.julia/packages/MLJModels/9mQfs/src/loading.jl:192 =#
                    (MLJModels.MLJModels).request("The package providing an interface " * "to `EvoTreeRegressor` " * "is not in your " * "current environment.\n" * "What do you want to do? ", "Install $(var"#103#api_pkg_str") in current environment.", "Abort.") == 1 || MLJModels.throw(MLJModels.InterruptException)
                end
                #= /Users/anthony/.julia/packages/MLJModels/9mQfs/src/loading.jl:202 =#
                (MLJModels.MLJModels)._import(Main, :Pkg, :Pkg, false)
                #= /Users/anthony/.julia/packages/MLJModels/9mQfs/src/loading.jl:203 =#
                (MLJModels.Pkg).add(var"#103#api_pkg_str")
            end
        end
        #= /Users/anthony/.julia/packages/MLJModels/9mQfs/src/loading.jl:206 =#
        (MLJModels.MLJModels)._import(Main, var"#102#api_pkg", var"#100#pkg", true)
        #= /Users/anthony/.julia/packages/MLJModels/9mQfs/src/loading.jl:207 =#
        (MLJModels.MLJModels)._eval(Main, var"#99#path")
    end
end

If I first manually add the required package, EvoTrees and then run @iload EvoTreeRegressor this issue disappears.

@ablaom
Copy link
Member Author

ablaom commented May 26, 2021

julia> versioninfo()
Julia Version 1.6.0
Commit f9720dc2eb (2021-03-24 12:55 UTC)
Platform Info:
  OS: macOS (x86_64-apple-darwin19.6.0)
  CPU: Intel(R) Core(TM) i7-8850H CPU @ 2.60GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-11.0.1 (ORCJIT, skylake)
Environment:
  JULIA_LTS_PATH = /Applications/Julia-1.0.app/Contents/Resources/julia/bin/julia
  JULIA_PATH = /Applications/Julia-1.6.app/Contents/Resources/julia/bin/julia
  JULIA_EGLOT_PATH = /Applications/Julia-1.5.app/Contents/Resources/julia/bin/julia
  JULIA_NUM_THREADS = 5
  JULIA_NIGHTLY_PATH = /Applications/Julia-1.7.app/Contents/Resources/julia/bin/julia

@giordano
Copy link
Member

I tried also on macOS in case there is a weird platform-specific issue, but I still can't reproduce your problem there.

The expansion of the macro for you is the same as for me, apart from paths and gensym-generated variables. Not that I was expecting something different, but I figured it'd be good to double check it.

What about using Pkg.Types.Context().env.project.deps to check whether a package is in the current environment, instead of trying to use it and catching the error? The good thing is that it should work in all current versions of Julia since v1.3 (the first one currently supported), the drawback however is that you're hooking into Pkg internals (this is non-public API), so it may break in the future (but it can be dealt with with conditionals inside the macro and see below).

Also, note that Julia v1.7 has the feature of suggesting to install a package if not available, so the try/catch work you're doing here will be a duplicate already available in base Julia.

@ablaom
Copy link
Member Author

ablaom commented May 28, 2021

@giordano Thanks for investigating.

What a pity you are unable to reproduce. I'm inclined to believe the problem lies outside of MLJModels. Why else would reproducibility be an issue? We are using identical Manifests/Project files, yes?

I will try your suggestion as a diagnostic. I'm not enthusiastic about it as a solution for the reasons you give.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants