ERROR: MethodError: Cannot convert an object of type Array{Float64,1} to an object of type Int64

[danortega2014]

Hi all! I was wondering if I could get some help with a generative POMDP I'm creating. I believe my issue is defining the state space whichcan't be explicity defined because I am repetitively adding to it in the transition function. Here is a link to the code:

https://github.com/danortega2014/v3-/blob/main/README.md

I get the error "ERROR: MethodError: Cannot convert an object of type Array{Float64,1} to an object of type Int64"

It also might be worth mentioning that the structure of the POMDP is a Tuple{Int64, Symbol, Tuple{Int64,Int64, Int64}} .

Thanks again for the help!

[zsunberg]

Hi @danortega2014

This error results because your initialstate function returns an actual state. It should return a distribution of states. To see why it outputs this particular error, run rand(initialstate(m)). It is having trouble converting that to a state, which it thinks should be an Int

[zsunberg]

I'd recommend using a static array from StaticArrays.jl instead of a Tuple for your observation. Then convert_o will not be needed and everything will be efficient.

In any case, the most robust way to implement convert_o if you are using a Tuple is probably

function POMDPs.convert_o(::Type{A}, o::Tuple{Float64,Float64,Float64,Float64}, m::firmgame) where A<:AbstractArray
    convert(A, [o...])
end

(the ... is called "splatting")

[zsunberg]

The +(::Deterministic{Float64}, ::Float64) error is because you are still returning a distribution (a Deterministic object is a distribution, just with all the probability concentrated on one state) from gen; You should just return a float :) e.g.

s1  = s +  rand(Normal(0.0, 1.0))
return (sp=s1,)

does that make sense?

[danortega2014]

Yes thank you for all your help! I implemented the convert_o as you suggested and fixed the transition and observation function. However, I'm not sure if I implemented the convert_o correctly as i get the error:

ERROR: MethodError: no method matching +(::Tuple{Float64}, ::Float64)
Closest candidates are:
  +(::Any, ::Any, ::Any, ::Any...) at operators.jl:538
  +(::Bool, ::T) where T<:AbstractFloat at bool.jl:103
  +(::Missing, ::Number) at missing.jl:115
  ...
Stacktrace:
 [1] gen(::firmgame, ::Float64, ::Symbol, ::Random.MersenneTwister) at .\REPL[26]:21
 [2] macro expansion at C:\Users\danor\.julia\packages\POMDPs\agdPZ\src\gen_impl.jl:23 [inlined]
 [3] genout(::DDNOut{(:sp, :o, :r, :info)}, ::firmgame, ::Float64, ::Symbol, ::Random.MersenneTwister) at C:\Users\danor\.julia\packages\POMDPs\agdPZ\src\gen_impl.jl:19
 [4] (::RLInterface.var"#23#f#4")(::firmgame, ::Float64, ::Symbol, ::Random.MersenneTwister) at C:\Users\danor\.julia\packages\POMDPs\agdPZ\src\generative.jl:65
 [5] step!(::RLInterface.POMDPEnvironment{Array{Float32,1},firmgame,Float64,Random.MersenneTwister}, ::Symbol) at C:\Users\danor\.julia\packages\RLInterface\Vw7YT\src\RLInterface.jl:107
 [6] #populate_replay_buffer!#13 at C:\Users\danor\.julia\packages\DeepQLearning\bpMyG\src\prioritized_experience_replay.jl:116 [inlined]
 [7] initialize_replay_buffer(::DeepQLearningSolver{EpsGreedyPolicy{LinearDecaySchedule{Float64},Random._GLOBAL_RNG,Array{Symbol,1}}}, ::RLInterface.POMDPEnvironment{Array{Float32,1},firmgame,Float64,Random.MersenneTwister}, ::Dict{Symbol,Int64}) at C:\Users\danor\.julia\packages\DeepQLearning\bpMyG\src\solver.jl:183
 [8] solve(::DeepQLearningSolver{EpsGreedyPolicy{LinearDecaySchedule{Float64},Random._GLOBAL_RNG,Array{Symbol,1}}}, ::RLInterface.POMDPEnvironment{Array{Float32,1},firmgame,Float64,Random.MersenneTwister}) at C:\Users\danor\.julia\packages\DeepQLearning\bpMyG\src\solver.jl:48        
 [9] solve(::DeepQLearningSolver{EpsGreedyPolicy{LinearDecaySchedule{Float64},Random._GLOBAL_RNG,Array{Symbol,1}}}, ::firmgame) at C:\Users\danor\.julia\packages\DeepQLearning\bpMyG\src\solver.jl:37
 [10] top-level scope at REPL[38]:1

Here's the updated code: https://github.com/danortega2014/v3-/blob/main/README.md

[zsunberg]

When you do sp = (s1,), you are making sp a tuple. It should just be a float, ie. sp=s1

[danortega2014]

Thank you so much for all your help it now works!