EKF based MCTS with DPW

[JuliBaCSE]

I am currently getting used to the different MCTS variants.
i was curious if there is any implementation of the paper https://arxiv.org/pdf/1707.09055.pdf. I would love to guide my self and try an implementation with quickpomdp Python interface.

For me the basic algorithm seems like a combination of classic MCTS DPW with a generative model based on an EKF update.
as far as I understood, we sample a possible observation and perform an update step which defines the generative model of the mcts.

additionally I found the PTF-DPW algorithm in https://arxiv.org/pdf/1709.06196.pdf and was curious if there is any further difference between the EKF based and this beside the kalman filter technique.

[zsunberg]

Note, there is an extended version of that paper: https://digital-library.theiet.org/content/journals/10.1049/iet-cps.2019.0045

As you note, the best way to implement that algorithm is to define a belief-space generative MDP and use MCTS with DPW (e.g. with the implementation here https://github.com/JuliaPOMDP/MCTS.jl or your own) to solve it.

I believe this is the code for that project: https://github.com/pslade2/SimulEstControl, but it is very out-of-date at this point, and it is probably easier to do what I described above.

I would love to guide my self and try an implementation with quickpomdp Python interface.

This is probably possible, but there are probably rough edges like the ones you are encountering in #513 .

additionally I found the PTF-DPW algorithm in https://arxiv.org/pdf/1709.06196.pdf and was curious if there is any further difference between the EKF based and this beside the kalman filter technique.

Nope, it is the same except for the belief update technique

[JuliBaCSE]

Hey Zach, Thanks a lot. I will try to get a minimal example running.

[JuliBaCSE]

I just tried to get the DPW solver running swell, but it seems to have similar issues as you already guessed.

Closest candidates are:
insert_state_node!(::MCTS.DPWTree{S, A}, !Matched::S, ::Any) where {S, A}
@ MCTS ~/.julia/packages/MCTS/0KDvG/src/dpw_types.jl:220
insert_state_node!(::MCTS.DPWTree{S, A}, !Matched::S) where {S, A}
@ MCTS ~/.julia/packages/MCTS/0KDvG/src/dpw_types.jl:220

does this need a similar fix as in #513 ?

[zsunberg]

Yeah, it needs a similar fix... will try to get to it this afternoon

[zsunberg]

Ok, I think I fixed that error (JuliaPOMDP/MCTS.jl#104), but there may be more :)

[zsunberg]

Should be fixed in MCTS.jl v0.5.3 which should be registered shortly.