Particle filter application

[Etwari3]

Hello,

The particle filter picks the costliest option in some iterations. Is that correct? Did I miss something?

I have attached an example.

Thank you.
example.pdf

[zsunberg]

I don't have enough context on the problem to make a confident judgement, but consider the following:

1. 10 particles in the particle filter probably won't be enough, perhaps try to increase to 1000
2. POMCP tries to optimize the long term reward, not just the reward on the next step
3. With the default random rollout policy, POMCP may not perform well

[Etwari3]

@zsunberg, Thank you for the reply.

On number 3, does that mean I would have to define policies for POMCP?

I am a bit confused about belief state formulation. When formulating the POMDP problem model, do I need to define only the core states or enumerate all the belief states?

To give context, I am trying to formulate discrete and continuous POMDP models for a maintenance problem. POMCP and ARDESPOT are the ones I have picked to solve the problem.

Once again, Thank you.

[zsunberg]

Yes, both POMCP and ARDESPOT will benefit greatly from a decent rollout policy. One good way to determine a good policy is to solve the underlying MDP and use that with POMCP via the FORollout value estimator.

Also, to see whether POMCP is currently performing well, you could compare it to a random policy using monte Carlo simulations.

[Etwari3]

Thank you. I will try them all.