Can EpsGreedyPolicy get correct action, if actions depend on state?

[NeroBlackstone]

Hi, I have checked the POMDPTools build-in EpsGreedyPolicy source code, and I think it can only select actions from action spaces.

But if actions that can be taken are limited by certain states, (a function is used to get available actions from action space, depending on what state is now), this EpsGreedyPolicy can't select the correct action because it selects actions from full action spaces

Is there any built-in function to do this?

If not implemented, I'm willing to contribute a policy. (Maybe need some help

[zsunberg]

Hi @NeroBlackstone

Good point - I am not sure why that policy uses actions(m) instead of actions(m, s). I think it should be updated. If you just want some special case for your own use, it may be easiest to use a function policy, e.g.

function special_eps_greedy(s)
    if rand() < 0.05
        return rand(actions(m, s))
    end
       return greedy(s)
    end
end
policy = FunctionPolicy(special_eps_greedy)

but it would be nice to put fix EpsGreedyPolicy so it does this.

[NeroBlackstone]

Do you think this is a good solution?
#449