Line 22: src/ProgressiveWidening.jl

[himanshugupta1009]

This seems incomplete. Shouldn't we keep sampling an action either until all the actions have been already added or you find an action that has not been added? Here it seems to only check one sampled action and if this action already exists, it doesn't add anything instead of looking for another action that has not already been added.

[WhiffleFish]

Nah, your proposed method would require a rejection sampling loop that could get stuck for an unpredictable amount of time. This would especially be the case for large discrete action spaces paired with similarly large $k_a$.

The current method for progressive action sampling is also employed by other JuliaPOMDP solvers like POMCPOW and MCTS. While it may not be perfect, it is simple, has a predictable runtime, and yields strong performance in benchmarks.

[WhiffleFish]

Granted, it may also be worthwhile admitting a custom action sampler as an option to the solver.