Uncertainty in Model Based RL

artificial-intelligence reinforcement-learning

St John St John Jun 13, 2020 · 1 min read
Uncertainty in Model Based RL

Perhaps we could use uncertainty estimation to detect where the model may be wrong and then correct for these potential errors without having to collect much more data. This uncertainty is useful in many respects. It maybe that a certain action yields high expected reward but there is a possibility (encoded in the uncertainty) that the agent takes too far a step and ’falls off the cliff’. A pessimistic agent may want to avoid this. Or perhaps this uncertainty can guide exploration as in UCB or Thompson sampling for an optimistic agent. If the agent only chooses actions for which it believes it will obtain the highest expected reward, the agent can avoid falsely exploiting the model.

St John
Written by St John
Author of the Asking Why Blog - a personal blog and website with everything I find interesting.