Zhenyu Yue

Consistent Variance Estimation for Q-Function Estimators in Finite-Horizon MDP Tree Search

Zhenyu Yue, Jie Xu, Chun-Hung Chen, Hadi El-Amine, Michael C. Fu

We study the variance of Q-function estimators in finite-horizon, finite-state Markov decision process (MDP) tree search. We show that the variance decomposes into three components attributed to the immediate reward collected, probabilistic state transitions, and uncertainty in f…Read more