An identity for the bias in Markov reward processes with applications to Markov chain perturbation and Kemeny’s constant

Given a unichain Markov reward process (MRP), we provide an explicit expression for the bias values in terms of mean first passage times. This result implies a generalization of known Markov chain perturbation bounds for the stationary distribution to the case where the perturbed chain is not irreducible. It further yields an improved perturbation bound in 1-norm. As a special case, Kemeny’s constant can be interpreted as the translated bias in an MRP with constant reward −1, which offers an intuitive explanation why it is a constant.