| 10BTAVADD4 | statement | theorem H.2 | arxiv |
| Horizon bias and planning depth | statement | corollary J.1 | arxiv |
| Reference trajectory | statement | lemma H.1 | arxiv |
| 197Q7NX6H9 | definition | definition E.1 | arxiv |
| Distance between the planning solution and the target | statement | lemma J.5 | arxiv |
| Span seminorms | definition | definition 1 | arxiv |
| Jacobian bound over a global block | statement | lemma H.7 | arxiv |
| Level coverage | statement | lemma J.9 | arxiv |
| Well-posedness | statement | proposition J.2 | arxiv |
| Mixing properties of $\Omega$-regularization | statement | lemma 1 | arxiv |
| Depth-indexed average-path representation | statement | lemma J.3 | arxiv |
| Mixing properties of $\Omega$-regularization | statement | lemma G.2 | arxiv |
| $\alpha_j\equiv1$: empirical $\Omega$-regularized value iteration | statement | corollary H.4 | arxiv |
| 4NNGXA530P | statement | lemma G.3 | arxiv |
| Uniqueness of $Q^*_\Omega$ up to an additive constant | statement | proposition G.4 | arxiv |
| Contraction of the virtual flow at radius $R$ | statement | lemma H.17 | arxiv |
| Constant step size | statement | corollary H.2 | arxiv |
| Mean value inequality in span seminorm | statement | proposition E.1 | arxiv |
| Epoch $\Omega$-regularized $Q$-learning with $\alpha_j\equiv 1$ | statement | theorem 2 | arxiv |
| $\rho^*_\Omega$ is the optimal regularized gain | statement | proposition G.5 | arxiv |
| Good event | statement | lemma H.12 | arxiv |
| Gain bounds from an approximate fixed point | statement | lemma G.9 | arxiv |
| Good event | statement | lemma H.22 | arxiv |
| Properties of $\Lambda_\Omega$ | statement | lemma G.1 | arxiv |
| The terminal action does not affect the contraction | statement | lemma E.4 | arxiv |
| Synchronous $\Omega$-regularized $Q$-learning for $m=1$ | statement | corollary H.3 | arxiv |
| Per-block contraction | statement | lemma H.8 | arxiv |
| Global properties of $\Omega_\epsilon$ | statement | lemma G.11 | arxiv |
| Mixing-regularization objects | definition | definition 2 | arxiv |
| Synchronous $\Omega$-regularized $Q$-iteration | statement | corollary 1 | arxiv |
| Pointwise regularization modulus | definition | definition G.2 | arxiv |
| The minimal horizon | statement | proposition D.2 | arxiv |
| Properties of $\Omega_\epsilon$ | statement | lemma G.10 | arxiv |
| Per-epoch noise | statement | lemma H.19 | arxiv |
| Noise | statement | lemma J.11 | arxiv |
| Tracking up to the exit time | statement | lemma H.13 | arxiv |
| Exact form of the epoch update | statement | lemma H.14 | arxiv |
| Epoch $\Omega$-regularized $Q$-learning for $m=1$ | statement | corollary H.5 | arxiv |
| The $\alpha$-divergence regularizer | statement | theorem K.1 | arxiv |
| Union graph | definition | definition D.1 | arxiv |
| Uniqueness of the gain | statement | proposition G.3 | arxiv |
| Contraction of products of at least $m$ regularized greedy kernels | statement | lemma H.5 | arxiv |
| Block martingale | statement | lemma H.20 | arxiv |
| Span seminorm of a row-stochastic matrix | statement | lemma E.6 | arxiv |
| Reachability under $\mu$ | statement | lemma J.1 | arxiv |
| Self-bounding lower tail | statement | lemma E.1 | arxiv |
| The gain gap vanishes with $\tau$ | statement | corollary G.2 | arxiv |
| E1F77BVDTC | statement | proposition G.1 | arxiv |
| Performance of the greedy policy | statement | corollary G.3 | arxiv |
| Contraction of the virtual flow at radius $R$ | statement | lemma H.10 | arxiv |
| Variational representation of the discounted values | statement | lemma G.5 | arxiv |
| E52X19KRR3 | statement | proposition G.2 | arxiv |
| Well-posedness | statement | proposition H.2 | arxiv |
| Operator span seminorm | definition | definition E.3 | arxiv |
| Geometric sum | statement | lemma J.7 | arxiv |
| Epoch noise: martingale structure and range | statement | lemma H.18 | arxiv |
| Normalized values are uniformly bounded | statement | lemma G.6 | arxiv |
| Cost of $\Omega$-regularization | statement | proposition G.6 | arxiv |
| $\epsilon$-floor-wrapped regularization | definition | definition G.4 | arxiv |
| Properties of $\Omega^*$ | statement | lemma F.1 | arxiv |
| Conditional cover time at a deterministic rate | statement | lemma H.15 | arxiv |
| Total-variation distance | definition | definition E.1 | arxiv |
| HP57TBWYJT | statement | lemma H.3 | arxiv |
| Discounted fixed point | statement | lemma G.4 | arxiv |
| Localized radii, exit time and localized constants | definition | definition H.2 | arxiv |
| Epoch and block counts | statement | lemma H.16 | arxiv |
| Four landmarks | statement | corollary K.1 | arxiv |
| Mixing-regularization objects | definition | definition G.1 | arxiv |
| Existence of a fixed point for $\mT_\Omega$ | statement | theorem G.2 | arxiv |
| Properties of $\mT_\Omega$ | statement | corollary F.1 | arxiv |
| Finite-time bound for $\Omega$-regularized synchronous $Q$-learning | statement | theorem H.1 | arxiv |
| Tracking within a block | statement | lemma H.21 | arxiv |
| Dobrushin coefficient | definition | definition E.2 | arxiv |
| Master lemma | statement | lemma J.8 | arxiv |
| Finite-time bound for epoch planning | statement | theorem J.1 | arxiv |
| Epoch $Q$-learning with resets | statement | theorem J.2 | arxiv |
| Extended trajectory | definition | definition H.1 | arxiv |
| Contraction of a product of depth kernels | statement | lemma J.6 | arxiv |
| Epoch length | statement | lemma J.12 | arxiv |
| Effective updates within a global block | statement | lemma H.6 | arxiv |
| Operator span seminorm | statement | lemma E.5 | arxiv |
| No escape | statement | proposition H.3 | arxiv |
| $\epsilon$-floored simplex | definition | definition G.3 | arxiv |
| Uniformly bounded absorption time | statement | lemma G.8 | arxiv |
| Synchronous $\Omega$-regularized $Q$-learning | statement | theorem 1 | arxiv |
| Horizon bias and model-based planning | statement | proposition J.1 | arxiv |
| Extension of the target to the transient states | statement | lemma J.2 | arxiv |
| SXEQEWY0VH | statement | proposition D.1 | arxiv |
| Sub-blocks and global blocks | definition | definition H.1 | arxiv |
| Boundedness along the frozen flow | statement | lemma H.4 | arxiv |
| TXYXR7MCCF | example | example G.1 | arxiv |
| Stopped evaluation | statement | lemma G.7 | arxiv |
| The planning solution is a restriction | statement | lemma J.4 | arxiv |
| Cover time with resets | statement | lemma J.13 | arxiv |
| Monotonicity of the error | statement | lemma H.2 | arxiv |
| Full-knowledge updates for $m=1$ | statement | corollary H.1 | arxiv |
| Stopped noise | statement | lemma H.11 | arxiv |
| Exact forgetting of the initialization | statement | lemma J.10 | arxiv |
| $\mathcal{C}^1$ case: an exact averaged kernel | statement | corollary E.1 | arxiv |
| Doeblin and Dobrushin coefficients | statement | lemma E.3 | arxiv |
| No escape | statement | proposition H.1 | arxiv |
| Y9ZB9PET4A | statement | corollary G.1 | arxiv |
| Existence of a fixed point for $\mT_\Omega$ | statement | theorem G.1 | arxiv |
| Epoch $\Omega$-regularized $Q$-planning | statement | theorem I.1 | arxiv |
| Number of completed blocks | statement | lemma H.9 | arxiv |
| Maximal Azuma--Hoeffding inequality with predictable ranges | statement | lemma E.2 | arxiv |
