A fundamental question in the theory of reinforcement learning is what properties govern our ability to generalize and avoid the curse of dimensionality.