Many results on generalization and distribution-free inference depend on the stability of a regression algorithm, which is often defined as the property that predictions on a new test point are not substantially altered by removing a single point at random from the training set.
