In statistics, a simple random sample is a subset of individuals chosen (one by one) from a population. Each individual is chosen randomly such that each individual has the same probability of being chosen at any stage during the sampling process, and each subset of k individuals has the same probability of being chosen for the sample as any other subset of k individuals. From a population of size N with finite variance, a simple random sample of size n is drawn without replacement, and a real-valued characteristic X measured to yield observations Xj(j =1,..., n).

Prove that the expected squared error of the sample mean, i.e. the variance of the sample mean, is smaller than that of the mean of a simple random sample of the same size n drawn with replacement.

The question came from the book "A Course in Mathematical Statistics and Large Sample Theory" by Rabi Bhattacharya, Lizhen Lin, and Victor Patrangenaru.