A random process for which ensemble averages are identical to time or spatial averages is said to be ergodic. (See Stochastic Process for definitions of averages.) Both types of averaging are acceptable ways of describing a random process.
Ensemble averages are directly linkend to a probabilistic formulation of random processes, i.e., to all realizable states to each of which can be ascribed or derived a unique probability of occurrence. Most current statistical theories use ensemble averaging, especially so in the theory of Turbulence. However, in practice, such averages are difficult to measure and recourse is made to the measurement of time or spatial averages instead. Meaningful comparison between theory and measurement thus relies on the process being ergodic. It is valid for a stationary random process [see, e.g., Beran (1968)]; however, in all other cases ‘ergodicity’ is generally an hypothesis to be validated by experiment.
For readable accounts see Beran (1968) and Yaglom (1962); more in depth discussions are to be found in Truesdell (1961) and in the basic article by Birkhoff (1931).
REFERENCES
Beran, M. 1. (1968) Statistical Continuum Theories, Interscience Publishers (JW).
Birkoff, G. D. (1931) Proof of the Ergodic Theorem Proc. Natl. Acad. Sci. 17: 656.
Truesdell, C. (1961) Ergodic theories. Proc. Intern. School Phys. “E. Fermi” XIV Course: 21. Academic Press, New York.
Yaglom, A. (1962) An Introduction to Mathematical Stationary Random Functions. PrenticeHall, Englewood Cliffs, N. J.
References

Beran, M. 1. (1968) Statistical Continuum Theories, Interscience Publishers (JW). DOI: 10.1119/1.1974326
 Birkoff, G. D. (1931) Proof of the Ergodic Theorem Proc. Natl. Acad. Sci. 17: 656. DOI: 10.1073/pnas.17.12.656
 Truesdell, C. (1961) Ergodic theories. Proc. Intern. School Phys. â€œE. Fermiâ€ XIV Course: 21. Academic Press, New York.
 Yaglom, A. (1962) An Introduction to Mathematical Stationary Random Functions. PrenticeHall, Englewood Cliffs, N. J.