This textbook investigates the theory and application of Markov chains and related stochastic processes. The author, Pierre Brémaud, provides a self-contained review of probability theory, supplemented by appendices with proofs for calculus, algebra, and number theory prerequisites, to facilitate self-study. The book progresses from elementary Markov chain theory to advanced topics, covering both discrete and continuous time, as well as connected subjects like Gibbs fields, regenerative processes, Monte Carlo simulation, simulated annealing, and queuing theory. It is designed to be accessible to students and researchers across various scientific and engineering disciplines.

The book is presented as an up-to-date textbook on stochastic processes, aiming for accessibility for students and researchers in operations research, electrical engineering, physics, and biology. The pedagogical approach emphasizes slow and careful mathematical development with problems of varying difficulty to aid self-study. The structure, starting from basic Markov chains and progressing to advanced topics, suggests a comprehensive and foundational treatment of the subject matter.