3 credits
Fall 2025 Lecture Upper Division
An introduction to some classes of stochastic processes that arise in probabilistic models of time-dependent random processes. The main stochastic processes studied will be discrete time Markov chains and Poisson processes. Other possible topics covered may include continuous time Markov chains, renewal processes, queueing networks, and martingales.
Learning Outcomes1Identify a verbal description of a stochastic process, identify the process as a Markov process and compute the transition matrix for the Markov process.
2Compute the stationary distribution of a Markov process and use this to draw conclusions on the long-term asymptotic behavior of the stochastic process using appropriate limit theorems for Markov processes.
3Understand what conditions are necessary to be able to apply the main limit theorems for Markov chains; also, students should be able to give examples of Markov chains where these limit theorems do not hold because the assumptions of the theorems are violated.
4Calculate hitting probabilities or expected hitting times of a Markov chain by solving a system of linear equations.
5Know how to use mathematics software (e.g. Matlab) to perform computations for Markov chains on large state spaces.
6Understand the thinning and superposition properties of Poisson processes and how to use these properties in computations.
Prerequisites
made by Purdue students.