STT 886: Stochastic processes and applications (Fall 2019)

Welcome to the course webpage!

Useful resources will be posted here.

• Course syllabus

• Lecture materials
  • Demo: convergence of Markov chains
• Homework & Solutions
  • Homework 1: Probability review – Solutions – Code
  • Homework 2: Random variables – Solutions – Code
  • Homework 3: Conditioning – Solutions – Code
  • Homework 4: Markov chains – Solutions
  • Homework 5: Applications of Markov chains – Solutions – Code
  • Homework 6: Exponential distribution and Poisson process – Solutions – Code
  • Homework 7: Poisson process, applications and generalizations – Solutions
  • Homework 8: Continuous-time Markov chains – Solutions – Code
• Exams and quizzes
  • Midterm 1: Markov chains – Solutions
  • Quizz 1: Exponential distribution and Poisson process – Solutions
  • Midterm 2: Point processes – Solutions
• Readings
  • Metropolis-Hastings – An intuitive presentation of the Metropolis-Hastings algorithm by Chib and Greenberg.
  • A tutorial on partially observable Markov decision processes (M. Littman, 2009) – Available online via MSU libraries.
• Useful links:
  • STT class page
  • Python basics
  • Colab – Cloud Jupyter notebook environment
  • Nbviewer – Cloud Jupyter notebooks viewer
  • Symbolab – Cloud symbolic calculus platform