School of Engineering and Technology, (SET)
 
AT72.08 : Stochastic Decision Models  3(3-0)
 
Course objectives:

The objective of this course is to impart knowledge on mathematical modeling process of decision problems in complex stochastic environments. This course covers stochastic operations research models, algorithms, and applications, including Markov chains and queuing models. Renewal theory, reliability theory, and stochastic models for manufacturing systems are also introduced. Further this course covers the analytical models which are the complements to the discrete event simulation approach.

 
Learning Outcomes:
The students on the completion of this course would be able to:
•         Apply stochasticity into decision making process
•         Analyze random processes through the use of Markov Chain and Renewal Theory.
•         Apply stochastic theories and models to find solution for decision making problems.
 
Pre-requisite(s):

None
 
Course Outline:
I.            Review on Probability Theory
1.       Experiments, Events, Sample Space
2.       Laws of Addition and Multiplication
3.       Conditional Probability and Expectation
4.       Posterior Probability and Bayes’ Theorem

II.           Review on Random Variable
1.       Discrete vs. Continuous Random Variables
2.       Expectation of Function of Random Variable
3.       Variance
4.       Moment Generating Function
5.       Joint Distributed Random Variables
6.       Conditional Expectation

III.          Discrete Time Markov Chains
1.       Stochastic Process
2.       Event Graph and Discrete Time Markov Chain
3.       Chapman-Kolmogorov Equations
4.       Classification of States
5.       Limiting Probabilities

IV.          Poisson Processes
1.       Properties of Exponential Distribution
2.       Counting Process - Poisson Process
3.       Generalizations of Poisson Process
V.           Continuous Time Markov Chains
1.       Continuous Time Markov Chains
2.       Birth-Death Processes
3.       Chapman-Kolmogorov Equations
4.       Limiting Probabilities

VI.          Renewal Theory
1.       Renewal Reward Processes
2.       Regenerative Processes
3.       Semi-Markov Processes

VII.         Queuing Theory
1.       M/M/1, M/M/k, M/G/1 and G/M/1 Models
2.       Models with Limited Queue Capacity
3.       Networks of Queues and Multiserver Queues
4.       Priority Queues

VIII.        Reliability Theory
1.       Structure Function
2.       Minimum Path and Minimum Cut Sets
3.       Reliability of Systems of Independent Components

IX.          Discrete Event Simulation
1.      Generation of Pseudo Random Numbers
2.       Simulation of Random Variables
3.       Simulation of Stochastic Processes
 
 
Learning Resources:

Textbook:
S.M. Ross:  Introduction to Probability Models, 10th edition, Academic Press, 2010.
 
Reference Books:
1.       A.H-S Ang, and W.H. Tang: Probability Concepts in Engineering Planning and Design, Vol. I: Basic Principles, John Wiley and Sons, 1973.
2.       A.H-S Ang, and W.H. Tang: Probability Concepts in Engineering Planning and Design, Vol. II: Decision, Risk, and Reliability, John Wiley and Sons, 1984.
3.      J.A. Buzacott, and J.G. Shanthikumar: Stochastic Models of Manufacturing Systems, 1st edition, Prentice-Hall, 1993.
4.       S.M. Ross: A First Course in Probability, 9th edition, Pearson, 2012.
5.       S.M. Ross: Stochastic Process, 2nd edition, Wiley, 1996.
6.       H.A. Taha: Operations Research : An Introduction, 9th edition, Prentice Hall, 2010.  
 
Journals and Magazines:
1.    European Journal of Operational Research, Elsevier.
2.    Journal of the Operational Research Society, Palgrave Macmillan.
3.    Management Science, Informs.
4.    Queuing Systems, Springer.

Others: Lecture Notes
 
Time Distribution and Study Load:
Lecture hours:   45 hours
Tutorials: 15 hours
Home assignments/Self-study: 120 hours
 
Teaching and Learning Methods:
The teaching is done via lectures by the instructor. The learning method includes individual assignments and tutorials.
 
Evaluation Scheme:
Mid-semester examination 30%, home assignments 20%, final examination 50%.   All examinations are open-book.

An “A” would be awarded if a student shows a deep understanding of the basic as well as advanced knowledge learned through home assignments and exam results. A “B” would be awarded if a student shows an overall understanding of all basic topics and some advanced topics. A “C” would be given if a student only meets average expectation in understanding and application of basic knowledge. A “D” would be given if a student does not meet expectations in both understanding and application of the basic knowledge.

 
Instructor(s):
SECTION NAME
A Prof. Huynh Trung Luong