In Chapter 1, we give a brief introduction to the classical theory on both discrete and
continuous time Markov chains. The relationship between Markov chains of finite states and matrix theory will also be discussed. Some classical iterative methods for solving linear systems will also be introduced. We then give the basic theory and algorithms for standard hidden Markov model (HMM) and Markov decision process (MDP).
Chapter 2 discusses the applications of continuous time Markov chains to model queueing systems and discrete time Markov chain for computing the PageRank, the ranking of website in the Internet.
Chapter 3 studies remanufacturing systems. We present Markovian models for re-manufacturing, closed form solutions and fast numerical algorithms are presented for solving the systems.
In Chapter 4, Hidden Markov models are applied to classify customers. We proposed a simple hidden Markov model with fast numerical algorithms for solving the model parameters. An application of the model to customer classification is discussed. Chapter 5 discusses Markov decision process for customer lifetime values. Customer Lifetime Values (CLV) is an important concept and quantity in marketing management. We present an approach based on Markov decision process to the calculation of CLV with practical data.
In Chapter 6, we discuss higher-order Markov chain models. We propose a class of higher-order Markov chain models with lower order of model parameters. Efficient numerical methods based on linear programming for solving the model parameters are presented. Applications to demand predictions, inventory control, data mining and DNA sequence analysis are discussed. In
Chapter 7, multivariate Markov models are discussed. We present a class of multivariate Markov chain model with lower order of model parameters. Efficient numerical methods based on linear programming for solving the model parameters are presented. Applications to demand predictions and gene expression sequences are discussed. In Chapter 8, higher-order hidden Markov models are studies.We proposed a class of higher-order hidden Markov models with efficient algorithm for solving the model parameters.
This book is aimed at students, professionals, practitioners, and researchers in applied mathematics, scientific computing, and operational research, who are interested in the formulation and computation of queueing and manufacturing systems. Readers are expected to have some basic knowledge of probability theory Markov processes and matrix theory. research described herein is supported in part by RGC grants.We are indebted It is our pleasure to thank the following people and organizations.
Approved by Englishcology
