| Preface | p. v |
| Discrete-time Singularly Perturbed Markov Chains | p. 1 |
| Singularly Perturbed Markov Chains | p. 2 |
| Motivation | p. 2 |
| Preliminary | p. 3 |
| Singularly Perturbed Models | p. 5 |
| Motivating Examples | p. 9 |
| Asymptotic Expansions | p. 12 |
| Occupation Measures | p. 18 |
| Nonstationary Markov Chains and Applications | p. 23 |
| Asymptotic Properties for Smooth Transition Matrices | p. 23 |
| Bounded and Measurable Transition Matrices | p. 29 |
| Applications to Nearly Optimal Controls | p. 32 |
| Notes and Remarks | p. 36 |
| Notes on the Literature | p. 36 |
| Possible Future Research Topics | p. 37 |
| References | p. 38 |
| Nearly Optimal Controls of Markovian Systems | p. 43 |
| Singularly Perturbed MDP | p. 44 |
| Irreducible MDP under Discounted Cost | p. 46 |
| Irreducible MDP under Long-Run Average Cost | p. 51 |
| MDP with General Transition Matrices | p. 54 |
| Historical Notes | p. 61 |
| Hybrid LQG Control | p. 62 |
| Aggregation and Approximation | p. 66 |
| Asymptotic Optimality | p. 71 |
| Hybrid LQG with General Transition Matrices | p. 75 |
| A Numerical Example | p. 80 |
| Historical Notes | p. 82 |
| Conclusions | p. 83 |
| References | p. 83 |
| Stochastic Approximation, with Applications | p. 87 |
| SA Algorithms | p. 87 |
| General Convergence Theorems by TS Method | p. 90 |
| Convergence Theorems Under State-Independent Conditions | p. 99 |
| Applications | p. 102 |
| Application to Optimization | p. 102 |
| Application to Signal Processing | p. 105 |
| Notes | p. 107 |
| References | p. 108 |
| Performance Potential Based Optimization and MDPs | p. 111 |
| Sensitivity Analysis and Performance Potentials | p. 112 |
| Markov Decision Processes | p. 116 |
| Problems with Discounted Performance Criteria | p. 118 |
| Single Sample Path Based Implementations | p. 121 |
| Time Aggregation | p. 123 |
| Connections to Perturbation Analysis | p. 126 |
| Application Examples | p. 128 |
| Notes | p. 130 |
| References | p. 134 |
| An Interior-Point Approach to Multi-Stage Stochastic Programming | p. 137 |
| Two-Stage Stochastic Linear Programming | p. 139 |
| A Case Study | p. 142 |
| Multiple Stage Stochastic Programming | p. 144 |
| An Interior Point Method | p. 146 |
| Finding Search Directions | p. 156 |
| Model Diagnosis | p. 164 |
| Notes | p. 167 |
| References | p. 168 |
| A Brownian Model of Stochastic Processing Networks | p. 171 |
| Preliminaries | p. 172 |
| Stochastic Processing Network Model | p. 174 |
| Examples of Stochastic Processing Networks | p. 176 |
| Scheduling Control of Multiclass Queueing Network | p. 176 |
| A Simple Queueing Network with both Scheduling and Routing | p. 177 |
| An Assemble-To-Order System | p. 179 |
| Brownian Model for Stochastic Processing Network | p. 181 |
| Comparison to Harrison's Brownian Model | p. 183 |
| Extensions | p. 184 |
| Brownian Approximation via Strong Approximation | p. 185 |
| Notes | p. 186 |
| Appendix: Strong Approximation vs. Heavy Traffic Approximation | p. 187 |
| References | p. 191 |
| Stability of General Processing Networks | p. 193 |
| Motivating Simulations | p. 195 |
| Open Processing Networks | p. 201 |
| Network Description | p. 202 |
| The Standard Network and Dispatch Policies | p. 205 |
| Production Policies and Sensible Policies | p. 206 |
| Rate Stability | p. 209 |
| Network and Fluid Model Equations | p. 210 |
| Network Dynamics | p. 210 |
| Fluid Models | p. 214 |
| Connection between Processing Networks and Fluid Models | p. 217 |
| Connection between Artificial and Standard Fluid Models | p. 219 |
| Batch Processing Networks and Normal Policies | p. 219 |
| Stability under Sensible Production Policies | p. 222 |
| Examples of Stable Policies | p. 223 |
| Early Steps First | p. 223 |
| Generalized Round Robin | p. 228 |
| Extensions | p. 230 |
| Appendix | p. 232 |
| Departures As a Function of Server Effort | p. 232 |
| Proofs of Lemmas 7.12 and 7.18 | p. 236 |
| Notes | p. 240 |
| References | p. 241 |
| Large Deviations, Long-Range Dependence, and Queues | p. 245 |
| Fractional Brownian Motion and a Related Filter | p. 246 |
| Moderate Deviations for Sample-Path Processes | p. 248 |
| MDP for the Filtered Process | p. 252 |
| Queueing Applications: The Workload Process | p. 258 |
| Verifying the Key Assumptions | p. 267 |
| Notes | p. 274 |
| References | p. 275 |
| Markowitz's World in Continuous Time, and Beyond | p. 279 |
| The Mean-Variance Portfolio Selection Model | p. 280 |
| A Stochastic LQ Control Approach | p. 283 |
| Efficient Frontier: Deterministic Market Parameters | p. 285 |
| Efficient Frontier: Random Adaptive Market parameters | p. 292 |
| Efficient Frontier: Markov-Modulated Market Parameters | p. 296 |
| Efficient Frontier: No Short Selling | p. 299 |
| Mean-Variance Hedging | p. 300 |
| Notes | p. 303 |
| References | p. 305 |
| Variance Minimization in Stochastic Systems | p. 311 |
| Variance Minimization Problem | p. 311 |
| General Variance Minimization Problem | p. 314 |
| Variance Minimization in Dynamic Portfolio Selection | p. 316 |
| Variance Minimization in Dual Control | p. 323 |
| Notes | p. 330 |
| References | p. 330 |
| A Markov Chain Method for Pricing Contingent Claims | p. 333 |
| The Markov Chain Pricing Method | p. 334 |
| The Black-Scholes (1973) Pricing Model | p. 336 |
| Choosing the Set of Asset Prices | p. 337 |
| Computing Transition Probabilities and Option Prices | p. 338 |
| An Illustrative Example | p. 339 |
| A Markov Chain Interpretation of Binomial Tree | p. 341 |
| Numerical Examples | p. 343 |
| The GARCH Pricing Model | p. 347 |
| Choosing the Set of Discrete Prices and Volatilities | p. 349 |
| Computing Transition Probabilities and Option Prices | p. 350 |
| Numerical Examples | p. 351 |
| Valuing Exotic Options | p. 355 |
| Appendix: The Conditional Expected Value of h[subscript T*] and h[superscript 2 subscript T*] | p. 360 |
| References | p. 361 |
| Stochastic Network Models and Optimization of a Hospital System | p. 363 |
| A Multi-Site Service Network Model | p. 364 |
| Patient Flow Management | p. 366 |
| Capacity Design | p. 371 |
| Switching Costs and Quality of Service | p. 382 |
| Insights and Future Research Directions | p. 387 |
| Notes | p. 390 |
| References | p. 391 |
| Optimal Airline Booking Control with Cancellations | p. 395 |
| Preliminaries | p. 396 |
| Model Description | p. 396 |
| Optimality Conditions and the Value Function | p. 398 |
| The Minimum Acceptable Fare and Threshold Control | p. 400 |
| The Minimum Acceptable Fare | p. 400 |
| Properties of MAF | p. 402 |
| Threshold Control and Computation of the Value Function | p. 412 |
| Extensions of the Basic Model | p. 414 |
| Time-Dependent Air Fares | p. 414 |
| Fare-Dependent Partial Refunds | p. 414 |
| Numerical Experiments | p. 418 |
| Notes | p. 421 |
| References | p. 424 |
| Information Revision and Decision Making in Supply Chain Management | p. 429 |
| Industrial Examples | p. 429 |
| The Procurement of Micro-Controller | p. 430 |
| Analysis of Demand Forecast Data | p. 431 |
| The Deregulated Energy Markets | p. 435 |
| A Multi-Period, Two-Decision Model | p. 435 |
| A One-Period, Multi-Information Revision Model | p. 443 |
| Applications | p. 450 |
| Decision-Making with Two Procurement Alternatives | p. 450 |
| The Application to Deregulated Energy Markets | p. 450 |
| Notes | p. 451 |
| References | p. 455 |
| About the Contributors | p. 459 |
| Table of Contents provided by Rittenhouse. All Rights Reserved. |
