Preface | p. xi |
Examples and Introduction | p. 1 |
Outline of the Chapter | p. 1 |
An Introductory Example: Controlling the Temperature of a Fluid Flow | p. 1 |
An Example from Internet Regulation | p. 5 |
Models With Finite-Dimensional Equivalences | p. 10 |
Weak Convergence and Martingales | p. 13 |
Outline of the Chapter | p. 13 |
Weak Convergence | p. 14 |
Basic Theorems of Weak Convergence | p. 15 |
The Function Spaces D(S;I) | p. 17 |
Martingales and the Martingale Method | p. 18 |
Martingales | p. 18 |
Verifying That a Process Is a Martingale | p. 20 |
Stochastic Delay Equations: Models | p. 23 |
Outline of the Chapter | p. 23 |
The System Model: Boundary Absorption | p. 25 |
Reflecting Diffusions | p. 30 |
The Reflected Diffusion | p. 30 |
Delayed Control, Reflection Term, and/or Wiener Process | p. 35 |
Neutral Equations | p. 37 |
Controlled Variance and Jumps | p. 37 |
The Girsanov Transformation | p. 38 |
Cost Functions | p. 40 |
Existence of an Optimal Control | p. 43 |
Reflecting or Absorbing Boundary: Discounted Cost | p. 43 |
Singular and Impulsive Controls | p. 50 |
Singular Controls | p. 50 |
Definition of and Existence of Solutions | p. 51 |
Existence of an Optimal Control | p. 56 |
Impulsive Controls | p. 58 |
Approximations to the Dynamical Models | p. 61 |
Outline of the Chapter | p. 61 |
Approximations of the Dynamical Systems | p. 63 |
A Basic Approximation | p. 63 |
Approximations by Time-Varying Delays | p. 69 |
Discretized Delays | p. 69 |
Periodic Delays | p. 71 |
Randomly Varying Delays | p. 73 |
Periodic-Erlang Delays | p. 75 |
Convergence of Costs and Existence of Optimal Controls | p. 77 |
Differential Operator for the Periodic-Erlang Approximation | p. 78 |
Simulations Illustrating the Model Approximations | p. 78 |
Simulations Based on the Periodic Approximation | p. 78 |
Simulations Based on the Periodic-Erlang Approximation | p. 82 |
Approximations: Path and Control Delayed | p. 84 |
Singular Controls | p. 90 |
Rapidly Varying Delays | p. 92 |
The Ergodic Cost Problem | p. 97 |
Outline of the Chapter | p. 97 |
The Basic Model | p. 98 |
Relaxed Feedback Controls | p. 98 |
Density Properties and Preliminary Results | p. 101 |
The Doeblin Condition | p. 105 |
Approximations of the Models | p. 107 |
Approximations with Periodic Delays | p. 110 |
Limit and Approximation Results for Periodic Delays | p. 110 |
Smoothed Nearly Optimal Controls | p. 115 |
Delays in the Variance Term | p. 117 |
The Periodic-Erlang Approximation | p. 118 |
Markov Chain Approximations: Introduction | p. 125 |
Outline of the Chapter | p. 125 |
The System Model | p. 126 |
Approximating Chains and Local Consistency | p. 127 |
Continuous-Time Interpolations | p. 131 |
The Continuous-Time Interpolation | p. 131 |
A Markov Continuous-Time Interpolation | p. 133 |
The "Explicit" Approximation Procedure | p. 137 |
The "Implicit" Approximating Processes | p. 140 |
The General Implicit Approximation Method | p. 142 |
Continuous-Time Interpolations | p. 144 |
Representations of the Cost Function | p. 146 |
Asymptotic Equivalence of the Timescales | p. 147 |
Convergence | p. 149 |
Singular and Impulsive Controls | p. 149 |
Singular Controls | p. 149 |
Impulsive Control | p. 153 |
The Ergodic Cost Function | p. 153 |
Introduction | p. 153 |
The Markov Chain Approximation Method | p. 154 |
Markov Chain Approximations: Path and Control Delayed | p. 159 |
Outline of the Chapter | p. 159 |
The Model and Local Consistency | p. 161 |
The Models | p. 161 |
Delay in Path Only: Local Consistency and Interpolations | p. 162 |
Delay in the Path and Control | p. 167 |
Absorbing Boundaries and Other Cost Functions | p. 171 |
Approximations to the Memory Segments | p. 172 |
Computational Procedures | p. 175 |
Delay in Path Only: State Representations and the Bellman Equation | p. 175 |
Delay in the Both Path and Control | p. 178 |
A Comment on Higher-Dimensional Problems | p. 179 |
The Implicit Numerical Approximation: Path Delayed | p. 180 |
Local Consistency and the Memory Segment | p. 180 |
The Cost Function and Bellman Equation | p. 185 |
The Use of Averaging in Constructing the Path Memory Approximation | p. 186 |
Timescales | p. 187 |
Convergence Theorems | p. 188 |
The Implicit Approximation Procedure and the Random Delay Model | p. 189 |
Path and Control Delayed: Continued | p. 193 |
Outline of the Chapter | p. 193 |
Periodic Approximations to the Delay: Path Delayed | p. 194 |
A Periodic-Erlang Model | p. 196 |
The Number of Points in the State Space: Path Only Delayed | p. 200 |
The Implicit and Periodic-Erlang Approximation Methods: Reduced Memory | p. 200 |
Control and Path Delayed | p. 203 |
A Periodic Approximating Memory Segment | p. 204 |
A Periodic-Erlang Approximation | p. 207 |
Proofs of Convergence | p. 213 |
Proofs of Theorems from Chapter 7 | p. 213 |
Proof of Theorem 4.1 | p. 218 |
Singular Controls | p. 219 |
Neutral Equations | p. 220 |
The Ergodic Cost Problem | p. 222 |
A Wave Equation Approach | p. 227 |
Outline of the Chapter | p. 227 |
The Model and Assumptions | p. 228 |
A Key Representation of x (.) | p. 230 |
A Representation of the Solution | p. 230 |
Comments on the Dimension and the System State | p. 231 |
Proof of the Representation | p. 232 |
Extensions | p. 235 |
A Discrete-Time Approximation | p. 236 |
The Markov Chain Approximation | p. 240 |
Preliminaries and Boundaries | p. 242 |
Transition Probabilities and Local Consistency: An Implicit Approximation Procedure | p. 242 |
Dynamical Representations, the Cost Function and Bellman Equation | p. 248 |
Size of the State Space for the Approximating Chain | p. 250 |
Proof of Convergence: Preliminaries | p. 252 |
The Randomization Errors | p. 252 |
Continuous Time Interpolations | p. 255 |
Convergence of the Numerical Algorithm | p. 259 |
Alternatives: Periodic and Periodic-Erlang Approximations | p. 261 |
A Periodic Approximation | p. 261 |
The Effective Delay and Numerical Procedures | p. 265 |
Singular and Impulsive Controls | p. 265 |
References | p. 267 |
Index | p. 273 |
Symbol Index | p. 277 |
Table of Contents provided by Ingram. All Rights Reserved. |