Probability and Stochastic Processes

List of Figures xvii

List of Tables xxi

Preface i

Acknowledgments iii

Introduction 1

PART I PROBABILITY

1 Elements of Probability Measure 3

1.1 Probability Spaces 4

1.2 Conditional Probability 16

1.3 Independence 23

1.4 Monotone Convergence properties of probability 25

1.5 Lebesgue measure on the unit interval (0,1] 31

Problems 34

2 Random Variables 39

2.1 Discrete and Continuous Random Variables 42

2.2 Examples of commonly encountered Random Variables 46

2.3 Existence of random variables with prescribed distribution. Skorohod representation of a random variable 59

2.4 Independence 62

2.5 Functions of random variables. Calculating distributions 66

Problems 76

3 Applied chapter: Generating Random Variables 81

3.1 Generating one dimensional random variables by inverting the CDF 82

3.2 Generating one dimensional normal random variables 85

3.3 Generating random variables. Rejection sampling method 88

3.4 Generating random variables. Importance sampling 104

Problems 113

4 Integration Theory 117

4.1 Integral of measurable functions 118

4.2 Expectations 124

4.3 Moments of a random variable. Variance and the correlation coefficient. 137

4.4 Functions of random variables. The Transport Formula. 139

4.5 Applications. Exercises in probability reasoning. 142

4.6 A Basic Central Limit Theorem: The DeMoivre-Laplace Theorem: 144

Problems 146

5 Conditional Distribution and Conditional Expectation 149

5.1 Product Spaces 150

5.2 Conditional distribution and expectation. Calculation in simple cases 154

5.3 Conditional expectation. General definition 157

5.4 Random Vectors. Moments and distributions 160

Problems 169

6 Moment Generating Function. Characteristic Function. 173

6.1 Sums of Random Variables. Convolutions 173

6.2 Generating Functions and Applications 174

6.3 Moment generating function 180

6.4 Characteristic function 184

6.5 Inversion and Continuity Theorems 191

6.6 Stable Distributions. Levy Distribution 196

Problems 200

7 Limit Theorems 205

7.1 Types of Convergence 205

7.2 Relationships between types of convergence 213

7.3 Continuous mapping theorem. Joint convergence. Slutsky's theorem 222

7.4 The two big limit theorem: LLN and CLT 224

7.5 Extensions of CLT 237

7.6 Exchanging the order of limits and expectations 243

Problems 244

8 Statistical Inference 251

8.1 The classical problems in statistics 251

8.2 Parameter Estimation problem 252

8.3 Maximum Likelihood Estimation Method 257

8.4 The Method of Moments 268

8.5 Testing, The likelihood ratio test 269

8.6 Confidence Sets 276

Problems 278

PART II STOCHASTIC PROCESSES

9 Introduction to Stochastic Processes 285

9.1 General characteristics of Stochastic processes 286

9.2 A Simple process? The Bernoulli process 293

Problems 296

10 The Poisson process 299

10.1 Definitions. 299

10.2 Interarrival and waiting time for a Poisson process 302

10.3 General Poisson Processes 309

10.4 Simulation techniques. Constructing Poisson Processes 315

Problems 318

11 Renewal Processes 323

11.1 Limit Theorems for the renewal process 326

11.2 Discrete Renewal Theory. 335

11.3 The Key Renewal Theorem 340

11.4 Applications of the Renewal Theorems 342

11.5 Special cases of renewal processes 344

11.6 The renewal Equation 350

11.7 Age dependent Branching processes 354

Problems 357

12 Markov Chains 361

12.1 Basic concepts for Markov Chains 361

12.2 Simple Random Walk on integers in d-dimensions 373

12.3 Limit Theorems 376

12.4 States in a MC. Stationary Distribution 377

12.5 Other issues: Graphs, first step analysis 384

12.6 A general treatment of the Markov Chains 385

Problems 395

13 Semi-Markov and Continuous time Markov Processes 401

13.1 Characterization Theorems for the general semi Markov process 403

13.2 Continuous time Markov Processes 407

13.3 The Kolmogorov Differential Equations 410

13.4 Calculating transition probabilities for a Markov process. General Approach 415

13.5 Limiting Probabilities for the Continuous time Markov Chain 416

13.6 Reversible Markov process 419

Problems 422

14 Martingales 427

14.1 Definition and examples 428

14.2 Martingales and Markov chains 430

14.3 Previsible process. The Martingale transform 432

14.4 Stopping time. Stopped process 434

14.5 Classical examples of Martingale reasoning 439

14.6 Convergence theorems. L1 convergence. Bounded martingales in L2 446

Problems 448

15 Brownian Motion 455

15.1 History 455

15.2 Definition 457

15.3 Properties of Brownian motion 461

15.4 Simulating Brownian motions 470

Problems 471

16 Stochastic Differential Equations 475

16.1 The construction of the stochastic integral 477

16.2 Properties of the stochastic integral 484

16.3 Ito lemma 485

16.4 Stochastic Differential equations. SDE's 489

16.5 Examples of SDE's 492

16.6 Linear systems of SDE's 503

16.7 A simple relationship between SDE's and PDE's 505

16.8 Monte Carlo Simulations of SDE's 507

Problems 512

A Appendix: Linear Algebra and solving difference equations and systems of differential equations 517

A.1 Solving difference equations with constant coefficients 518

A.2 Generalized matrix inverse and pseudodeterminant 519

A.3 Connection between systems of differential equations and matrices520

A.4 Linear Algebra results 523

A.5 Finding fundamental solution of the homogeneous system 526

A.6 The nonhomogeneous system 528

A.7 Solving systems when P is nonconstant 530

Index 533