Probability, Statistics, and Stochastic Processes

Preface xi

Preface to the First Edition xiii

1 Basic Probability Theory 1

1.1 Introduction 1

1.2 Sample Spaces and Events 3

1.3 The Axioms of Probability 7

1.4 Finite Sample Spaces and Combinatorics 15

1.4.1 Combinatorics 17

1.5 Conditional Probability and Independence 27

1.6 The Law of Total Probability and Bayes' Formula 41

Problems 63

2 Random Variables 76

2.1 Introduction 76

2.2 Discrete Random Variables 77

2.3 Continuous Random Variables 82

2.4 Expected Value and Variance 95

2.5 Special Discrete Distributions 111

2.6 The Exponential Distribution 123

2.7 The Normal Distribution 127

2.8 Other Distributions 131

2.9 Location Parameters 137

2.10 The Failure Rate Function 139

Problems 144

3 Joint Distributions 156

3.1 Introduction 156

3.2 The Joint Distribution Function 156

3.3 Discrete Random Vectors 158

3.4 Jointly Continuous Random Vectors 160

3.5 Conditional Distributions and Independence 164

3.5.1 Independent Random Variables 168

3.6 Functions of Random Vectors 172

3.7 Conditional Expectation 185

3.8 Covariance and Correlation 196

3.9 The Bivariate Normal Distribution 209

3.10 Multidimensional Random Vectors 216

3.11 Generating Functions 231

3.12 The Poisson Process 240

Problems 247

4 Limit Theorems 263

4.1 Introduction 263

4.2 The Law of Large Numbers 264

4.3 The Central Limit Theorem 268

4.4 Convergence in Distribution 275

Problems 278

5 Simulation 281

5.1 Introduction 281

5.2 Random Number Generation 282

5.3 Simulation of Discrete Distributions 283

5.4 Simulation of Continuous Distributions 285

5.5 Miscellaneous 290

Problems 292

6 Statistical Inference 294

6.1 Introduction 294

6.2 Point Estimators 294

6.3 Confidence Intervals 304

6.4 Estimation Methods 312

6.5 Hypothesis Testing 327

6.6 Further Topics in Hypothesis Testing 334

6.7 Goodness of Fit 339

6.8 Bayesian Statistics 351

6.9 Nonparametric Methods 363

Problems 378

7 Linear Models 391

7.1 Introduction 391

7.2 Sampling Distributions 392

7.3 Single Sample Inference 395

7.4 Comparing Two Samples 402

7.5 Analysis of Variance 409

7.6 Linear Regression 415

7.7 The General Linear Model 431

Problems 436

8 Stochastic Processes 444

8.1 Introduction 444

8.2 Discrete -Time Markov Chains 445

8.3 Random Walks and Branching Processes 464

8.4 Continuous -Time Markov Chains 475

8.5 Martingales 494

8.6 Renewal Processes 502

8.7 Brownian Motion 509

Problems 517

Appendix A Tables 527

Appendix B Answers to Selected Problems 535

Further Reading 551

Index 553