Statistical Methods in Software Engineering

1 Introduction and Overview.- 1.1 What is Software Engineering?.- 1.2 Uncertainty in Software Production.- 1.2.1 The Software Development Process.- 1.2.2 Sources of Uncertainty in the Development Process.- 1.3 The Quantification of Uncertainty.- 1.3.1 Probability as an Approach for Quantifying Uncertainty.- 1.3.2 Interpretations of Probability.- 1.3.3 Interpreting Probabilities in Software Engineering.- 1.4 The Role of Statistical Methods in Software Engineering.- 1.5 Chapter Summary.- 2 Foundational Issues: Probability and Reliability.- 2.0 Preamble.- 2.1 The Calculus of Probability.- 2.1.1 Notation and Preliminaries.- 2.1.2 Conditional Probabilities and Conditional Independence.- 2.1.3 The Calculus of Probability.- 2.1.4 The Law of Total Probability, Bayes' Law, and the Likelihood Function.- 2.1.5 The Notion of Exchangeability.- 2.2 Probability Models and Their Parameters.- 2.2.1 What is a Software Reliability Model?.- 2.2.2 Some Commonly Used Probability Models.- 2.2.3 Moments of Probability Distributions and Expectation of Random Variables.- 2.2.4 Moments of Probability Models: The Mean Time to Failure.- 2.3 Point Processes and Counting Process Models.- 2.3.1 The Nonhomogeneous Poisson Process Model.- 2.3.2 The Homogeneous Poisson Process Model.- 2.3.3 Generalizations of the Point Process Model.- 2.4 Fundamentals of Reliability.- 2.4.1 The Notion of a Failure Rate Function.- 2.4.2 Some Commonly Used Model Failure Rates.- 2.4.3 Covariates in the Failure Rate Function.- 2.4.4 The Concatenated Failure Rate Function.- 2.5 Chapter Summary.- Exercises for Chapter 2.- 3 Models for Measuring Software Reliability.- 3.1 Background: The Failure of Software.- 3.1.1 The Software Failure Process and Its Associated Randomness.- 3.1.2 Classification of Software Reliability Models.- 3.2 Models Based on the Concatenated Failure Rate Function.- 3.2.1 The Failure Rate of Software.- 3.2.2 The Model of Jelinski and Moranda (1972).- 3.2.3 Extensions and Generalizations of the Model by Jelinski and Moranda.- 3.2.4 Hierarchical Bayesian Reliability Growth Models.- 3.3 Models Based on Failure Counts.- 3.3.1 Time Dependent Error Detection Models.- 3.4 Models Based on Times Between Failures.- 3.4.1 The Random Coefficient Autoregressive Process Model.- 3.4.2 A Non-Gaussian Kalman Filter Model.- 3.5 Unification of Software Reliability Models.- 3.5.1 Unification via the Bayesian Paradigm.- 3.5.2 Unification via Self-Exciting Point Process Models.- 3.5.3 Other Approaches to Unification.- 3.6 An Adaptive Concatenated Failure Rate Model.- 3.6.1 The Model and Its Motivation.- 3.6.2 Properties of the Model and Interpretation of Model Parameters.- 3.7 Chapter Summary.- Exercises for Chapter 3.- 4 Statistical Analysis of Software Failure Data.- 4.1 Background: The Role of Failure Data.- 4.2 Bayesian Inference, Predictive Distributions, and Maximization of Likelihood.- 4.2.1 Bayesian Inference and Prediction.- 4.2.2 The Method of Maximum Likelihood.- 4.2.3 Application: Inference and Prediction Using Jelinski and Moranda's Model.- 4.2.4 Application: Inference and Prediction Under an Error Detection Model.- 4.3 Specification of Prior Distributions.- 4.3.1 Standard of Reference-Noninformative Priors.- 4.3.2 Subjective Priors Based on Elicitation of Specialist Knowledge.- 4.3.3 Extensions of the Elicitation Model.- 4.3.4 Example: Eliciting Priors for the Logarithmic-Poisson Model.- 4.3.5 Application: Failure Prediction Using Logarithmic-Poisson Model.- 4.4 Inference and Prediction Using a Hierarchical Model.- 4.4.1 Application to NTDS Data: Assessing Reliability Growth.- 4.5 Inference and Predictions Using Dynamic Models.- 4.5.1 Inference for the Random Coefficient Exchangeable Model.- 4.5.2 Inference for the Adaptive Kalman Filter Model.- 4.5.3 Inference for the Non-Gaussian Kalman Filter Model.- 4.6 Prequential Prediction, Bayes Factors, and Model Comparison.- 4.6.1 Prequential Likelihoods and Prequential Prediction.- 4.6.2 Bayes' Factors and Model Averaging.- 4.6.3 Model Complexity Occam's Razor.- 4.6.4 Application: Comparing the Exchangeable, Adaptive, and Non-Gaussian Models.- 4.6.5 An Example of Reversals in the Prequential Likelihood Ratio.- 4.7 Inference for the Concatenated Failure Rate Model.- 4.7.1 Specification of the Prior Distribution.- 4.7.2 Calculating Posteriors by Markov Chain Monte Carlo.- 4.7.3 Testing Hypotheses About Reliability Growth or Decay.- 4.7.4 Application to System 40 Data.- 4.8 Chapter Summary.- Exercises for Chapter 4.- 5 Software Productivity and Process Management.- 5.1 Background: Producing Quality Software.- 5.2 A Growth-Curve Model for Estimating Software Productivity.- 5.2.1 The Statistical Model.- 5.2.2 Inference and Prediction Under the Growth-Curve Model.- 5.2.3 Application: Estimating Individual Software Productivity.- 5.3 The Capability Maturity Model for Process Management.- 5.3.1 The Conceptual Framework.- 5.3.2 The Probabilistic Approach for Hierarchical Classification.- 5.3.3 Application: Classifying a Software Developer.- 5.4 Chapter Summary.- Exercises for Chapter 5.- 6 The Optimal Testing and Release of Software.- 6.1 Background: Decision Making and the Calculus of Probability.- 6.2 Decision Making Under Uncertainty.- 6.3 Utility and Choosing the Optimal Decision.- 6.3.1 Maximization of Expected Utility.- 6.3.2 The Utility of Money.- 6.4 Decision Trees.- 6.4.1 Solving Decision Trees.- 6.5 Software Testing Plans.- 6.6 Examples of Optimal Testing Plans.- 6.6.1 One-Stage Testing Using the Jelinski-Moranda Model.- 6.6.2 One-and Two-Stage Testing Using the Model by Goel and Okumoto.- 6.6.3 One-Stage Lookahead Testing Using the Model by Goel and Okumoto.- 6.6.4 Fixed-Time Lookahead Testing for the Goel-Okumoto Model.- 6.6.5 One-Bug Lookahead Testing Plans.- 6.6.6 Optimality of One-Stage Look Ahead Plans.- 6.7 Application: Testing the NTDS Data.- 6.8 Chapter Summary.- Exercises for Chapter 6.- 7 Other Developments: Open Problems.- 7.0 Preamble.- 7.1 Dynamic Modeling and the Operational Profile.- 7.1.1 Martingales, Predictable Processes, and Compensators: An Overview.- 7.1.2 The Doob-Meyer Decomposition of Counting Processes.- 7.1.3 Incorporating the Operational Profile.- 7.2 Statistical Aspects of Software Testing: Experimental Designs.- 7.2.1 Inferential Issues in Random and Partition Testing.- 7.2.2 Comparison of Random and Partition Testing.- 7.2.3 Design of Experiments in Software Testing.- 7.2.4 Design of Experiments in Multiversion Programming.- 7.2.5 Concluding Remarks.- 7.3 The Integration of Module and System Performance.- 7.3.1 The Protocols of Control Flow and Data Flow.- 7.3.2 The Structure Function of Modularized Software.- Appendices.- Appendix A Statistical Computations Using the Gibbs Sampler.- A.1 An Overview of the Gibbs Sampler.- A.2 Generating Random Variates The Rejection Method.- A.3 Examples: Using the Gibbs Sampler.- A.3.1 Gibbs Sampling the Jelinski-Moranda Model.- A.3.2 Gibbs Sampling the Hierarchical Model.- A.3.3 Gibbs Sampling the Adaptive Kalman Filter Model.- A.3.4 Gibbs Sampling the Non-Gaussian Kalman Filter Model.- Appendix B The Maturity Questionnaire and Responses.- B. 1 The Maturity Questionnaire.- B.2 Binary (Yes, No) Responses to the Maturity Questionnaire.- B.3 Prior Probabilities and Likelihoods.- References.- Author Index.