Modelling software reliability using Hybrid Bayesian networks

Abstract

In this research, we analyse the problem of predicting software reliability from AI perspective. We observe that existing models are built based on expert knowledge including de ning a set of metrics through surveys and causal relationships. We overcome their limitations by introducing new data collection, model construction and inference methodology. We propose a Hybrid Bayesian network that would estimate reliability of consecutive releases of software projects before a release decision, in terms of their residual (post-release) defects. We form this hybrid model by incorporating quantitative factors of development and testing processes into qualitative factors of requirements speci cation and documentation process without the need for any transformation. As quantitative factors, we select popularly used product, in-process and people metrics as well as introduce new ones depending on the availability of local data in the organizations. We also identify qualitative factors representing requirements speci cation process via surveys with development teams. Dependencies between software metrics and defects are determined according to correlation and independence tests and graphical dependence analysis with chi-plots. We utilize a Monte Carlo technique to approximate joint probability distribution of the model over conditionals by inferring unknown distribution parameters. Empirical analyses on two industrial datasets show that (i) Hybrid Bayesian networks are capable of estimating reliability in terms of residual defects, (ii) proposed way of de ning causal relationships, chi-plots, decreases error rates signi cantly, (iii) expert judgement-based models may not achieve as good prediction performances as statistical models, (iv) local data are so valuable and representative as expert knowledge in software organizations that they should be used primarily and strengthened with expert knowledge in predicting software reliability.