International Journal of Software Engineering and Knowledge Engineering

Requirement engineering is a key issue in the development of a software project. Like any other development activity it is not without risks. This work is about the empirical study of risks of requirements by applying machine learning techniques, specifically Bayesian networks classifiers. We have defined several models to predict the risk level for a given requirement using three dataset that collect metrics taken from the requirement specifications of different projects. The classification accuracy of the Bayesian models obtained is evaluated and compared using several classification performance measures. The results of the experiments show that the Bayesians networks allow obtaining valid predictors. Specifically, a tree augmented network structure shows a competitive experimental performance in all datasets. Besides, the relations established between the variables collected to determine the level of risk in a requirement, match with those set by requirement engineers. We show that Bayesian networks are valid tools for the automation of risks assessment in requirement engineering.

We propose a test case prioritization strategy for risk based testing, in which the risk exposure is employed as the key criterion of evaluation. Existing approaches to risk based testing typically employ risk exposure values as assessed by the tester. In contrast, we employ exposure values that have been determined by experts during the risk assessment stage of the risk management process. If a given method produces greater accuracy in fault detection, that approach is considered more valuable for software testing. We demonstrate the value of our proposed risk based testing method in this sense through its application.

Software systems in safety-critical industrial automation systems, such as power plants and steel mills, become increasingly large, complex, and distributed. For assessing risks, like low product quality and project cost and duration overruns, to trustworthy services provided by software as part of automation systems there are established risk analysis approaches based on data collection from project participants and data models. However, in multi-disciplinary engineering projects there are often semantic gaps between the software tools and data models of the participating engineering disciplines, e.g., mechanic, electrical, and software engineering.

In this paper we discuss current limitations to risk assessment in (software+) engineering projects and introduce the SEMRISK approach for risk assessment in projects with semantically heterogeneous software tools and data models. The SEMRISK approach provides the knowledge engineering foundation to allow an end-to-end view for service-relevant data elements such as signals, by providing a project domain ontology and mappings to the tool data models of the involved engineering disciplines.

We empirically evaluate the effectiveness and efficiency of the approach based on a real-world industrial use case from the safety-critical power plant domain. Major results are that the approach was effective and considerably more efficient than the current approach at the industry partner.

Real-world software systems are becoming larger, more complex, and much more unpredictable. Software systems face many risks in their life cycles. Software practitioners strive to improve software quality by constructing defect prediction models using metric (feature) selection techniques. Finding faulty components in a software system can lead to a more reliable final system and reduce development and maintenance costs. This paper presents an empirical study of six commonly used filter-based software metric rankers and our proposed ensemble technique using rank ordering of the features (mean or median), applied to three large software projects using five commonly used learners. The classification accuracy was evaluated in terms of the AUC (Area Under the ROC (Receiver Operating Characteristic) Curve) performance metric. Results demonstrate that the ensemble technique performed better overall than any individual ranker and also possessed better robustness. The empirical study also shows that variations among rankers, learners and software projects significantly impacted the classification outcomes, and that the ensemble method can smooth out performance.

Software risk comes mainly from its poor reliability, but how to effectively achieve high reliability still remains a challenge. This paper puts forward a framework for systematically integrating formal specification, review, and testing, and shows how it can be applied to effectively eliminate errors in the major phases of software development process to enhance software reliability. In this framework, requirements errors can be removed and missing requirements can be identified by formalizing requirements into formal specifications whose validity can be ensured by rigorous review. The valid specification can then be used as a firm foundation for implementation and for rigorous inspection, testing, and walkthrough of the implemented program. We discuss how formalization, review, and testing work together at different levels of software development to improve software reliability through detecting and removing errors in documentation.

Metamorphic testing is a technique for the verification of software output without a complete testing oracle. Mathematical optimization, implemented in software, is a problem for which verification can often be challenging. In this paper, we apply metamorphic testing to one such optimization problem, the quadratic assignment problem (QAP). From simple observations of the properties of the QAP, we describe how to derive a number of metamorphic relations useful for verifying the correctness of a QAP solver. We then compare the effectiveness of these metamorphic relations, in "killing" mutant versions of an exact QAP solver, to a simulated oracle. We show that metamorphic testing can be as effective as the simulated oracle for killing mutants. We examine the relative effectiveness of different metamorphic relations, both singly and in combination, and conclude that combining metamorphic relations can be significantly more effective than using a single relation.