Narrow Results

We describe action machines, a framework for encoding and composing partial behavioral descriptions. Action machines encode behavior as a variation of labeled transition systems where the labels are observable activities of the described artifact and the states capture full data models. Labels may also have structure, and both labels and states may be partial with a symbolic representation of the unknown parts. Action machines may stem from software models or programs, and can be composed in a variety of ways to synthesize new behaviors. The composition operators described here include synchronized and interleaving parallel composition, sequential composition, and alternating simulation. We use action machines in analysis processes such as model checking and model-based testing. The current main application is in the area of model-based conformance testing, where our approach addresses practical problems users at Microsoft have in applying model-based testing technology.

A predicate sequencing constraint logic (PSCL) is proposed to represent test purpose for concurrent program testing. The logic is capable of expressing not only sequencing relationships among input and output events, but also data dependencies between event parameters. A PSCL-based symbolic test generation method is developed to automatically derive symbolic test cases that incorporate given data dependency constraints as verdict conditions. The method works in a syntactic way without referring to concrete program states and the derived test cases allow dynamic test data selection according to the response from the software under test. The advantage of the approach is demonstrated with a case study.

A method of functional testing for software components according to model-based testing techniques is proposed. Test cases are generated from UML diagrams and OCL constraints that comprise a component interface and realization specification. The method uses a reduced set of UML artifacts that constitute the main requirements for its application along with a component development process, making use of development artifacts. Also, the set of generated test artifacts are packed together with the provided components to reduce the overall testing effort when clients assemble applications. A tool has been developed to automate the method with test cases generated as Java test components. Test execution and result analysis is also supported. For each component, the tool generates a test component that can be easily upgraded and configured for testing the services provided by a component throughout its life cycle.

User inputs are critical for the security, safety, and reliability of software systems. This paper proposes a new concept called user input contracts, which is an integral part of a design-by-contract supplemented development process, and a model-based testing approach to detect violations of user input contracts. The approach generates test cases from an input contract integrated with graph-based model of user interface specification and applies them to the system under consideration. The paper presents a proof-of-concept tool that has been developed and used to validate the approach by experiments. The experiments are conducted on a web-based system for marketing tourist services to analyze input robustness of system under consideration with respect to user input contracts.

Model-based testing (MBT) seems to be gaining interest in industry and academia due to its provision of systematic, automated and comprehensive testing. The challenge in MBT is to generate optimal test data to execute test cases. Recently, researchers have successfully applied search-based techniques (SBTs) by automating the search for an optimal set of test data at reasonable cost compared to other more expensive techniques. In real complex systems, effectiveness and cost of SBTs for MBT in industrial context are little known. The objective of this study is to empirically evaluate the cost and the effectiveness of SBTs for MBT on industrial case studies. We applied a model-driven approach and SBTs to automatically generate executable feasible test cases. The results show that the model-driven approach generated high number of infeasible test cases with less time while genetic algorithm (GA) and simulating annealing (SA) outperformed significantly random search (RS) with high generation time. We concluded that local SBTs are more appropriate to generate test data when the type of the constraints is simple. Current work on analyzing the cost and effectiveness on SBTs for MBT indicates possible enhancement using the model-driven approach to detect the infeasible paths and SBTs to achieve optimal success rate.

Testing processes and workflows in information and Internet of Things systems is a major part of the typical software testing effort. Consistent and efficient path-based test cases are desired to support these tests. Because certain parts of software system workflows have a higher business priority than others, this fact has to be involved in the generation of test cases. In this paper, we propose a Prioritized Process Test (PPT), which is a model-based test case generation algorithm that represents an alternative to currently established algorithms that use directed graphs and test requirements to model the system under test. The PPT accepts a directed multigraph as a model to express priorities, and edge weights are used instead of test requirements. To determine the test-coverage level of test cases, a test-depth-level concept is used. We compared the presented PPT with five alternatives (i.e. the Process Cycle Test (PCT), a naive reduction of test set created by the PCT, Brute Force algorithm, Set-covering-Based Solution and Matching-based Prefix Graph Solution) for edge coverage and edge-pair coverage. To assess the optimality of the path-based test cases produced by these strategies, we used 14 metrics based on the properties of these test cases and 59 models that were created for three real-world systems. For all edge coverage, the PPT produced more optimal test cases than the alternatives in terms of the majority of the metrics. For edge-pair coverage, the PPT strategy yielded similar results to those of the alternatives. Thus, the PPT strategy is an applicable alternative as it reflects both the required test coverage level and the business priority in parallel.

This paper proposes a transformation-based method to automatically generate functional test cases from use cases named USLTG (Use case Specification Language (USL)-based Test Generation). We first focus on developing a modeling language named Test Case Specification Language (TCSL) in order to express test cases. Test cases in TCSL can contain detailed information including test steps, test objects within steps, actions of test objects, and test data. Such information is often ignored in currently available test case specifications. We then aim to generate test cases in a TCSL model by a transformation from use cases that are represented by a USL. The USLTG transformation includes three main steps in generating (1) scenarios, (2) test data, and (3) a TCSL model. Within our transformation, the OCL solver is employed in order to build system snapshots as the part of test cases and to identify other test data. We applied our method to two case studies and evaluated our method by comparing it with other recent works.

Finite State Machine is a popular modeling notation for various systems, especially software and electronic. Test paths (TPs) can be automatically generated from the system model to test such systems using a suitable algorithm. This paper presents a strategy that generates TPs and allows to start and end TPs only in defined states of the finite state machine. The strategy also simultaneously supports generating TPs only of length in a given range. For this purpose, alternative system models, test coverage criteria, and a set of algorithms are developed. The strategy is compared with the best alternative based on the reduction of the test set generated by the established N-switch coverage approach on a mix of 171 industrial and artificially generated problem instances. The proposed strategy outperforms the compared variant in a smaller number of TP steps. The extent varies with the used test coverage criterion and preferred TP length range from none to two and half fold difference. Moreover, the proposed technique detected up to 30% more simple artificial defects inserted into experimental SUT models per one test step than the compared alternative technique. The proposed strategy is well applicable in situations where a possible TP starts and ends in a state machine needs to be reflected and, concurrently, the length of the TPs has to be in a defined range.

Existing software product line (SPL) engineering testing approaches generally provide positive testing that validates the SPL’s functionality. Negative testing is commonly neglected. This research aims to unify behavioral and feature models of an SPL, enable testing before and after variability binding for domain-centric and product-centric testing, and combine positive and negative testing for a holistic testing view. This study suggests behavioral modeling with event sequence graphs (ESGs). This heterogeneous modeling strategy supports bottom-up domain testing and top-down product testing with the feature model. This new feature-oriented ESG test creation method generates shorter test sequences than the original ESG optimum test sequences. Statechart and original ESG test-generating methods are compared. Positive testing findings are similar. The Statechart technique generated 12 test cases with 59 events, whereas the ESG technique created six test cases with 60 events. The ESG technique generated 205 negative test cases with 858 events with the Test Suite Designer tool. However, the Conformiq Designer tool for the Statechart technique does not have a negative test case generation capability. It is shown that the proposed ESG-based holistic approach confirms not only the desirable (positive) properties but also the undesirable (negative) ones. As an additional research, the traditional ESG test-generating approach is compared to the new feature-oriented method on six SPLs of different sizes and features. Our case study results show that the traditional ESG test generation approach demonstrated higher positive test generation scores compare to the proposed feature-oriented test generation approach. However, our proposed feature-oriented test generation approach is capable of generating shorter test sequences, which could be beneficial for reducing the execution time of test cases compared to traditional ESG approach. Finally, our case study has also shown that regardless of the test generation approach, there has been found no significant difference between the Bottom-up and Top-down test strategies with respect to their positive test generation scores.