International Journal of Software Engineering and Knowledge Engineering

The process of software testing is a crucial phase in determining the quality of software, and this phase requires significant costs and a considerable amount of time for testers. This paper discusses the development of a framework for software quality assessment, involving flexible choices of software testing methods and variables in the form of an application. The method used is experimental, developing a new framework based on previous research, where previous research was limited to specific methods and testing variables. The result of this research is the creation of a new framework for software quality assessment. It is hoped that this framework can serve as a reference for software companies in evaluating software quality. In terms of complexity, this framework has the advantage of allowing a tester to choose methods with more flexible or unlimited testing variables. Regarding the estimated time and costs, with $\text{PF}=4,5\text{ and }10$, the practical application complexity of the developed framework is estimated to have the best costs, time and human resources at IDR 254,240,000, with an estimated time of 3,178 work hours and 6,356 work hours with a team of 3 people.

Event logs often record the execution of business process instances. Detecting traces in the event logs that do not comply with access control policies, such as role-based access control (RBAC) policies, is essential to ensuring system security. Moreover, process mining has been extensively utilized for security analysis in recent years. However, pattern-based approaches for designing and analyzing RBAC policies in the context of business processes through process mining are notably absent. In this paper, we present a systematic framework for checking the conformance of RBAC implemented in the event logs of business processes with the RBAC policies specified in domain knowledge. To facilitate the representation of the RBAC policies derived from the domain knowledge, we employ an RBAC domain-specific language (DSL) combined with our RBAC-driven object constraint language (OCL) invariant patterns built from the various types of RBAC constraints. The implemented RBAC in an event log is represented as snapshots within our framework. Then, we validate the snapshots with the RBAC policies to be able to detect RBAC conformance issues. The proposed framework is experimented with and evaluated on two business process logs, one simulated log and one real-world event log named “BPI Challenge 2017”.

Fog computing can provide an effective solution to the challenges presented by today’s ever-emerging Internet of Things (IoT) infrastructures. As the number of interconnected devices progressively increases, these infrastructures require better solutions to ensure high scalability and processing capacity, along with an efficient use of available resources. This is why this paper presents a distributed Fog architecture, specifically designed to address the challenges and difficulties presented by heterogeneous IoT environments. This Fog architecture is used as an intermediate layer between the IoT devices and the final layer, it has been designed after the previous analysis of the requirements to be met for the solution, then the modularization of the architecture has been carried out so that it can be easily distributed, and finally, an implementation has been generated on a real environment as a validation case of the proposal.

We are living in an era of knowledge explosion, where all kinds of knowledge are emerging and becoming more and more complicated with the development of new techniques and new ideas. When we study knowledge and apply them to understand and solve problems, the credibility of knowledge is becoming our main concerns. Usually, high credible domain knowledge can guide us correctly understand all concepts and the relationships between them in this domain. Due to its good layer structure and scalability, domain knowledge network is widely used to represent knowledge in knowledge engineering, artificial intelligence and others in recent years. How to ensure the credibility of domain knowledge network? This is an important and interesting topic.

In this paper, we propose a method to evaluate the knowledge credibility for domain knowledge network, which means that we can start from the layer structure of domain knowledge network, and evaluate the credibility of knowledge layer by layer using validated expert knowledge such as domain dictionary, domain ontology and domain expert experience.

We conduct experiments with six domain knowledge network constructed based on network data and six domain knowledge network constructed manually based on published books or domain dictionaries, which describe the same domain knowledge in pairs. Experimental results show that the knowledge credibility of domain knowledge network constructed from validated expert knowledge is significantly higher than the knowledge credibility of domain knowledge network constructed directly from network data, which satisfy our expectation and also prove the effectiveness of our credibility evaluation method.

The paper addresses the limitations of traditional evaluation metrics for Question Answering (QA) systems that primarily focus on syntax and n-gram similarity. We propose a novel model-based evaluation metric, MQA-metric, and create a human-judgment-based dataset, squad-qametric and marco-qametric, to validate our approach. The research aims to solve several key problems: the objectivity in dataset labeling, the effectiveness of metrics when there is no syntax similarity, the impact of answer length on metric performance, and the influence of real answer quality on metric results. To tackle these challenges, we designed an interface for dataset labeling and conducted extensive experiments with human reviewers. Our analysis shows that the MQA-metric outperforms traditional metrics like BLEU, ROUGE and METEOR. Unlike existing metrics, MQA-metric leverages semantic comprehension through large language models (LLMs), enabling it to capture contextual nuances and synonymous expressions more effectively. This approach sets a standard for evaluating QA systems by prioritizing semantic accuracy over surface-level similarities. The proposed metric correlates better with human judgment, making it a more reliable tool for evaluating QA systems. Our contributions include the development of a robust evaluation workflow, creation of high-quality datasets, and an extensive comparison with existing evaluation methods. The results indicate that our model-based approach provides a significant improvement in assessing the quality of QA systems, which is crucial for their practical application and trustworthiness.

Change impact analysis (CIA) is an essential method in software maintenance and evolution. Its accuracy and usability play a crucial role in its application. However, most CIAs are coarse-grained and limited to class and method levels. Despite the fine-grained CIAs’ success in giving the statement-level impact set, they are still limited without the sub-statement level dependency analysis, leading to low precision. Additionally, their unstructured impact sets make it challenging for users to comprehend the impact content. This paper proposes Hierarchical Change Impact Analysis (HCIA), a Hierarchical CIA technique based on the sub-statement level dependence graph. HCIA can perform a forward hierarchy program slicing on the change set from five levels: sub-statement, statement, method, class, and package. Based on the program slices, HCIA calculates the impact factor of the impact sets at the five levels to generate the final impact set. In the experiment, we evaluate the relationship between the impact factor and the actual affected codes and assess the most appropriate size of HCIA impact sets. Furthermore, we evaluate HCIA on 10 open-source projects by comparing our approach with popular CIAs at the five levels. The experimental result shows that HCIA is more accurate than the popular CIAs.