International Journal of Software Engineering and Knowledge Engineering

Source code is essential for researchers to reproduce the methods and replicate the results of artificial intelligence (AI) papers. Some organizations and researchers manually collect AI papers with available source code to contribute to the AI community. However, manual collection is a labor-intensive and time-consuming task. To address this issue, we propose a method to automatically identify papers with available source code and extract their source code repository URLs. With this method, we find that 20.5% of regular papers of 10 top AI conferences published from 2010 to 2019 are identified as papers with available source code and that 8.1% of these source code repositories are no longer accessible. We also create the XMU NLP Lab README Dataset, the largest dataset of labeled README files for source code document research. Through this dataset, we have discovered that quite a few README files have no installation instructions or usage tutorials provided. Further, a large-scale comprehensive statistical analysis is made for a general picture of the source code of AI conference papers. The proposed solution can also go beyond AI conference papers to analyze other scientific papers from both journals and conferences to shed light on more domains.

Model checking is well known to be a computer-aided method for verifying concurrent systems. Temporal logics and their Kripke-style semantics have been widely used in model checking. Falsification-aware Kripke-style semantics for temporal logics have been required for the theoretical basis of model checking because falsification plays a critical role in obtaining counterexample traces for the underlying object specifications in model checking. However, a useful falsification-aware Kripke-style semantics has yet to be developed for standard temporal logics. Hence, this study introduces two types of falsification-aware Kripke-style semantics for standard temporal logics that have been typically used in model checking. The equivalences among the proposed falsification-aware and standard Kripke-style semantics for standard temporal logics are proved. Furthermore, some inconsistency-tolerant subsystems of standard temporal logics are semantically obtained from the proposed falsification-aware Kripke-style semantics for standard temporal logics by deleting a characteristic condition on the labeling function of the semantics. The proposed semantic framework for standard and inconsistency-tolerant temporal logics is regarded as a unified framework for generalizing and combining the existing standard, inconsistency-tolerant, and many-valued semantic frameworks. This unified semantic framework is useful for obtaining a theoretical basis for generalized (inconsistency-tolerant) model checking, referred to here as falsification-aware model checking.

We propose an Ontology-Based Information Extraction (OBIE) system to automate the extraction of the criteria and values applied in Land-Use Suitability Analysis (LUSA) from bylaw and regulation documents related to the geographic area of interest. The results obtained by our proposed LUSA OBIE system (land-use suitability criteria and their values) are presented as an ontology populated with instances of the extracted criteria and property values. This latter output ontology is incorporated into a Multi-Criteria Decision-Making (MCDM) model applied for constructing suitability maps for different kinds of land uses. The resulting maps may be the final desired product or can be incorporated into the cellular automata urban modeling and simulation for predicting future urban growth. A case study has been conducted where the output from LUSA OBIE is applied to help produce a suitability map for the City of Regina, Saskatchewan, to assist in the identification of suitable areas for residential development. A set of Saskatchewan bylaw and regulation documents were downloaded and input to the LUSA OBIE system. We accessed the extracted information using both the populated LUSA ontology and the set of annotated documents. In this regard, the LUSA OBIE system was effective in producing a final suitability map.

Automatic code generation plays an important role in ensuring the reliability and correctness of software programs. Reliable programs can be obtained automatically from verified program specifications by code generators. The target languages of the existing code generators are mainly functional languages, which are relatively less used than C/C$++$. As C/C$++$ is widely used in the industry and many fundamental software facilities and the correctness verification of C/C$++$ programs is difficult and cumbersome, this paper provides an automatic conversion framework that allows to generate C$++$ implementation from verified Isabelle/HOL specifications. The framework is characterized by combining the verification convenience of Isabelle/HOL and the efficiency of C$++$. Since the correctness of the functional Isabelle/HOL specification can be guaranteed by interactive proofs, the correctness of the relevant generated C$++$ implementation can also be maintained.

Code clone refers to two or more identical or similar source code fragments. Research on code clone detection has lasted for decades. Investigation and evaluation of existing clone detection techniques indicate that they are resilient to function-level clone detection. Still, there may be room for further research in block-level clone detection. Particularly, type-3 clones that include large gaps, are ongoing challenges. To solve these problems, we propose a clone detection method based on multiple code features. It aims to improve the recall rate of code block clone detection and overcome large-gap and hard-to-detect type-3 clones. This method first splits the source code files based on the program’s structural features and context features to obtain code blocks. The collection of code blocks obtained in this way is complete, and the large gaps in clone pairs will also be removed. In addition, we only need to compute the similarity between code blocks with the same structural features, which can also significantly save time and resources. The similarity is obtained by calculating the proportion of the same tokens between two code blocks. Moreover, since different types of tokens have different weights in similarity calculation, we use supervised learning to obtain a classifier model between token features and code clone. We divide the tokens into 13 types and train the machine learning model with the manually confirmed clone or non-clone pair. Finally, we develop a prototype system and compare our tools with existing tools under the Mutation Framework and in several actual C projects. The experimental results also demonstrate the advancement and practicality of our prototype.

The development of a question answering (QA) system for application programming interface (API) documentation can greatly facilitate developers in API-related tasks. However, when applying deep learning technology, API QA systems suffer from the spurious solution problem. That is, the answer can literally appear in multiple positions (i.e. start-end indices) in the API documentation, though only one of them (called golden solution) correctly solves the question given its context. The other incorrect candidates (called spurious solutions) hinder the neural network model to learn reasonable solutions or correct answers. In this work, we propose Clean-and-Learn, an effective and robust method for API QA over documents. In order to reduce the spuriousness of candidate solutions used for training, we design several scoring functions to rank the candidate occurrences (clean). Only high-quality (top-$k$) candidate solutions are involved in training. Then, we perform multi-task learning by weighing the losses computed from the top-k occurrences (learn). We evaluate our method on the constructed APIQASet dataset. The experiment results show that Clean-and-Learn achieves a ROUGE-L score of 75.8 and accuracy of 70.5% in API QA, which significantly outperforms state-of-the-art approaches.