Research ArticleOpen Access

A Visual Analytics Approach Applying for Discovering Knowledge from Multivariate Datasets of Stakeholders Feedback in the University

Dang Van Pham

Faculty of Information Technology, Nguyen Tat Thanh University, Ho Chi Minh City, Vietnam

Graduate University of Science and Technology, Vietnam Academy of Science and Technology, Hanoi, Vietnam

Institute of Applied Mechanics and Informatics, Ho Chi Minh City, Vietnam

E-mail Address: pvdang@ntt.edu.vn

E-mail Address: pvdang.tps@gmail.com

Search for more papers by this author

and

Bao Khang Nguyen

British International School, Ho Chi Minh City, Vietnam

E-mail Address: knguyen22@bisvietnam.com

E-mail Address: nguyenbaokhang.kevin@gmail.com

Search for more papers by this author

https://doi.org/10.1142/S2196888823500082Cited by:0 (Source: Crossref)

Abstract

In accrediting training quality, assessing the educational quality of a university is an extremely important task to affirm the prestige of the university to the society. Areas of assessment include training program, teaching and learning activities, capacity of graduate student, and the reputation of the university in society. This research studies the characteristics of the survey dataset structures to form data variables and analyzes the relations between data variables to build non-spatial and temporal multidimensional model (nSTM), applies the establishment of visual graphs according to the data variables representing the datasets with vertical bars indicating the values of the data variables being visually represented. This research conducts experiments on the dataset structures surveying the areas of a university. With this scientific approach, it will help university administrators to build policies to improve in all aspects and especially improve the quality of university education.

Keywords:

1. Introduction

Educational management at universities is increasingly oriented towards being both modern and adaptive to the practical needs of society. Many universities have been conducting verification at many levels with different scales, domestically and internationally. Especially, the quality verification activities help the university leaders to be aware of the real values that the university owns and is more aware of the university’s position in the face of modern requirements of international integration and practical requirements of society as a basis for forming necessary policies to raise the university’s level in competitive environments.

Collecting opinions from the society about the university’s training products is one of the activities of the quality verification process. With the opinion of employers, they evaluate the university’s created products as generations of university graduates who have contributed to the society. With the opinion of the university graduates, they will evaluate the adaptability of the life baggage provided by the university’s faculty during their time at the university. With the instructor’s opinions, they will assess the opinions of their satisfying ability towards teaching job and the appropriateness of university’s policies. In reality, there are many cases that university leaders have not fully analyzed and applied the meaning of data to create appropriate policies.

Approaching data visualization and visual analytics can make it easy for university administrators to understand the mean of the data which they have with their experiences, management knowledges, and own requirements. Approaching data visualization is a mapping that transforms data into information and knowledge through visual perception by the human vision organs. In other words, data visualization transforms data from different forms and structures into images so that people can perceive information and knowledge by the human vision organs and by the viewing–thinking method. Regarding the visual analytics approach, this method uses an interactive visual interface to assist users in extracting information and discovering knowledge from multivariate datasets.¹ It can be said that data analysis cannot be performed without the question being asked. The question is does the initial step and the basis for the impulsion of data analysis process need to be conducted?^2,3 Therefore, data analysis is the process of perception by the human vision organs that serves as the key to unlocking the deep meaning of things and phenomena by asking people questions and from there through the model of visual viewing–thinking will upgrade people’s knowledge so that they can answer the questions posed by themselves. With this scientific approach, it will help humans, especially the leadership levels of a university, have an insight into the survey datasets of stakeholders, from which to make the right decisions policy making and decision making.

This objective of this paper is to study the characteristics of the survey datasets structures to form data variables and analyze the relations between data variables to build non-spatial and temporal multidimensional model (nSTM), apply the set of visual graphs according to the data variables representing the datasets with vertical bars indicating the values of the data variables. Visualization techniques are applied to represent survey data about the teaching and learning activities of the university into visual graphs so that university administrators can perceive the meaning hidden within multivariate datasets. This paper focuses on building a process for visual analysis of the survey dataset and transforming the survey dataset into an evaluative dataset and the dataset structure as a set of data variables and relations of them. This paper classifies the types of questions (the elementary analysis questions, overall analysis questions, and relation analysis questions see details in Sec. 4.2.2) to set up data visual analysis questionnaires and designing visual graphs help users can view - think and answer analytical questions for university development management policy making.

The rest of this paper is organized as follows. Section 2 carries out overview of the related research work from which comments and suggestions are made. Section 3 focuses on researching the characteristics of the survey dataset structures to find out the data variables to build nSTM representing the visual graphs of multivariate data. Section 4 illustrates the experimental scenario on the areas and the evaluation criteria of a university’s survey dataset to help university leaderships in formulating policies to improve and enhance the quality of university training. Section 5 briefly presents conclusions and proposes directions for development in the future.

2. Related Works

Educational data mining, data analysis, exploratory data analysis, knowledge discovery in databases, data mining, data visualization, and educational data visual analytics are promising areas in the future. These areas have been of great interest to researchers, especially as humans make decisions increasingly based on feelings and influences on the mind to make policies to improve and enhance the training quality of universities and social organizations as a premise for the future. This paper makes an overview of some scientific achievements in data analysis by visual methods.

With one’s own cognitive stream, human realizes the perception of the objects, that cognitive stream is made into a hierarchy system from viewing — perceiving — cognizing — analyzing — understanding — remembering. This hierarchy system helps human perceive real-world objects that happen automatically and continuously over time and create a closely linked chain. The data visualization process is a process of information retrieval and knowledge discovery from data, Chen and associates propose that this visualization process consists of two main components, the computational spatial component and the perceiving spatial component and visual cognizing. In addition, this process has additional components including, the visualization process complementary component, the knowledge-based system component, and the component that is extended into knowledge support infrastructure engineering.⁴ Knowledge-based visualization workflows with visualization process that simulate visualization transform data through the stream of visualizations over time and transform that data into knowledge using ability of argument based on specific components already present. This approach alleviates the difficulty of recording human knowledge.

The model of knowledge generation in the area of visual analytics proposed by Sacha and associates takes the role of humans involved in the process as the center and the rest of the process is automatically performed by the computer. Humans perform activities through discovery, inspection, and knowledge generation processes, which express human–computer interactions.⁵ Daniel Keim and associates propose a combination of modeling and visualization methods with the main purpose of helping humans participate in the process of knowledge discovering from data to build a visual analytics system.⁶ With the modeling method, the data is automatically analyzed by the user requesting it, while with the visualization method, it is human-centered in the cognitive process. In other words, the visualization approach is based on the experience and available knowledge of each human in many areas of expertise to be able to extract information and discover knowledge from a multivariate dataset. The data visual analytics process consists of three main components proposed by Keim and associates⁷ based on the visualization model of Wijk.⁸ The component receives raw data sources which are analyzed using data analysis methods to create data structures from datasets of surveying. The component applies the visualization method to convert the classified data structure into visual images.

Regular evaluation of students about lecturer teaching is one of the important criteria in higher education to determine the effectiveness of lectures, courses, and majors. This assessment is still unsatisfactory because of low-response rate and quality. This research⁹ shows how to increase the response rate as well as the quality of student feedback. This research¹⁰ also addressed student evaluation of lecture teaching, with the results of this assessment being used for both courses improvement and quality assurance purposes. This research¹¹ interprets the correctness of student rating data and uses this data appropriately to evaluate lecturers and courses. Now that, more and more universities are using student ranking data to guide staffing decisions, it is important that senior administrators and experts must know information about lecturer scientific research to use and interpret in assessing lecture teaching quality. In the research,¹² the assessment of student about teaching is being widely used to measure the teaching quality of lecturers in higher education and applied it to courses, lecturers, departments, and organizations. However, the response to these surveys is low which makes this usage unreliable.

Modeling of student competency is one of the most challenging and popular research topics in educational data mining. The strength of the education dataset is its large availability, which further drives research, especially in online education.¹³ Educational data mining is to discover student-generated learning behaviors in courses and to predict student learning competency. Student learning models that have been studied and proposed by researchers¹⁴ include active interaction groups, stable learning groups, active teaching document groups, and negative learning groups. These groups also answered questionnaires on student-learning consciousness. With the results of this research, lecturers can use the mid-term forecasting system to identify groups that may be at high risks in the term exam and to overcome their academic tasks in the future.¹⁴ Over time, educational organizations have generated and stored huge amounts of educational data leading to major challenges in educational data mining.^15,16

Through the analysis of the above researches, the data mining techniques, models and processes have been appreciably contributing to discover the hidden values in the educational data. Some authors have applied for the approaches of visualization processes, knowledge generation models, modeling of student competency, and regular evaluation of students to analyze the correlation between learning and teaching activities. Therefore, this research studies educational data variables to form nSTM of multivariable data that visually represents data about teaching and learning activities to extract information and discover knowledge to serve university leaders to develop policies to improve and enhance the quality of university education. In the visual analytics and mathematical modeling, the values of dependent variables depend on the values of independent variables. The dependent variables represent the output or outcome for the change being studied. The independent variables represent the input or cause, that is, the reason for the change. In addition, a symbol representing an arbitrary input is also called an independent variable while a symbol representing an arbitrary output is also called a dependent variable. Through the above analysis, this paper has transformed all survey datasets of a university through the academic years 2016–2017, 2017–2018, 2018–2019, 2019–2020, 2020–2021 ≡ 16–17, 17–18, 18–19, 19–20, 20–21 into visual graphs based on nSTM to apply data visual analytics, which will be discussed in detail in this paper in the following sections. In the process of this survey, this paper discovered that the dataset variables including the human variable, the criterion variable, the time variable, the survey variable, the area variable, and the evaluative variable are presented in following sections.

3. Survey Datasets of the University

3.1. Datasets

The survey dataset is the evaluation sheets of a university on the occasion of the 2021 quality accreditation. These evaluation sheets are prepared according to each criterion related to the training quality of the university. The evaluation criteria are developed from the areas to be surveyed, including: Training programs, teaching and learning activities, and the capacity of graduates (including newly graduated engineers, engineers with three years of experience), and the prestige of the university to society. The evaluation sheets are scored on a point ladder of 1, 2, 3, 4, and 5. In which, point 5 indicates the best rating for each criterion, and point 1 indicates the worst rating for each criterion. These survey sheets were conducted in a survey with the humans who are working in the university related to the quality of training and those who are working outside the university related to the quality of training, including faculty leaders, departmental leaders, faculty members and lecturers, graduate students, and businesses (specifically, employers from businesses, by partners related to the training program of the university).

3.2. Survey contents

One of the important tasks in the process of accrediting the educational quality of a university is to survey the evaluation of those who are related to the training of the university according to each criterion. The survey is carried out according to the areas, according to the parent criteria and according to the child criteria. These criteria will be grouped into the respective areas.

3.2.1. Training program

The area of training program d₁ surveyed is the program of a university. Therefore, the proposed evaluation criteria are designed in accordance with the university training levels.

Training objectives and expected learning outcomesp₁ :

	–	The training objectives and expected learning outcomes of the training program are consistent with the reality of the university $c_{1.1}$ : This criterion is applied to survey and evaluate the appropriateness of the training program with the actual conditions of the training university.
	–	The training objectives and output standards of the training program are consistent with the needs of society and businesses $c_{1.2}$ : This criterion is applied to survey and evaluate the appropriateness of training programs with social and business needs.
	–	The training objectives and output standards of the training program are consistent with the needs of the learners themselves $c_{1.3}$ : This criterion is applied to survey and evaluate the appropriateness of the training program for learners.

Features of the training program $p_{2}$ :

	–	Updating of the training program $c_{2.1}$ : Updating of training program was surveyed and assessed the compatibility of training programs with social needs and corporate recruiters.
	–	The appropriateness of the training program $c_{2.2}$ : The rationality of the training program was surveyed and assessed the reasonableness of the time distribution between the number of theoretical and practical periods of the subjects.
	–	The flexibility of the training program $c_{2.3}$ : The flexibility of the training program was surveyed and assessed the flexibility of the training program structure, in which the modules of the program can be swapped in time reasonably.
	–	The responsiveness of the training program $c_{2.4}$ : The responsiveness of the training program was surveyed and assessed the responsiveness of the training program’s content to the announced training objectives and output standards.

3.2.2. Teaching activities

The area of teaching activities $d_{2}$ surveyed is one of the core activities of a university. Therefore, the proposed evaluation criteria are designed in accordance with university teaching lecturer. The criteria surveyed in the content of teaching activities include the following:

Evaluation of teaching activities $p_{3}$ :

	–	Lecturer qualifications $c_{3.1}$ : The criteria for the qualifications of the lecturers are used to survey and evaluate the lecturers’ deep understanding of the modules and/or courses in charge of teaching.
	–	Combination of theory and practice $c_{3.2}$ : The criteria for the combination of theory and practice are used to survey and evaluate the ability of teachers to combine theory and practice when teaching.
	–	The impartation ability of lecturers $c_{3.3}$ : The criteria of lecturer ability to communicate is used to survey and evaluate the ability of lecturers to convey the content of the lecture to students. The more knowledge students receive from the lecturer, the better the lecturer’s ability to communicate is assessed.
	–	Lecturer’s method of communication $c_{3.4}$ : The criteria for the lecturer’s method of communication is used to survey and evaluate the function of the lecturer, in which the teaching method of the lecturer is required by the content of the lecture and the existing knowledge of the students. The more appropriate the teacher’s method of communication is with the lecture and the existing knowledge of the students, the better the criteria for the method of communication will be evaluated.
	–	Study adviser $c_{3.5}$ : The criteria for academic counseling are used to survey the lecturer’s sense of responsibility for providing academic counseling to students, helping students achieve more effective, and improved learning results. Lecturers implement good counseling and regular counseling for students in learning, leading to more effective and improved student learning outcomes, so the criteria for academic counseling are highly appreciated.

3.2.3. Competence of new graduates

Competence areas of new graduates $d_{3}$ are the results of the university training. The quality training of the university is assessed by the knowledge, skills, and attitudes of students when starting to integrate into the community of professionals and the community of people in the same agency.

Knowledge criteria $p_{4}$ :

	–	The new graduates’ knowledge $c_{4.1}$ : The criteria for new graduates’ knowledge are used to survey the knowledge of professional theory and practical skills in the professional area of new graduates.
	–	The knowledge of students after three years graduation $c_{4.2}$ : The criteria for knowledge of students after three years of graduation are used to investigate the creative application of professional theory and practical skills to practical work at enterprises. This criterion is evaluated by comparing the knowledge and skills of students after three years of graduation, who must meet the requirements of proficient making and guiding others, this is a requirement for people working in specialized area.

The criteria of skills $p_{5}$ :

	–	Foreign language skill $c_{5.1}$ : The criteria for foreign language skills are used to survey the ability of new graduates to use foreign languages in professional work and communication.
	–	Information technology skills $c_{5.2}$ : The criteria for Information Technology (IT) skill are used to survey the ability of applying IT in their work of new graduates. The more flexible students use IT in their work, the higher the criteria for IT skills are appreciated.
	–	Communication skills $c_{5.3}$ : The criteria for communication skills are the skills and distinct attitudes of each person in communicating with others. The criteria of communication skills are used to survey the ability and attitude of fresh graduates in communicating with colleagues and administrators in a practical working environment.
	–	Job search skills $c_{5.4}$ : The criteria for job search skills are used to survey the ability of new graduates to find jobs. Job search skills are related to the flexibility and acumen of new graduates in job search activities in order to find a job that is suitable for their expertise, good working environment, and good salary.
	–	Critical thinking skills $c_{5.5}$ : The criteria of critical thinking skills are used to survey the thinking ability of newly graduated students in problem analysis for commenting, remarking and evaluation leading to increase the operational efficiency of enterprises.
	–	Creative thinking skills $c_{5.6}$ : The criteria for creative thinking skills are used to survey the thinking ability of newly graduated students to discover new problems of students in actual activities of enterprises. Creative thinking involves individual skills and is enhanced on the basis of knowledge and skills learned in school.
	–	Economic thinking skills $c_{5.7}$ : The criteria of economic thinking skills are used to survey the interest of new graduates about the economic efficiency of enterprises in the jobs performed at enterprises.
	–	Job handling skills $c_{5.8}$ : The criteria for job handling skills are used to survey the capacity of new graduates in solving arising problems in order to bring benefits to the enterprises.
	–	Teamwork skills $c_{5.9}$ : The criteria for teamwork skills are used to survey the attitudes of new graduates in cooperation and sharing experiences with colleagues to complete the tasks of the enterprise with the highest possible efficiency.

Motivation, confidence and responsibility criteria $p_{6}$ :

	–	Motivation $c_{6.1}$ : The motivation of each student plays a very important role throughout the learning process at the university. The motivation criteria are used to survey the spirit of hard work, eagerness to learn, and each student must have a sense of responsibility for the future career they have chosen to study.
	–	Confidence $c_{6.2}$ : The confidence is a personality of each person. The confidence criteria are used to survey the attitude of new graduates who fully meet relevant activities in the enterprise.
	–	The sense of responsibility $c_{6.3}$ : The sense of responsibility is also a personality of each person. The responsibility criteria are used to survey students’ responsibilities in learning, researching, self-study, participating in school, community, business, and social activities.

3.2.4. Prestige of the university to society

The area of prestige of the $d_{4}$ university is related to the role and position of the university in society and is influenced by the objectives, mission, and vision of the university as announced. The higher the prestige of the university, the greater the competitive advantage of the degree awarded by the university. The higher the university’s prestige, the easier it is for employers to accept a university’s degree, meaning that its graduates are more likely to get a job.

The advantage of competition, employment, and cooperation with partners $p_{7}$ :

	–	The advantage of competition of the degree $c_{7.1}$ : The university’s competitive advantage is related to the value on which it has built its teaching and scientific research activities as well as its contributions to society. The criteria for competitive advantage are used to survey the position of the university in society, for employers, for affiliated schools, etc.
	–	Employment waiting period $c_{7.2}$ : Employment waiting period is related to the time which is necessary for a new graduate to find a job, from graduation to full employment. This time depends on the prestige of the school and the flexibility and responsiveness of new graduates. The criteria for employment waiting period are used to survey the difficulty level of new graduates in applying for jobs. The shorter the student’s employment waiting period, the higher the university’s prestige and position with employers, partners, and society, contributing to the value of the school’s educational quality.
	–	Coordination between universities and partners $c_{7.3}$ : The criteria for coordination between universities and partners are used to survey the degree of relationship between the university and partners from many related areas, the level of this relationship has an impact on the job prospects of graduate students.

3.3. Survey dataset structures

3.3.1. Human variable

Human variable is individuals who work within the university and those outside the university who jointly influence the training program and the overall value of the university invited by the university for an assessment of the university’s performance. The humans of university are classified into human classes according to the nature of the relations between the human and university. The survey dataset is classified into five human classes, including lecturer object class, school administrative management object class, school educational administration object class, student and alumni object class of the university, and the university business partner object class. The mathematically encoded human classes are $H_{m} | m = 1, 2, 3, 4, 5$ .

$H_{1}$ : The lecturer object class of the training programme, including $N_{1} \equiv N_{m}$ members.
$H_{2}$ : The administrative management object class of university, including $N_{2} \equiv N_{m}$ members.
$H_{3}$ : The educational administration object class of university, including $N_{3} \equiv N_{m}$ members.
$H_{4}$ : The student and alumni object class of university, including $N_{4} \equiv N_{m}$ members.
$H_{5}$ : The business partner object class of university, including $N_{5} \equiv N_{m}$ members.

3.3.2. Criteria variable

The criteria variable is one of the various activity aspects of the university that is related to the training quality of a training program. In the above criteria, for evaluating, the teaching and learning activities of the university were presented. This number of criteria can be added if more detailed assessment is required. These criteria have been grouped into groups of criteria according to each area to evaluate the quality of the training program, and group of criteria according to a parent criterion that has many child criteria. Criteria variables are mathematically coded as a set of criteria or a set of areas $C \equiv {c_{k . j} | k = 1, 2, 3, \dots; j = 1, 2, 3, \dots} \equiv {D_{k} | k = 1, 2, 3, \dots}$ , in which, each area $D_{k}$ is a set of many criteria $c_{k . j}$ . In which, each parent criterion is a set of several child criteria $c_{j}$ .

3.3.3. Temporal variable

Temporal variable is a set of time units, notations $T \equiv {t_{i} | i = 1, 2, 3, 4, 5, \dots}$ , in which $t_{i}$ is each time unit of an academic year. This survey is carried out 5-time units $T = {t_{1}, t_{2}, t_{3}, t_{4}, t_{5}} = {16 - 17, 17 - 18, 18 - 19, 19 - 20, 20 - 21}$ .

3.3.4. Survey variable

Survey variable is a set of values $r_{m . x . k . j}^{n_{m}}$ , each value given by a survey human $h^{n_{m}} | n_{m} = 1, 2, 3, \dots, N_{m}; m = 1, 2, 3, 4, 5$ for parent criteria $p_{k} | k = 1, 2, 3, \dots$ there are child criteria $c_{j} | j = 1, 2, 3, \dots$ according to each area $d_{x} | x = 1, 2, 3, \dots$ on the evaluation sheet is shown in the form $c_{k . j} | k = 1, 2, 3, \dots$ .; $j = 1, 2, 3, \dots$ Each value $r_{m . x . k . j}^{n_{m}}$ of the survey variable is assigned a value of the set of scores $S \equiv {s_{δ} | δ = 1, 2, 3, 4, 5} \equiv {1, 2, 3, 4, 5}$ by human $h^{n_{m}}$ . Here, score $s_{δ} = s_{5} \equiv 5$ means best rating and score $s_{δ} = s_{1} \equiv 1$ means worst rating.

3.3.5. Evaluative variable

Evaluative variable is the number of humans of human class $H_{m} | m =1, 2, 3, 4, 5$ for criteria $c_{k . j} | k = 1, 2, 3, \dots$ ; $j = 1, 2, 3, \dots$ same score $s_{δ} | δ =1, 2, 3, 4, 5$ as $q_{m . x . k . j}^{δ} (t_{i})$ . The below data in Table 1 illustrates that lecturer objects of human class $H_{1}$ for criterion $c_{1.3}$ have the same score as $q_{1.1.1.3}^{5} = 25$ of school year 19–20. The evaluative variable R is defined as a set of values evaluated by a feature class against a parent criterion and certain child criteria or a certain area. The $R_{m . x . k . j} (t_{i})$ value of the R evaluator is calculated from the value $r_{m . x . k . j}^{n_{m}} (t_{i})$ or from the value $q_{m . x . k . j}^{δ} (t_{i})$ together with corresponding score $s_{δ}$ .

Table 1. Data table structure established from survey datasets. Table values using survey data from a faculty within a university for the 2019–2020 school year (Notation: M is arithmetic average values).
						H₁						H₂
$δ$						1	2	3	4	5	$R_{1 . x . k . j}$	1	2	3	4	5	$R_{2 . x . k . j}$
$x$	$d_{x}$	k	$p_{k}$	j	$c_{j}$	$q_{1 . x . k . j}^{δ}$					M	$q_{2 . x . k . j}^{δ}$					M
1	d₁	1	p₁	1	$c_{1.1}$	1	5	12	7	21	3.91	19	55	2	11	5	2.22
				2	$c_{1.2}$	1	7	7	10	21	3.93	14	64	2	11	1	2.14
				3	$c_{1.3}$	1	3	10	7	25	4.13	26	55	2	6	3	1.97
		2	p₂	1	$c_{2.1}$	2	3	10	23	8	3.7	2	8	22	55	5	3.58
				2	$c_{2.2}$	2	5	8	23	8	3.65	2	12	14	60	4	3.57
				3	$c_{2.3}$	2	13	8	11	12	3.39	2	6	25	56	3	3.57
				4	$c_{2.4}$	2	3	10	23	8	3.7	3	2	36	46	5	3.52
2	d₂	3	p₃	1	$c_{3.1}$	3	5	8	20	10	3.63	2	5	10	55	20	3.93
				2	$c_{3.2}$	2	11	10	15	8	3.35	2	4	11	63	12	3.86
				3	$c_{3.3}$	2	3	11	18	12	3.76	3	3	5	55	26	4.07
				4	$c_{3.4}$	2	3	10	23	8	3.7	3	5	2	46	36	4.16
				5	$c_{3.5}$	2	8	8	18	10	3.57	3	12	5	52	20	3.8
…	…	…	…	…	…	…	…	…	…	…	…	…	…	…	…	…	…
						H₃						H₄
$δ$						1	2	3	4	5	$R_{3 . x . k . j}$	1	2	3	4	5	$R_{4 . x . k . j}$
$x$	$d_{x}$	k	$p_{k}$	j	$c_{j}$	$q_{3 . x . k . j}^{δ}$					M	$q_{4 . x . k . j}^{δ}$					M
1	d₁	1	p₁	1	$c_{1.1}$	4	12	3	10	4	2.94	2	19	64	20	5	3.06
				2	$c_{1.2}$	3	15	3	7	5	2.88	2	18	63	22	5	3.09
				3	$c_{1.3}$	2	8	4	15	4	3.33	2	29	55	18	6	2.97
		2	p₂	1	$c_{2.1}$	4	3	12	3	11	3.42	2	24	56	23	5	3.05
				2	$c_{2.2}$	2	5	12	3	11	3.48	2	14	60	30	4	3.18
				3	$c_{2.3}$	4	7	10	3	9	3.18	2	20	56	29	3	3.1
				4	$c_{2.4}$	4	5	11	3	10	3.3	3	31	46	25	5	2.98
2	d₂	3	p₃	1	$c_{3.1}$	1	5	13	8	6	3.39	2	5	20	55	28	3.93
				2	$c_{3.2}$	2	6	9	7	9	3.45	2	4	12	63	29	4.03
				3	$c_{3.3}$	2	10	5	8	8	3.3	3	3	21	55	28	3.93
				4	$c_{3.4}$	1	5	5	12	10	3.76	3	5	36	46	20	3.68
				5	$c_{3.5}$	2	7	7	8	9	3.45	3	12	20	49	26	3.75
…	…	…	…	…	…	…	…	…	…	…	…	…	…	…	…	…	…

3.4. The relations between data variables

Mathematically, exploring the relations between variables helps us to know that the structural variables of datasets have independent and dependent relations.^2,3,17,18 The independent variable varies independently itself without having to depend on the changes of other variables, the independent variable also is referenced by other variables. A dependent variable is a variable whose value depends on the value of one or more other variables. The dependent variable can also be referenced by one or more other variables. From here, the paper defines the evaluation variable R that depends on the human variable H, the score variable S, the criterion variable C, and the time variable T.

•

H \times S \times C \times T \to R, <math display="block" altimg="eq-00123.gif"><mi>H</mi><mo>\times</mo><mi>S</mi><mo>\times</mo><mi>C</mi><mo>\times</mo><mi>T</mi><mo>\to</mo><mi>R</mi><mo>,</mo></math> (1)

•

{h m} \times {s δ} \times {c k . j} \times {t i} \to {R m . k . j (t i)} | m = 1, 2, 3, 4, 5; k = 1, 2, \dots; δ = 1, 2, 3, 4, 5; j = 1, 2, \dots; i = 1, 2, \dots, <math display="block" altimg="eq-00124.gif"><mtable displaystyle="true" equalrows="false" equalcolumns="false"><mtr><mtd columnalign="center"><mo stretchy="false">{</mo><msub><mrow><mi>h</mi></mrow><mrow><mi>m</mi></mrow></msub><mo stretchy="false">}</mo><mo>\times</mo><mo stretchy="false">{</mo><msub><mrow><mi>s</mi></mrow><mrow><mi>δ</mi></mrow></msub><mo stretchy="false">}</mo><mo>\times</mo><mo stretchy="false">{</mo><msub><mrow><mi>c</mi></mrow><mrow><mi>k</mi><mo>.</mo><mi>j</mi></mrow></msub><mo stretchy="false">}</mo><mo>\times</mo><mo stretchy="false">{</mo><msub><mrow><mi>t</mi></mrow><mrow><mi>i</mi></mrow></msub><mo stretchy="false">}</mo><mo>\to</mo><mo stretchy="false">{</mo><msub><mrow><mi>R</mi></mrow><mrow><mi>m</mi><mo>.</mo><mi>k</mi><mo>.</mo><mi>j</mi></mrow></msub><mo stretchy="false">(</mo><msub><mrow><mi>t</mi></mrow><mrow><mi>i</mi></mrow></msub><mo stretchy="false">)</mo><mo stretchy="false">}</mo><mi>|</mi><mi>m</mi><mo>=</mo><mn>1</mn><mo>,</mo><mn>2</mn><mo>,</mo><mn>3</mn><mo>,</mo><mn>4</mn><mo>,</mo><mn>5</mn><mo>;</mo><mspace width="1em"></mspace><mi>k</mi><mo>=</mo><mn>1</mn><mo>,</mo><mn>2</mn><mo>,</mo><mo>\dots</mo><mo>;</mo></mtd></mtr><mtr><mtd columnalign="center"><mi>δ</mi><mo>=</mo><mn>1</mn><mo>,</mo><mn>2</mn><mo>,</mo><mn>3</mn><mo>,</mo><mn>4</mn><mo>,</mo><mn>5</mn><mo>;</mo><mspace width="1em"></mspace><mi>j</mi><mo>=</mo><mn>1</mn><mo>,</mo><mn>2</mn><mo>,</mo><mo>\dots</mo><mo>;</mo><mspace width="1em"></mspace><mi>i</mi><mo>=</mo><mn>1</mn><mo>,</mo><mn>2</mn><mo>,</mo><mo>\dots</mo><mo>,</mo></mtd></mtr></mtable></math> (1.1a)

•

–

Human $h_{m}^{n_{m}}$ of human class $H_{m}$ for criterion $c_{k . j}$ score $s_{δ}$ at time unit $t_{i}$ specified by

r δ . n m m . k . j (t i) : (h n m m, s δ, c k . j, t i) \to r δ . n m m . k . j (t i) . <math display="block" altimg="eq-00132.gif"><msubsup><mrow><mi>r</mi></mrow><mrow><mi>m</mi><mo>.</mo><mi>k</mi><mo>.</mo><mi>j</mi></mrow><mrow><mi>δ</mi><mo>.</mo><msub><mrow><mi>n</mi></mrow><mrow><mi>m</mi></mrow></msub></mrow></msubsup><mo stretchy="false">(</mo><msub><mrow><mi>t</mi></mrow><mrow><mi>i</mi></mrow></msub><mo stretchy="false">)</mo><mo>:</mo><mo stretchy="false">(</mo><msubsup><mrow><mi>h</mi></mrow><mrow><mi>m</mi></mrow><mrow><msub><mrow><mi>n</mi></mrow><mrow><mi>m</mi></mrow></msub></mrow></msubsup><mo>,</mo><msub><mrow><mi>s</mi></mrow><mrow><mi>δ</mi></mrow></msub><mo>,</mo><msub><mrow><mi>c</mi></mrow><mrow><mi>k</mi><mo>.</mo><mi>j</mi></mrow></msub><mo>,</mo><msub><mrow><mi>t</mi></mrow><mrow><mi>i</mi></mrow></msub><mo stretchy="false">)</mo><mo>\to</mo><msubsup><mrow><mi>r</mi></mrow><mrow><mi>m</mi><mo>.</mo><mi>k</mi><mo>.</mo><mi>j</mi></mrow><mrow><mi>δ</mi><mo>.</mo><msub><mrow><mi>n</mi></mrow><mrow><mi>m</mi></mrow></msub></mrow></msubsup><mo stretchy="false">(</mo><msub><mrow><mi>t</mi></mrow><mrow><mi>i</mi></mrow></msub><mo stretchy="false">)</mo><mo>.</mo></math> (1.1d)

–

Evaluation by criterion $c_{k . j}$ at time unit $t_{i}$ of human class $H_{m}$ can be mathematically defined by arithmetic average. Arithmetic average of the set of scores ${r_{m . k . j}^{δ . n_{m}} (t_{i}) | n_{m} = 1, 2, 3, \dots, N_{m}; k = 1, 2, 3, \dots; δ = 1, 2, 3, 4, 5}$ given by human class $H_{m}$ for criterion $c_{k . j}$ at time unit $t_{i}$ is as follows:

Rm.k.j(ti)=1NmNm∑nm=1rδ.nmm.k.j(ti)=∑5δ=1δ.qδm.k.j(ti)∑5δ=1qδm.k.j(ti).<math display="block" altimg="eq-00140.gif"><msub><mrow><mi>R</mi></mrow><mrow><mi>m</mi><mo>.</mo><mi>k</mi><mo>.</mo><mi>j</mi></mrow></msub><mo stretchy="false">(</mo><msub><mrow><mi>t</mi></mrow><mrow><mi>i</mi></mrow></msub><mo stretchy="false">)</mo><mo>=</mo><mfrac><mrow><mn>1</mn></mrow><mrow><msub><mrow><mi>N</mi></mrow><mrow><mi>m</mi></mrow></msub></mrow></mfrac><munderover accentunder="true" accent="true"><mrow><mo>∑</mo></mrow><mrow><msub><mrow><mi>n</mi></mrow><mrow><mi>m</mi></mrow></msub><mo>=</mo><mn>1</mn></mrow><mrow><msub><mrow><mi>N</mi></mrow><mrow><mi>m</mi></mrow></msub></mrow></munderover><msubsup><mrow><mi>r</mi></mrow><mrow><mi>m</mi><mo>.</mo><mi>k</mi><mo>.</mo><mi>j</mi></mrow><mrow><mi>δ</mi><mo>.</mo><msub><mrow><mi>n</mi></mrow><mrow><mi>m</mi></mrow></msub></mrow></msubsup><mo stretchy="false">(</mo><msub><mrow><mi>t</mi></mrow><mrow><mi>i</mi></mrow></msub><mo stretchy="false">)</mo><mo>=</mo><mfrac><mrow><msubsup><mrow><mo>∑</mo></mrow><mrow><mi>δ</mi><mo>=</mo><mn>1</mn></mrow><mrow><mn>5</mn></mrow></msubsup><mi>δ</mi><mo>.</mo><msubsup><mrow><mi>q</mi></mrow><mrow><mi>m</mi><mo>.</mo><mi>k</mi><mo>.</mo><mi>j</mi></mrow><mrow><mi>δ</mi></mrow></msubsup><mo stretchy="false">(</mo><msub><mrow><mi>t</mi></mrow><mrow><mi>i</mi></mrow></msub><mo stretchy="false">)</mo></mrow><mrow><msubsup><mrow><mo>∑</mo></mrow><mrow><mi>δ</mi><mo>=</mo><mn>1</mn></mrow><mrow><mn>5</mn></mrow></msubsup><msubsup><mrow><mi>q</mi></mrow><mrow><mi>m</mi><mo>.</mo><mi>k</mi><mo>.</mo><mi>j</mi></mrow><mrow><mi>δ</mi></mrow></msubsup><mo stretchy="false">(</mo><msub><mrow><mi>t</mi></mrow><mrow><mi>i</mi></mrow></msub><mo stretchy="false">)</mo></mrow></mfrac><mo>.</mo></math>(1.1e)

–

Here, $q_{m . k . j}^{δ} (t_{i})$ and $r_{m . k . j}^{δ . n_{m}} (t_{i})$ are the number of humans of class $H_{m}$ for the same score $s_{δ}$ for criterion $c_{k . j}$ at unit time time $t_{i}$ .

Arithmetic average values of the set of scores

{r δ . n m m . k . j (t i) | n m = 1, 2, 3, \dots, N m; k = 1, 2, 3, \dots; δ = 1, 2, 3, 4, 5; j = 1, 2, 3, \dots; i = 1, 2, \dots} <math display="block" altimg="eq-00147.gif"><mrow><mtable displaystyle="true" equalrows="false" equalcolumns="false"><mtr><mtd columnalign="left"><mo stretchy="false">{</mo><msubsup><mrow><mi>r</mi></mrow><mrow><mi>m</mi><mo>.</mo><mi>k</mi><mo>.</mo><mi>j</mi></mrow><mrow><mi>δ</mi><mo>.</mo><msub><mrow><mi>n</mi></mrow><mrow><mi>m</mi></mrow></msub></mrow></msubsup><mo stretchy="false">(</mo><msub><mrow><mi>t</mi></mrow><mrow><mi>i</mi></mrow></msub><mo stretchy="false">)</mo><mi>|</mi><msub><mrow><mi>n</mi></mrow><mrow><mi>m</mi></mrow></msub><mo>=</mo><mn>1</mn><mo>,</mo><mn>2</mn><mo>,</mo><mn>3</mn><mo>,</mo><mo>\dots</mo><mo>,</mo><msub><mrow><mi>N</mi></mrow><mrow><mi>m</mi></mrow></msub><mo>;</mo><mspace width=".5em"></mspace><mi>k</mi><mo>=</mo><mn>1</mn><mo>,</mo><mn>2</mn><mo>,</mo><mn>3</mn><mo>,</mo><mo>\dots</mo><mo>;</mo><mi>δ</mi><mo>=</mo><mn>1</mn><mo>,</mo><mn>2</mn><mo>,</mo><mn>3</mn><mo>,</mo><mn>4</mn><mo>,</mo><mn>5</mn><mo>;</mo><mspace width=".5em"></mspace><mi>j</mi><mo>=</mo><mn>1</mn><mo>,</mo><mn>2</mn><mo>,</mo><mn>3</mn><mo>,</mo><mo>\dots</mo><mo>;</mo><mspace width=".5em"></mspace><mi>i</mi><mo>=</mo><mn>1</mn><mo>,</mo><mn>2</mn><mo>,</mo><mo>\dots</mo><mo stretchy="false">}</mo></mtd></mtr></mtable></mrow></math>

is defined as the evaluation value of criterion

$c_{k . j}$ by human class

$H_{m}$ at time unit

$t_{i}$ . The arithmetic average is often used as a quantitative value to indicate the average evaluation level of each criterion.

For each training program-related area, the evaluation variable R depends on the human variable H, the score variable S, the area variable D, and the time variable T. In which, the area of evaluation $d_{k}$ for the program includes the training program, teaching and learning activities, the capacity of the new graduates, and the prestige of the university.

•

H \times S \times D \times T \to R, <math display="block" altimg="eq-00152.gif"><mi>H</mi><mo>\times</mo><mi>S</mi><mo>\times</mo><mi>D</mi><mo>\times</mo><mi>T</mi><mo>\to</mo><mi>R</mi><mo>,</mo></math> (2)

•

{h m \times {s δ} \times {d x} \times {t i} \to R m . x (t i)} | m = 1, 2, 3, 4, 5; δ = 1, 2, 3, 4, 5; x = 1, 2, 3, 4, \dots; i = 1, 2, 3, 4, 5, \dots . <math display="block" altimg="eq-00153.gif"><mtable displaystyle="true" equalrows="false" equalcolumns="false"><mtr><mtd columnalign="center"><mo stretchy="false">{</mo><msub><mrow><mi>h</mi></mrow><mrow><mi>m</mi></mrow></msub><mo>\times</mo><mo stretchy="false">{</mo><msub><mrow><mi>s</mi></mrow><mrow><mi>δ</mi></mrow></msub><mo stretchy="false">}</mo><mo>\times</mo><mo stretchy="false">{</mo><msub><mrow><mi>d</mi></mrow><mrow><mi>x</mi></mrow></msub><mo stretchy="false">}</mo><mo>\times</mo><mo stretchy="false">{</mo><msub><mrow><mi>t</mi></mrow><mrow><mi>i</mi></mrow></msub><mo stretchy="false">}</mo><mo>\to</mo><msub><mrow><mi>R</mi></mrow><mrow><mi>m</mi><mo>.</mo><mi>x</mi></mrow></msub><mo stretchy="false">(</mo><msub><mrow><mi>t</mi></mrow><mrow><mi>i</mi></mrow></msub><mo stretchy="false">)</mo><mo stretchy="false">}</mo><mi>|</mi><mi>m</mi><mo>=</mo><mn>1</mn><mo>,</mo><mn>2</mn><mo>,</mo><mn>3</mn><mo>,</mo><mn>4</mn><mo>,</mo><mn>5</mn><mo>;</mo></mtd></mtr><mtr><mtd columnalign="center"><mi>δ</mi><mo>=</mo><mn>1</mn><mo>,</mo><mn>2</mn><mo>,</mo><mn>3</mn><mo>,</mo><mn>4</mn><mo>,</mo><mn>5</mn><mo>;</mo><mspace width="1em"></mspace><mi>x</mi><mo>=</mo><mn>1</mn><mo>,</mo><mn>2</mn><mo>,</mo><mn>3</mn><mo>,</mo><mn>4</mn><mo>,</mo><mo>\dots</mo><mo>;</mo><mspace width="1em"></mspace><mi>i</mi><mo>=</mo><mn>1</mn><mo>,</mo><mn>2</mn><mo>,</mo><mn>3</mn><mo>,</mo><mn>4</mn><mo>,</mo><mn>5</mn><mo>,</mo><mo>\dots</mo><mspace width=".17em"></mspace><mo>.</mo></mtd></mtr></mtable></math> (2.1a)

The evaluation value of each human of the human class $H_{m}$ for the area $d_{x}$ at the time unit $t_{i}$ is specifically calculated as follows:

•

(h n m m, s δ, d x, t i) \to R n m m . x (t i), <math display="block" altimg="eq-00157.gif"><mo stretchy="false">(</mo><msubsup><mrow><mi>h</mi></mrow><mrow><mi>m</mi></mrow><mrow><msub><mrow><mi>n</mi></mrow><mrow><mi>m</mi></mrow></msub></mrow></msubsup><mo>,</mo><msub><mrow><mi>s</mi></mrow><mrow><mi>δ</mi></mrow></msub><mo>,</mo><msub><mrow><mi>d</mi></mrow><mrow><mi>x</mi></mrow></msub><mo>,</mo><msub><mrow><mi>t</mi></mrow><mrow><mi>i</mi></mrow></msub><mo stretchy="false">)</mo><mo>\to</mo><msubsup><mrow><mi>R</mi></mrow><mrow><mi>m</mi><mo>.</mo><mi>x</mi></mrow><mrow><msub><mrow><mi>n</mi></mrow><mrow><mi>m</mi></mrow></msub></mrow></msubsup><mo stretchy="false">(</mo><msub><mrow><mi>t</mi></mrow><mrow><mi>i</mi></mrow></msub><mo stretchy="false">)</mo><mo>,</mo></math> (2.1b)

•

Rm.x(ti)=1JxJx∑j∑5δ=1δ.qδm.k.j(ti)∑5δ=1qδm.k.j(ti)=1Jx.NmJx∑jNm∑nm=1rδ.nmm.x.j(ti).<math display="block" altimg="eq-00158.gif"><msub><mrow><mi>R</mi></mrow><mrow><mi>m</mi><mo>.</mo><mi>x</mi></mrow></msub><mo stretchy="false">(</mo><msub><mrow><mi>t</mi></mrow><mrow><mi>i</mi></mrow></msub><mo stretchy="false">)</mo><mo>=</mo><mfrac><mrow><mn>1</mn></mrow><mrow><msub><mrow><mi>J</mi></mrow><mrow><mi>x</mi></mrow></msub></mrow></mfrac><munderover accentunder="true" accent="true"><mrow><mo>∑</mo></mrow><mrow><mi>j</mi></mrow><mrow><msub><mrow><mi>J</mi></mrow><mrow><mi>x</mi></mrow></msub></mrow></munderover><mfrac><mrow><msubsup><mrow><mo>∑</mo></mrow><mrow><mi>δ</mi><mo>=</mo><mn>1</mn></mrow><mrow><mn>5</mn></mrow></msubsup><mi>δ</mi><mo>.</mo><msubsup><mrow><mi>q</mi></mrow><mrow><mi>m</mi><mo>.</mo><mi>k</mi><mo>.</mo><mi>j</mi></mrow><mrow><mi>δ</mi></mrow></msubsup><mo stretchy="false">(</mo><msub><mrow><mi>t</mi></mrow><mrow><mi>i</mi></mrow></msub><mo stretchy="false">)</mo></mrow><mrow><msubsup><mrow><mo>∑</mo></mrow><mrow><mi>δ</mi><mo>=</mo><mn>1</mn></mrow><mrow><mn>5</mn></mrow></msubsup><msubsup><mrow><mi>q</mi></mrow><mrow><mi>m</mi><mo>.</mo><mi>k</mi><mo>.</mo><mi>j</mi></mrow><mrow><mi>δ</mi></mrow></msubsup><mo stretchy="false">(</mo><msub><mrow><mi>t</mi></mrow><mrow><mi>i</mi></mrow></msub><mo stretchy="false">)</mo></mrow></mfrac><mo>=</mo><mfrac><mrow><mn>1</mn></mrow><mrow><msub><mrow><mi>J</mi></mrow><mrow><mi>x</mi></mrow></msub><mo>.</mo><msub><mrow><mi>N</mi></mrow><mrow><mi>m</mi></mrow></msub></mrow></mfrac><munderover accentunder="true" accent="true"><mrow><mo>∑</mo></mrow><mrow><mi>j</mi></mrow><mrow><msub><mrow><mi>J</mi></mrow><mrow><mi>x</mi></mrow></msub></mrow></munderover><munderover accentunder="true" accent="true"><mrow><mo>∑</mo></mrow><mrow><msub><mrow><mi>n</mi></mrow><mrow><mi>m</mi></mrow></msub><mo>=</mo><mn>1</mn></mrow><mrow><msub><mrow><mi>N</mi></mrow><mrow><mi>m</mi></mrow></msub></mrow></munderover><msubsup><mrow><mi>r</mi></mrow><mrow><mi>m</mi><mo>.</mo><mi>x</mi><mo>.</mo><mi>j</mi></mrow><mrow><mi>δ</mi><mo>.</mo><msub><mrow><mi>n</mi></mrow><mrow><mi>m</mi></mrow></msub></mrow></msubsup><mo stretchy="false">(</mo><msub><mrow><mi>t</mi></mrow><mrow><mi>i</mi></mrow></msub><mo stretchy="false">)</mo><mo>.</mo></math>(2.1c)

•

Here

	–	$J_{x}$ is the number of criteria $c_{k . j}$ in the area $d_{x}$ .
	–	$D \equiv {d_{1}, d_{2}, d_{3}, d_{4}, d_{5}}$ is the evaluative area variable, where each area $d_{x}$ includes a number of parent criteria $p_{k}$ and a number of child criteria $c_{k . j}$ to be surveyed.

4. Analyzing Visually the Survey Datasets

4.1. Non-spatial and temporal multidimensional model

This paper focuses on analyzing survey datasets through the survey dataset analysis process to represent and to analyze survey dataset by visual methods. The nSTM was chosen to represent the survey datasets, and the value bar chart was chosen to represent the values of the data variables because of its visual characteristics, which has high efficiency.^3,18,19,20 The method of viewing–thinking graphs representing visually the survey datasets is applied in seminars on accrediting the university’s education quality. There, participating members view visual graphs that visually represent the survey datasets to jointly answer pre-compiled data visual analysis questions or questions that arise while members viewing–thinking visual graphs.

The survey datasets are the set of point values ${1, 2, 3, 4, 5}$ given by each human according to each criterion and each school year, each point value of the survey data variable refers to the human for the score, the assessment criteria, and the year of the university being assessed. The survey data variable is converted into an evaluative data variable for visual representation using arithmetic average operation. The evaluative variable depends on the human variable, the criterion variable or the area variable, and the time variable. Therefore, a visual graph is used to illustrate the viewing–thinking method to visually analyze the educational quality of the training program, applied in conferences on educational accreditation of the university is very important and urgent today. From the above analysis, this paper proposes nSTM that visually represents the survey datasets on the evaluative variable according to the following criteria (see Fig. 1).

Fig. 1. Models (nSTM) of value representation $R_{m . k . j} (t_{i})$ : Model (a) represents the value $R_{m . x . k . j} (t_{i})$ each area of the parent criteria with the child criteria; Model (b) represents the value $R_{m . j} (t_{i})$ by a human class; Model (c) represents the value $R_{m . k . j} (t_{i})$ for a school year; Model (d) represents the value $R_{m . k . j} (t_{i})$ for a criterion; Model (e) represents the value $R_{m . k . j} (t_{i})$ for a parent criterion with child criteria.

Based on Table 1 above, the value of $R_{m . x . k . j} (t_{i})$ is related to the human class evaluating the criteria, the score of the survey variable, the criterion, and the time unit, in which, the time unit is called the school years. $(h_{m}, s_{δ}, c_{k . j}, t_{i}) \to R_{m . k . j} (t_{i})$ (1), where the values $R_{m . k . j} (t_{i})$ are represented on the model visually as shown in Fig. 1. Figure 1(a) shows the value of $R_{m . x . k . j} (t_{i})$ rated each area by each human class for each school year; The Q plane parallel to the $(R, T)$ plane moving along the H axis represents the change of views of the human classes when evaluating each area for the criteria for many years of study. Figure 1(b) shows the $R_{m . j} (t_{i})$ value of a pre-specified school year evaluated by each feature human class $H_{m} | m = 1, 2, 3, 4, 5$ for each criterion $c_{j} | j = 1, 2, 3, \dots$ ; the Q plane parallel to the $(R, H)$ plane moving along the C axis shows different views of different human classes when evaluating criteria $c_{j} | j = 1, 2, 3, \dots$ . Figure 1(c) shows the value of $R_{m . j} (t_{i})$ evaluated by a human class for each criterion $c_{j} | j = 1, 2, 3, \dots$ of a school year; the Q plane parallel to the $(R, T)$ plane moving along the C axis represents the change of perspective of a human class when evaluating the $c_{j} | j = 1, 2, 3, \dots$ for many years of study or the change in education quality for each criterion over many years of study. Figure 1(d) shows the value $R_{m . k . j} (t_{i})$ that evaluates a parent criterion with child criteria $c_{k . j} | k = 1, 2, 3, \dots; j = 1, 2, 3, \dots$ by each human class for each school year; the Q plane parallel to the $(R, T)$ plane moving along the H axis represents the change of opinion of a human class when evaluating a parent criterion with child criteria $c_{k . j} | k = 1, 2, 3, \dots; j = 1, 2, 3, \dots$ for many years of study. Figure 1(e) shows the value of $R_{m . k . j} (t_{i})$ that evaluates a parent criterion along with the child criteria $c_{k . j} | k = 1, 2, 3, \dots; j = 1, 2, 3, \dots$ by each human class for each school year; the Q plane parallel to the $(R, T)$ plane moving along the H axis represents the change of opinion of the human classes when evaluating the parent criteria together with the child criteria for many university years or the change in educational quality for the criteria over many years of schooling.

4.2. Experiments

4.2.1. Process of visual representation of survey datasets

The education quality survey dataset of a university is collected by surveying tables consisting of many questions, in which one question corresponds to one criterion, in which each survey object scores according to each criterion on a scale of ${1, 2, 3, 4, 5}$ , a score of 5 is the best, and a score of 1 is the worst. The points in the questionnaire are stored in the computer for visual representation and data analysis according to the following steps.

Step 1: Determine the survey variable. Group the survey tables according to each human class and design the survey dataset to restructure as a data table and count the number of humans of each human class given the same score for each criterion $q_{m . k . j}^{s_{δ}}$ .
Step 2: Establish the evaluation variable. Transform the survey variable into an evaluative variable by choosing an arithmetic average transformation algorithm to calculate the value $R_{m . k . j} (t_{i})$ .
Step 3: Design 3D models. Design 3D models for representing visual graph with the value $R_{m . x . k . j} (t_{i})$ referring to $h_{m}$ , $d_{x}$ , $p_{k}$ , $c_{j}$ , and $t_{i}$ .
Step 4: Prepare analytical questionnaires. Set up an analytical questionnaire consisting of elementary questions, overall questions, and relational questions with clear assumptions and conclusions in each question.
Step 5: Analyze data. Look at the visual graphs on the models of 3D to understand the meaning of the educational accreditation of the training program and to identify the correlation between the $R_{m . x . k . j} (t_{i})$ values by answering the questions of data visual analysis.

4.2.2. Question for visual analytics of survey datasets

Visually analyzing data means answering data analysis questions. These questions can be prepared before the user begins viewing–thinking visual graph or arise during viewing visual graph and thinking about data. Each question consists of two parts, a hypothesis and a conclusion, in which the hypothesis is related to the data. Due to the different nature of the hypothetical part of the questions, the questions are classified into three categories as follows: Elementary analysis questions, overall analysis questions, and relation analysis questions.^2,3,17,18 In which, the elementary level question has assumptions related to each value of each data variable, the overall question has assumptions related to many values of a variable, and the relational question has related assumptions to the correlation between the data variables. Following are some questions related to the teaching and learning survey dataset.

Elementary level analysis questions:
Question 1: How did the business objects, which are the school’s partners, evaluate the school year 16–17, 17–18, 18–19, 19–20, or 20–21 training program? (see Fig. 2).
Question 2: How did the business object ${h_{5}}$ evaluates the capacity of new graduates through the school years 16–17, 17–18, 18–19, 19–20, and 20–21? (see Fig. 3).
Global level analysis questions:
Question 3: How did each object evaluates teaching activities according to each school year 16–17, 17–18, 18–19, 19–20, and 20–21? (see Fig. 4).
Question 4: Parent criterion $p_{1}$ has child criteria $c_{1.1}, c_{1.2}, c_{1.3}$ how is evaluated by objects $h_{1}$ , $h_{2}$ , $h_{3}$ , $h_{4}$ , $h_{5}$ according to each school year 16–17, 17–18, 18–19, 19–20, and 20–21? (see Fig. 5).
Questions to analyze the relationship between variables:
Question 5: Based on Fig. 2, how is the training program of the school years evaluated by the objects ${h_{1}, h_{2}, h_{3}, h_{4}, h_{5}}$ ? What’s the difference? Why different if yes, why not different?
– Answer: The analysts who view–think the visual graph (as shown in Fig. 2) can answer that the objects rated for the training program are increasing.
Question 6: Based on Fig. 3, compare the evaluation of the humans ${h_{1}, h_{2}, h_{3}, h_{4}, h_{5}}$ on the student’s ability over each academic year? What’s the difference? What does that difference mean?
– Answer: The analysts who view–think the visual graph (as shown in Fig. 3) can answer that the humans in the school ${h_{1}, h_{2}, h_{3}}$ evaluate increase points, and the business objects ${h_{5}}$ evaluate reduced points according to each university year. This shows that the training program, teaching quality of the university need to be adjusted and changed to be more suitable for students and the actual situation and needs of society.

Fig. 2. Humans $h_{1}, h_{2}, h_{3}, h_{4}, h_{5}$ evaluate for the area of training programme for the school year 16–17, 17–18, 18–19, 19–20, and 20–21.

Fig. 3. Humans $h_{1}, h_{2}, h_{3}, h_{5}$ evaluate the competence areas of new graduates for each school year 16–17, 17–18, 18–19, 19–20, and 20–21.

Fig. 4. Humans $h_{1}, h_{2}, h_{3}, h_{4}, h_{5}$ evaluate the area of teaching activities according to each school year 16–17, 17–18, 18–19, 19–20, and 20–21.

Fig. 5. Humans $h_{1}, h_{2}, h_{3}, h_{4}, h_{5}$ evaluate the parent criterion $p_{1}$ with child criteria $c_{1.1}, c_{1.2}, c_{1.3}$ according to each school year 16–17, 17–18, 18–19, 19–20, and 20–21.

4.2.3. The case of surveying a multivariate dataset

In actual deploy, the process of visual representation and analysis is applied to the survey dataset of a university in the school years 16–17, 17–18, 18–19, 19–20, and 20–21. This paper presents models representing survey datasets in many different cases in knowledge discovery from multivariate datasets. In this section, Fig. 5 is presented as an illustration for the educational quality assessment survey that the humans ${h_{1}, h_{2}, h_{3}, h_{4}, h_{5}}$ evaluate the parent criterion $p_{1}$ with the child criterion $c_{1.2}$ according to school years 16–17, 17–18, 18–19, 19–20, and 20–21 are illustrated in general form as shown in Fig. 6.

The analysts who observe Fig. 6 representing a value $R_{m . k . j} (t_{i})$ of the evaluation variable on the model $(R, H, T)$ representing the response criteria of the training programme with $c_{1.2}$ criterion. The values $R_{m . k . j} (t_{i})$ display as value bars at positions with coordinates in the model $(h_{m}, t_{i}) | m = 1, 2, 3, 4; i = 1, 2, 3, 4, 5$ with elevation proportional to the values $R_{m . 1.2} (t_{i}) | m = 1, 2, 3, 4; k = 1, 2, 3, \dots; i = 1, 2, 3, 4, 5$ . At the university’s education quality meetings, all models representing the survey dataset are presented so that all members of the panel can be seen understood and jointly evaluated to find out strengths, the weakness of the training program to form policies to improve or enhance the university’s educational quality.

In addition, the criterion $c_{1.2}$ is represented by a visual graph (see Fig. 6) which has attracted the attention of the university’s direct administrators. Here, the responsiveness of the training program is reduced from school year 16–17 to 18–19 in the order of assessment of each human class ${h_{1}, h_{2}, h_{3}, h_{4}, h_{5}}$ while from school year 19–20 to 20–21 is again incremented by these ${h_{1}, h_{2}, h_{3}, h_{4}, h_{5}}$ human classes. Thus, the question posed to the senior administrators of the university at these discussions is why these target groups have assessed the responsiveness of the training program to be lower than in previous years and how the policy to improve should be.

5. Conclusions and Future Work

This paper reviewed the researches related to data visualization, data visual analytics, and made new comments and suggestions. This paper focused on visual analytics of datasets of the quality survey of a university’s training program for use in conferences on accrediting the university’s educational quality. We studied the characteristics of the survey dataset structure and built a process to analyze the survey dataset about the educational quality of a curriculum at a university to establish the set of evaluation data variables to analyze the relations of the evaluative data variable with the evaluative object variable, the evaluation criterion variable, and the temporal variable, to establish visual graphs representing the dataset. This paper builds nSTM to represent a visual graph with bars indicating the value of data variables. This research was deployed to transform all survey datasets for school years 16–17, 17–18, 18–19, 19–20, and 20–21 into visual graphs. In addition, this research can also be extended towards forming a visual analytics model of data related to university education quality in order to support leaders at all levels to make policies to improve and enhance the quality of university education in the future.

References

1. J. J. Thomas and K. A. Cook , Illuminating the Path the Research and Development Agenda for Visual Analytics (IEEE Computer Society Press, Los Alamitos, 2005). Google Scholar
2. N. Andrienko and G. Andrienko , Exploratory Analysis of Spatial and Temporal Data - A Systematic Approach (Springer-Verlag, Berlin Heidelberg, 2006). https://doi.org/10.1007/3-540-31190-4 Google Scholar
3. D. Van Pham and P. Vinh Tran , A System and Model of Visual Data Analytics Related to Junior High School Students, in Context-Aware Systems and Applications, and Nature of Computation and Communication, Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering, Vol. 298 (Springer, Cham, 2019), pp. 105–126. https://doi.org/10.1007/978-3-030-34365-1_9 Crossref, Google Scholar
4. M. Chen et al., Data, information, and knowledge in visualization, IEEE Comput. Graph. Appl. 29(1) (2009) 12–19. https://doi.org/10.1109/MCG.2009.6 Crossref, Google Scholar
5. D. Sacha, A. Stoffel, F. Stoffel, B. C. Kwon, G. Ellis and D. A. Keim , Knowledge generation model for visual analytics, IEEE Trans. Vis. Comput. Graph. 20(12) (2014) 1604–1613. https://doi.org/10.1109/TVCG.2014.2346481 Crossref, Google Scholar
6. D. Keim, J. Kohlhammer, G. Ellis and M. Florian , Mastering the Information Age Solving Problems with Visual Analytics (Eurographics Association, Goslar, 2010). https://doi.org/10.2312/14803 Google Scholar
7. D. Keim, G. Andrienko, J.-D. Fekete, C. Görg, J. Kohlhammer and G. Melançon , Visual analytics: Definition, process and challenges, in Information Visualization - Human-Centered Issues and Perspectives, eds. K. Andreas, T. S. John, F. Jean-Daniel and N. Chris , Lecture Notes in Computer Science Book Series, Vol. 4950 (Springer, Berlin, Heidelberg, 2008), pp. 154–175. Crossref, Google Scholar
8. J. J. V. Wijk , The value of visualization, in VIS 05. IEEE Visualization (IEEE, 2005), pp. 79–86. https://doi.org/10.1109/VISUAL.2005.1532781 Crossref, Google Scholar
9. M. Nederhand, J. Auer, B. Giesbers, A. Scheepers and E. van der Gaag , Improving student participation in SET: effects of increased transparency on the use of student feedback in practice, Assess. Eval. High. Educ. 48(1) (2023) 1–14. https://doi.org/10.1080/02602938.2022.2052800 Crossref, Google Scholar
10. L. P. Macfadyen, S. Dawson, S. Prest and D. Gašević , Whose feedback? A multilevel analysis of student completion of end-of-term teaching evaluations, Assess. Eval. High. Educ. 41(6) (2016) 821–839. https://doi.org/10.1080/02602938.2015.1044421 Crossref, Google Scholar
11. A. R. Linse , Interpreting and using student ratings data: Guidance for faculty serving as administrators and on evaluation committees, Stud. Educ. Eval. 54 (2017) 94–106. https://doi.org/10.1016/j.stueduc.2016.12.004 Crossref, Google Scholar
12. M. Goos and A. Salomons , Measuring teaching quality in higher education: Assessing selection bias in course evaluations, Res. High. Educ. 58(4) (2016) 341–364. https://doi.org/10.1007/s11162-016-9429-8 Crossref, Google Scholar
13. A. Khan and S. K. Ghosh , Student performance analysis and prediction in classroom learning: A review of educational data mining studies, Educ. Inf. Technol. 26(1) (2021) 205–240. https://doi.org/10.1007/s10639-020-10230-3 Crossref, Google Scholar
14. H.-C. Hung, I. F. Liu, C.-T. Liang and Y.-S. Su , Applying educational data mining to explore students’ learning patterns in the flipped learning approach for coding education, Symmetry 12(2) (2020) 213. https://doi.org/10.3390/sym12020213 Crossref, Google Scholar
15. A. Dutt, M. A. Ismail and T. Herawan , A systematic review on educational data mining, IEEE Access 5 (2017) 15991–16005. https://doi.org/10.1109/access.2017.2654247 Crossref, Google Scholar
16. C. Romero and S. Ventura , Educational data mining: A review of the state of the art, IEEE Trans. Syst. Man Cybern. C, Appl. Rev. 40(6) (2010) 601–618. https://doi.org/10.1109/tsmcc.2010.2053532 Crossref, Google Scholar
17. H. Thi Nguyen, T. M. Thi Pham, T. A. Thi Nguyen, A. V. Thi Tran, P. Vinh Tran and D. Van Pham , Two-stage approach to classifying multidimensional cubes for visualization of multivariate data, in Context-Aware Systems and Applications, and Nature of Computation and Communication, Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering, Vol. 266 (Springer, Cham, 2019), pp. 70–80. https://doi.org/10.1007/978-3-030-06152-4_7 Crossref, Google Scholar
18. D. V. Pham, V. C. Phan and N. H. Do , A new approach for visual analytics applying to multivariate data of student intakes in the university, in Nature of Computation and Communication, Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering, Vol. 473 (Springer Nature, Switzerland, 2023), pp. 88–120. https://doi.org/10.1007/978-3-031-28790-9_7 Crossref, Google Scholar
19. H. Thi Nguyen, T. Xuan Le, P. Vinh Tran and D. Van Pham , An approach of taxonomy of multidimensional cubes representing visually multivariable data, in Context-Aware Systems and Applications, and Nature of Computation and Communication, Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering, Vol. 298 (Springer, Cham, 2019), pp. 90–104. https://doi.org/10.1007/978-3-030-34365-1_8 Crossref, Google Scholar
20. H. Siirtola , The cost of pie charts, 23rd Int. Conf. Information Visualisation, 2–5 July 2019, Paris, France, pp. 151–156. https://doi.org/10.1109/IV.2019.00034 Google Scholar

Vol. 10, No. 04

Metrics

Downloaded 480 times

History

Received 15 August 2022

Revised 30 May 2023

Accepted 24 June 2023

Published: 26 July 2023

Information

This is an Open Access article published by World Scientific Publishing Company. It is distributed under the terms of the Creative Commons Attribution 4.0 (CC BY) License which permits use, distribution and reproduction in any medium, provided the original work is properly cited.

Keywords

PDF download