A NOVEL ALGORITHM FOR IMPROVED TRACTABILITY AND EXPLAINABILITY OF INFERENCES ON HIGH-DIMENSIONAL DATA USING CONDITIONED ATTENTION MECHANISM

ASHUZA, Kuderha Rifin and Covenant University, Theses Masters (2024) A NOVEL ALGORITHM FOR IMPROVED TRACTABILITY AND EXPLAINABILITY OF INFERENCES ON HIGH-DIMENSIONAL DATA USING CONDITIONED ATTENTION MECHANISM. Masters thesis, Covenant University.

PDF Download (368kB)

Abstract

The availability of a large amount of data that can be mined is a challenge for data mining techniques that cannot keep up with the exponentially increasing quantity of data. Most of the classical techniques have been reported inefficient to handle large amount of data. The aim of this work is to develop a novel attention mechanism for high-dimensional tabular data understanding for knowledge representation that can enhance tractability of decision rule learning. The conditioning theory was mathematically formalized as an attention mechanism that models an unconditioned response (or reflex response) to unconditioned stimulus by pairing conditioned stimulus with unconditioned stimulus. The formalization of the conditioning theory built was translated into an algorithm presented as conditioned attention mechanism and applied on input tabular dataset producing vector embeddings of attribute-value pairs. The learned representation of attribute-value pairs as vector embeddings are then exploited to build decision rules using a developed novel Coupling-Sorting-Branching technique (CoSoBra), formalized as energy-based model optimization. A Python-based compression algorithm was developed to reduce the dimension of the resulting very large vector embeddings while maintaining the quality of the attribute-value pairs representations. The developed novel Conditioned Attention High Dimensional Rule Learning (CAHDRL) algorithm, which captures both CoSoBra algorithm and Condition Attention mechanism was benchmarked against existing approaches for decision rule learning on ten datasets from six different domains. The efficacy of the CAHDRL approach was evaluated with respect to the effectiveness of the extracted decision rules and its efficiency during data processing. An ablation study was carried out to assess the efficiency CAHDRL. Results of the evaluation on the 8 first datasets showed that CADHRL had higher accuracy of 0.85 in average against 0.8175 in average for the EAV-60 model, equal length of decision rule of 2.125 in average against 2.125 in average for the reduct-alpha-0.1 model and a better support of 392.5263 in average against 350.2663 in average for the reduct-alpha-0.1. t-distributed Stochastic Neighborhood Embedding (t-SNE) was used to visualize the embeddings on a 2D manifold. The results support literature findings on Mushroom dataset that “odor=creosote” pair is a good descriptor of the poisonous mushroom. The second visualization results using “Adult” dataset showed that USA natives are more likely to earn $50,000.00 and above. Wilcoxon test of significance of efficacy results for CAHDRL validated its relatively superior performance against existing approaches, yielding values of p-values inferior to 0.05 for accuracy (3 of the seven comparisons), length of decision rules (6 of the seven comparisons) and support (4 of the five comparisons) respectively. CAHDRL can effectively find embedding vectors of attribute-value pairs and efficiently extract decision rules from diverse datasets for classification tasks, with a higher tractability and inherent explainability. CAHDRL also demonstrates a great scalability feature that allows for the efficient processing very high dimensional datasets by the parallelizable nature of CAHDRL.

Item Type:	Thesis (Masters)
Uncontrolled Keywords:	attention mechanism, decision rule, explainability, tractability, machine learning
Subjects:	Q Science > QA Mathematics > QA75 Electronic computers. Computer science Q Science > QA Mathematics > QA76 Computer software
Divisions:	Faculty of Engineering, Science and Mathematics > School of Electronics and Computer Science
Depositing User:	Patricia Nwokealisi
Date Deposited:	30 Sep 2024 09:02
Last Modified:	30 Sep 2024 09:02
URI:	http://eprints.covenantuniversity.edu.ng/id/eprint/18462

Actions (login required)

View Item