Browsing by Author "Richhariya, Bharat"

Now showing 1 - 20 of 20

Diagnosis of Alzheimer's disease using universum support vector machine based recursive feature elimination (USVM-RFE)
(Elsevier, 2020-05) Richhariya, Bharat
Alzheimer's disease is one of the most common causes of death in today's world. Magnetic resonance imaging (MRI) provides an efficient and non-invasive approach for diagnosis of Alzheimer's disease. Efficient feature extraction techniques are needed for accurate classification of MRI images. Motivated by the work on support vector machine based recursive feature elimination (SVM-RFE) [16], we propose a novel feature selection technique to incorporate prior information about data distribution in the recursive feature elimination process. Our method is termed as universum support vector machine based recursive feature elimination (USVM-RFE). The proposed method provides global information about data in the RFE process as compared to the local approach of feature selection in SVM-RFE. We also present the application of feature selection and classification algorithms on both voxel based as well as volume based morphometry analysis of structural MRI images (ADNI database). Feature selection is performed using MRI data of brain tissues such as gray matter, white matter, and cerebrospinal fluid. USVM-RFE provides improvement over SVM-RFE in classification of control normal (CN), mild cognitive impairment (MCI), and Alzheimer's disease (AD) subjects. Moreover, better accuracy is obtained by USVM-RFE with lesser number of features in comparison to SVM-RFE. This leads to identification of prominent brain regions for feature selection and classification of MRI images. The highest accuracies obtained by our method for classification of CN vs AD, CN vs MCI, and MCI vs AD are 100%, 90%, and 73.68%, respectively.
EEG signal classification using universum support vector machine
(Elsevier, 2018-09) Richhariya, Bharat
Support vector machine (SVM) has been used widely for classification of electroencephalogram (EEG) signals for the diagnosis of neurological disorders such as epilepsy and sleep disorders. SVM shows good generalization performance for high dimensional data due to its convex optimization problem. The incorporation of prior knowledge about the data leads to a better optimized classifier. Different types of EEG signals provide information about the distribution of EEG data. To include prior information in the classification of EEG signals, we propose a novel machine learning approach based on universum support vector machine (USVM) for classification. In our approach, the universum data points are generated by selecting universum from the EEG dataset itself which are the interictal EEG signals. This removes the effect of outliers on the generation of universum data. Further, to reduce the computation time, we use our approach of universum selection with universum twin support vector machine (UTSVM) which has less computational cost in comparison to traditional SVM. For checking the validity of our proposed methods, we use various feature extraction techniques for different datasets consisting of healthy and seizure signals. Several numerical experiments are performed on the generated datasets and the results of our proposed approach are compared with other baseline methods. Our proposed USVM and proposed UTSVM show better generalization performance compared to SVM, USVM, Twin SVM (TWSVM) and UTSVM. The proposed UTSVM has achieved highest classification accuracy of 99% for the healthy and seizure EEG signals.
An Efficient Angle-based Universum Least Squares Twin Support Vector Machine for Classification
(ACM Digital Library, 2021-08) Richhariya, Bharat
Universum-based support vector machine incorporates prior information about the distribution of data in training of the classifier. This leads to better generalization performance but with increased computation cost. Various twin hyperplane-based models are proposed to reduce the computation cost of universum-based algorithms. In this work, we present an efficient angle-based universum least squares twin support vector machine (AULSTSVM) for classification. This is a novel approach of incorporating universum in the formulation of least squares-based twin SVM model. First, the proposed AULSTSVM constructs a universum hyperplane, which is proximal to universum data points. Then, the classifying hyperplane is constructed by minimizing the angle with the universum hyperplane. This gives prior information about data distribution to the classifier. In addition to the quadratic loss, we introduce linear loss in the optimization problem of the proposed AULSTSVM, which leads to lesser computation cost of the model. Numerical experiments are performed on several benchmark synthetic, real-world, and large-scale datasets. The results show that proposed AULSTSVM performs better than existing algorithms w.r.t. generalization performance as well as computation time. Moreover, an application to Alzheimer’s disease is presented, where AULSTSVM obtains accuracy of 95% for classification of healthy and Alzheimers subjects. The results imply that the proposed AULSTSVM is a better alternative for classification of large-scale datasets and biomedical applications.
Efficient implicit Lagrangian twin parametric insensitive support vector regression via unconstrained minimization problems
(Springer, 2020-11) Richhariya, Bharat
In this paper, an efficient implicit Lagrangian twin parametric insensitive support vector regression is proposed which leads to a pair of unconstrained minimization problems, motivated by the works on twin parametric insensitive support vector regression (Peng: Neurocomputing. 79, 26–38, 2012), and Lagrangian twin support vector regression (Balasundaram and Tanveer: Neural Comput. Applic. 22(1), 257–267, 2013). Since its objective function is strongly convex, piece-wise quadratic and differentiable, it can be solved by gradient-based iterative methods. Notice that its objective function having non-smooth ‘plus’ function, so one can consider either generalized Hessian, or smooth approximation function to replace the ‘plus’ function and further apply the simple Newton-Armijo step size algorithm. These algorithms can be easily implemented in MATLAB and do not require any optimization toolbox. The advantage of this method is that proposed algorithms take less training time and can deal with data having heteroscedastic noise structure. To demonstrate the effectiveness of the proposed method, computational results are obtained on synthetic and real-world datasets which clearly show comparable generalization performance and improved learning speed in accordance with support vector regression, twin support vector regression, and twin parametric insensitive support vector regression.
Enhancing class imbalance solutions: A projection-based fuzzy LS-TSVM approach
(Elsevier, 2024-07) Richhariya, Bharat
Class imbalance and noise present significant challenges in numerous real-world classification tasks. The prevalence of an uneven distribution of samples typically results in a bias towards the majority class in Support Vector Machine (SVM) classifiers, compounded by the often inherent noise within these samples. Addressing both class imbalance and noise, we introduce two fuzzy-based methodologies. The first method employs intuitionistic fuzzy membership, resulting in the development of the Robust Energy-based Intuitionistic Fuzzy Least Squares Twin Support Vector Machine (IF-RELSTSVM), a model specifically designed for class imbalance learning. The IF-RELSTSVM model is distinguished by its use of intuitionistic fuzzy scores for both classes, significantly attenuating the detrimental effects of noise and outliers. A distinctive attribute of IF-RELSTSVM is its proficiency in processing noisy data points, whether proximate to or distant from the hyperplane. Additionally, we introduce a novel concept of hyperplane-based fuzzy membership, calculating fuzzy memberships through a projection-based approach. This foundation supports the formulation of a Robust Energy-based Fuzzy Least Squares Twin Support Vector Machine (F-RELSTSVM), also aimed at class imbalance learning. The efficacy of the proposed IF-RELSTSVM and F-RELSTSVM algorithms is rigorously evaluated across several benchmark and synthetic datasets, employing the Area Under the ROC Curve (AUC) as a performance metric. Experimental findings indicate that these algorithms surpass baseline models in the majority of datasets tested. Statistical analyses further validate the significance of our proposed methods, demonstrating their suitability for application in environments characterized by noise and class imbalance. A case study in credit card fraud detection showcases the F-RELSTSVM algorithm achieving an impressive average AUC of 90.84%, thereby outperforming comparable algorithms and highlighting the practical applicability of our approaches in tackling challenging datasets.
Entropy based fuzzy least squares twin support vector machine for class imbalance learning
(Springer, 2018-06) Richhariya, Bharat
In classification problems, the data samples belonging to different classes have different number of samples. Sometimes, the imbalance in the number of samples of each class is very high and the interest is to classify the samples belonging to the minority class. Support vector machine (SVM) is one of the widely used techniques for classification problems which have been applied for solving this problem by using fuzzy based approach. In this paper, motivated by the work of Fan et al. (Knowledge-Based Systems 115: 87–99 2017), we have proposed two efficient variants of entropy based fuzzy SVM (EFSVM). By considering the fuzzy membership value for each sample, we have proposed an entropy based fuzzy least squares support vector machine (EFLSSVM-CIL) and entropy based fuzzy least squares twin support vector machine (EFLSTWSVM-CIL) for class imbalanced datasets where fuzzy membership values are assigned based on entropy values of samples. It solves a system of linear equations as compared to the quadratic programming problem (QPP) as in EFSVM. The least square versions of the entropy based SVM are faster than EFSVM and give higher generalization performance which shows its applicability and efficiency. Experiments are performed on various real world class imbalanced datasets and compared the results of proposed methods with new fuzzy twin support vector machine for pattern classification (NFTWSVM), entropy based fuzzy support vector machine (EFSVM), fuzzy twin support vector machine (FTWSVM) and twin support vector machine (TWSVM) which clearly illustrate the superiority of the proposed EFLSTWSVM-CIL
Facial expression recognition using iterative universum twin support vector machine
(Elsevier, 2019-03) Richhariya, Bharat
Facial expressions are one of the most important characteristics of human behaviour. They are very useful in applications on human computer interaction. To classify facial emotions, different feature extraction methods are used with machine learning techniques. In supervised learning, information about the distribution of data is given by data points not belonging to any of the classes. These data points are known as universum data. In this work, we use universum data to perform multiclass classification of facial emotions from human facial images. Moreover, the existing universum based models suffer from the drawback of high training cost, so we propose an iterative universum twin support vector machine (IUTWSVM) using Newton method. Our IUTWSVM gives good generalization performance with less computation cost. To solve the optimization problem of proposed IUTWSVM, no optimization toolbox is required. Further, improper selection of universum points always leads to degraded performance of the model. For generating better universum, a novel scheme is proposed in this work based on information entropy of data. To check the effectiveness of proposed IUTWSVM, several numerical experiments are performed on benchmark real world datasets. For multiclass classification of facial emotions, the performance of IUTWSVM is compared with existing algorithms using different feature extraction techniques. Our proposed algorithm shows better generalization performance with less training cost in both binary as well as multiclass classification problems.
A fuzzy twin support vector machine based on information entropy for class imbalance learning
(Springer, 2018-05) Richhariya, Bharat
In real-world binary class datasets, the total number of samples may not be the same in both the classes, i.e. size of the majority class is much larger than minority class which is called as imbalance problem. In various classification problems, the main interest is to correctly classify the samples belonging to the minority class. Since support vector machine (SVM) and twin support vector machine (TWSVM) obtain the resultant classifier by giving same importance to all the training samples, it results in a biased classifier towards the majority class in imbalanced datasets. In this paper, by considering the fuzzy membership value for each sample, we have proposed an efficient approach, entropy-based fuzzy twin support vector machine for class imbalanced datasets (EFTWSVM-CIL) where fuzzy membership values are assigned based on the entropy values of samples. Here, we give more importance to the minority class by assigning relatively larger fuzzy memberships to the minority class samples. Further, it solves a pair of smaller-size quadratic programming problems (QPPs) rather than a large one as in the case of SVM. Experiments are performed on various real-world imbalanced datasets, and results of our proposed EFTWSVM-CIL are compared with twin support vector machine (TWSVM), fuzzy twin support vector machine (FTWSVM) and entropy-based fuzzy SVM (EFSVM) for imbalanced datasets.
A fuzzy universum least squares twin support vector machine (FULSTSVM)
(Springer, 2021-01) Richhariya, Bharat
Universum based twin support vector machines give prior information about the distribution of data to the classifier. This leads to better generalization performance of the model, due to the universum. However, in many applications the data points are not equally useful for the classification task. This leads to the use of fuzzy membership functions for the datasets. Similarly, in universum based algorithms, all the universum data points are not equally important for the classifier. To solve these problems, a novel fuzzy universum least squares twin support vector machine (FULSTSVM) is proposed in this work. In FULSTSVM, the membership values are used to provide weights for the data samples of the classes, as well as to the universum data. Further, the optimization problem of proposed FULSTSVM is obtained by solving a system of linear equations. This leads to an efficient fuzzy based algorithm. Numerical experiments are performed on various benchmark datasets, with discussions on generalization performance, and computational cost of the algorithms. The proposed FULSTSVM outperformed the existing algorithms on most datasets. A comparison is presented for the performance of the proposed and other baseline algorithms using statistical significance tests. To show the applicability of FULSTSVM, applications are also presented, such as detection of Alzheimer’s disease, and breast cancer.
A Fuzzy Universum Support Vector Machine Based on Information Entropy
(Springer, 2018-08) Richhariya, Bharat
Universum-based support vector machines (USVMs) are known to give better generalization performance than standard SVM methods by incorporating prior information about the data. In datasets involving noise and outliers, this universum-based scheme is not so effective because the generated universum data points do not lie in between the two classes. In this paper, we propose a fuzzy universum support vector machine (FUSVM) by introducing the weights to the universum data points based on their information entropy. Since there is no standard approach of selecting the universum, our information entropy based approach is helpful in giving less weight to the outlier universum points and thus gives prior information about the data in an appropriate manner. In addition, we also propose a fuzzy-based approach for universum twin support vector machine named as fuzzy universum twin support vector machine (FUTSVM). Experimental results on several benchmark datasets indicate that, comparing to SVM, USVM, TWSVM and UTSVM our proposed FUSVM and FUTSVM have shown better generalization performance
Improved universum twin support vector machine
(IEEE, 2018) Richhariya, Bharat
Universum based learning provides prior information about data in the optimization problem of support vector machine (SVM). Universum twin support vector machine (UTSVM) is a computationally efficient algorithm for classification problems. It solves a pair of quadratic programming problems (QPPs) to obtain the classifier. In order to include the structural risk minimization (SRM) principle in the formulation of UTSVM, we propose an improved universum twin support vector machine (IUTSVM). Our proposed IUTSVM implicitly makes the matrices non-singular in the optimization problem by adding a regularization term. Several numerical experiments are performed on benchmark real world datasets to verify the efficacy of our proposed IUTSVM. The experimental results justifies the better generalization performance of our proposed IUTSVM in comparison to existing algorithms.
Lagrangian twin parametric insensitive support vector regression (LTPISVR)
(Springer, 2019-03) Richhariya, Bharat
In this paper, motivated by the works on twin parametric insensitive support vector regression (TPISVR) (Peng in Neurocomputing 79(1):26–38, 2012), and Lagrangian twin support vector regression (Balasundaram and Tanveer in Neural Comput Appl 22(1):257–267, 2013), a new efficient approach is proposed as Lagrangian twin parametric insensitive support vector regression (LTPISVR). In order to make the objective function strongly convex, we consider square of 2-norm of slack variables in the optimization problem. To reduce the computation cost, the solution of proposed LTPISVR is obtained by solving simple linearly convergent iterative schemes, instead of quadratic programming problems as in TPISVR. There is no requirement of any optimization toolbox for proposed LTPISVR. To demonstrate the effectiveness of proposed method, we present numerical results on well-known synthetic and real-world datasets. The results clearly show similar or better generalization performance of proposed method with lesser training time in comparison with support vector regression, twin support vector regression and twin parametric insensitive support vector regression.
Least squares projection twin support vector clustering (LSPTSVC)
(Elsevier, 2020-09) Richhariya, Bharat
Clustering is a prominent unsupervised learning technique. In the literature, many plane based clustering algorithms are proposed, such as the twin support vector clustering (TWSVC) algorithm. In this work, we propose an alternative algorithm based on projection axes termed as least squares projection twin support vector clustering (LSPTSVC). The proposed LSPTSVC finds projection axis for every cluster in a manner that minimizes the within class scatter, and keeps the clusters of other classes far away. To solve the optimization problem, the concave-convex procedure (CCCP) is utilized in the proposed method. Moreover, the solution of proposed LSPTSVC involves a set of linear equations leading to very less training time. To verify the performance of the proposed algorithm, several experiments are performed on synthetic and real world benchmark datasets. Experimental results and statistical analysis show that the proposed LSPTSVC performs better than existing algorithms w.r.t. clustering accuracy as well as training time. Moreover, a comparison of the proposed method with existing algorithms is presented on biometric and biomedical applications. Better generalization performance is achieved by proposed LSPTSVC on clustering of facial images, and Alzheimer’s disease data.
Machine Learning Techniques for the Diagnosis of Alzheimer’s Disease: A Review
(ACM Digital Library, 2020-04) Richhariya, Bharat
Alzheimer’s disease is an incurable neurodegenerative disease primarily affecting the elderly population. Efficient automated techniques are needed for early diagnosis of Alzheimer’s. Many novel approaches are proposed by researchers for classification of Alzheimer’s disease. However, to develop more efficient learning techniques, better understanding of the work done on Alzheimer’s is needed. Here, we provide a review on 165 papers from 2005 to 2019, using various feature extraction and machine learning techniques. The machine learning techniques are surveyed under three main categories: support vector machine (SVM), artificial neural network (ANN), and deep learning (DL) and ensemble methods. We present a detailed review on these three approaches for Alzheimer’s with possible future directions.
Minimum variance-embedded deep kernel regularized least squares method for one-class classification and its applications to biomedical data
(Elsevier, 2020-03) Richhariya, Bharat
Deep kernel learning has been well explored for multi-class classification tasks; however, relatively less work is done for one-class classification (OCC). OCC needs samples from only one class to train the model. Most recently, kernel regularized least squares (KRL) method-based deep architecture is developed for the OCC task. This paper introduces a novel extension of this method by embedding minimum variance information within this architecture. This embedding improves the generalization capability of the classifier by reducing the intra-class variance. In contrast to traditional deep learning methods, this method can effectively work with small-size datasets. We conduct a comprehensive set of experiments on 18 benchmark datasets (13 biomedical and 5 other datasets) to demonstrate the performance of the proposed classifier. We compare the results with 16 state-of-the-art one-class classifiers. Further, we also test our method for 2 real-world biomedical datasets viz.; detection of Alzheimer’s disease from structural magnetic resonance imaging data and detection of breast cancer from histopathological images. Proposed method exhibits more than 5% score compared to existing state-of-the-art methods for various biomedical benchmark datasets. This makes it viable for application in biomedical fields where relatively less amount of data is available.
A reduced universum twin support vector machine for class imbalance learning
(Elsevier, 2020-06) Richhariya, Bharat
In most of the real world datasets, there is an imbalance in the number of samples belonging to different classes. Various pattern classification problems such as fault or disease detection involve class imbalanced data. The support vector machine (SVM) classifier becomes biased towards the majority class due to class imbalance. Moreover, in the existing SVM based techniques for class imbalance, there is no information about the distribution of data. Motivated by the idea of prior information about data distribution, a reduced universum twin support vector machine for class imbalance learning (RUTSVM-CIL) is proposed in this paper. For the first time, universum learning is incorporated with SVM to solve the problem of class imbalance. Oversampling and undersampling of data is performed to remove the imbalance in the classes. The universum data points are used to give prior information about the data. To reduce the computation time of our universum based algorithm, we use a small sized rectangular kernel matrix. The reduced kernel matrix needs less storage space, and thus applicable for large scale imbalanced datasets. Comprehensive experimentation is performed on various synthetic, real world and large scale imbalanced datasets. In comparison to the existing approaches for class imbalance, the proposed RUTSVM-CIL gives better generalization performance for most of the benchmark datasets. Also, the computation cost of RUTSVM-CIL is very less, making it suitable for real world applications.
A robust fuzzy least squares twin support vector machine for class imbalance learning
(Elsevier, 2018-10) Richhariya, Bharat
Twin support vector machine is one of the most prominent techniques for classification problems. It has been applied in various real world applications due to its less computational complexity. In most of the applications on classification, there is imbalance in the number of samples of the classes which leads to incorrect classification of the data points of the minority class. Further, while dealing with imbalanced data, noise poses a major challenge in various applications. To resolve these problems, in this paper we propose a robust fuzzy least squares twin support vector machine for class imbalance learning termed as RFLSTSVM-CIL using 2-norm of the slack variables which makes the optimization problem strongly convex. In order to reduce the effect of outliers, we propose a novel fuzzy membership function specifically for class imbalance problems. Our proposed function gives the appropriate weights to the datasets and also incorporates the knowledge about the imbalance ratio of the data. In our proposed model, a pair of system of linear equations is solved instead of solving a quadratic programming problem (QPP) which makes our model efficient in terms of computation complexity. To check the performance of our proposed approach, several numerical experiments are performed on synthetic and real world benchmark datasets. Our proposed model RFLSTSVM-CIL has shown better generalization performance in comparison to the existing methods in terms of AUC and training time
Sparse Twin Support Vector Clustering Using Pinball Loss
(IEEE, 2021-10) Richhariya, Bharat
Clustering is a widely used machine learning technique for unlabelled data. One of the recently proposed techniques is the twin support vector clustering (TWSVC) algorithm. The idea of TWSVC is to generate hyperplanes for each cluster. TWSVC utilizes the hinge loss function to penalize the misclassification. However, the hinge loss relies on shortest distance between different clusters, and is unstable for noise-corrupted datasets, and for re-sampling. In this paper, we propose a novel Sparse Pinball loss Twin Support Vector Clustering (SPTSVC). The proposed SPTSVC involves the ϵ -insensitive pinball loss function to formulate a sparse solution. Pinball loss function provides noise-insensitivity and re-sampling stability. The ϵ -insensitive zone provides sparsity to the model and improves testing time. Numerical experiments on synthetic as well as real world benchmark datasets are performed to show the efficacy of the proposed model. An analysis on the sparsity of various clustering algorithms is presented in this work. In order to show the feasibility and applicability of the proposed SPTSVC on biomedical data, experiments have been performed on epilepsy and breast cancer datasets.
An unconstrained primal based twin parametric insensitive support vector regression
(World Scientific, 2025) Richhariya, Bharat
In this paper, we propose an efficient regression algorithm based on primal formulation of twin support vector machine. This is an efficient approach to solve the optimization problem leading to reduced computation time. The proposed method is termed as twin parametric insensitive support vector regression (UPTPISVR). The optimization problems of the proposed (UPTPISVR) are a pair of unconstrained convex minimization problems. Moreover, the objective functions of UPTPISVR are strongly convex, differentiable and piecewise quadratic. Therefore, an approximate solution is obtained in primal variables instead of solving the dual formulation. Further, an absolute value equation problem is solved by using a functional iterative algorithm for UPTPISVR, termed as FUPTPISVR. The objective function of the proposed formulation involves the plus function which is non-smooth and therefore, smooth approximation functions are used to replace the plus function, termed as SUPTPISVR. The Newton-Armijo algorithm is then used to iteratively obtain the solutions, thus eliminates the requirement of any optimization toolbox. Various numerical experiments on synthetic and benchmark real-world datasets are presented for justifying the applicability and effectiveness of the proposed UPTPISVR. The results clearly indicate that the proposed algorithms outperform the existing algorithms in terms of root mean square error (RMSE) on most datasets.
Universum least squares twin parametric-margin support vector machine
(IEEE, 2020-07) Richhariya, Bharat
Universum based algorithms involve universum samples in the classification problem to improve the generalization performance. In order to provide prior information about data, we utilized universum data to propose a novel classification algorithm. In this paper, a novel parametric model for universum based twin support vector machine is presented for classification problems. The proposed model is termed as universum least squares twin parametric-margin support vector machine (ULSTPMSVM). The solution of ULSTPMSVM involves a system of linear equations. This makes the ULSTPMSVM efficient w.r.t. training time. In order to verify the performance of the proposed model, various experiments are carried out on real world benchmark datasets. Statistical tests are performed to verify the significance of the proposed method. The proposed ULSTPMSVM performed better than existing algorithms in terms of classification accuracy and training time for most of the datasets. Moreover, an application of proposed ULSTPMSVM is presented for classification of Alzheimer's disease data.