A Hybrid Meta-Learner Technique for Credit Scoring of Banks’ Customers

—Financial institutions are exposed to credit risk due to issuance of consumer loans. Thus, developing reliable credit scoring systems is very crucial for them. Since, machine learning techniques have demonstrated their applicability and merit, they have been extensively used in credit scoring literature. Recent studies concentrating on hybrid models through merging various machine learning algorithms have revealed compelling results. There are two types of hybridization methods namely traditional and ensemble methods. This study combines both of them and comes up with a hybrid meta-learner model. The structure of the model is based on the traditional hybrid model of ‘classification + clustering’ in which the stacking ensemble method is employed in the classification part. Moreover, this paper compares several versions of the proposed hybrid model by using various combinations of classification and clustering algorithms. Hence, it helps us to identify which hybrid model can achieve the best performance for credit scoring purposes. Using four real-life credit datasets, the experimental results show that the model of (KNN-NN-SVMPSO)-(DL)-(DBSCAN) delivers the highest prediction accuracy and the lowest error rates.


INTRODUCTION
Owing to the recent global financial crisis and European sovereign debt crisis, credit risk assessment has turn out to be an increasingly vital issue for banks and credit institutions throughout the world.Also, the sharp competition in financial sector has caused a large decline in banking profit.This leads banks toward more consumer loans to make higher interest profits.However, the expected profitability depends on the quality of consumer loans issued by the banks, which requires a vigilant credit scoring process.It is worthwhile to mention that even 1% enhancement on the accuracy of credit scoring system would significantly increase the profit of banks and other financial institutions [1].Traditionally, credit decisions were made by human experts based on past experiences, historical performances, and some guidelines specially the classic five C's of credit: character, capacity, capital, collateral and conditions [2].But this approach suffers from some drawbacks including inconsistent decisions, repeated incorrect decisions, and high training costs.Therefore, with the quick development in credit industry, various credit scoring techniques are being used for the credit evaluation.The credit scoring models have been developing at a fast pace to distinguish bad credit applicants from good ones through their associated features such as gender, age, education, income, job and marital status or based on their historical credit performance.The advantages of credit scoring models can be enumerated as cost reduction of credit analysis, faster credit decisions, higher rate of credit collections, efficient performance monitoring of the model, mitigating possible risks and changes in economic conditions or policies can simply be integrated into the model [3][4][5].Even a minor betterment in the accuracy of credit scoring models may diminish a significant amount of credit risks and generate noteworthy future savings.Due to both the impacts of financial crisis and soaring risk appetite, the number of non-performing loans has sharply intensified along with banks giving more credits to applicants without sufficient assessments.Thus, the use of efficient credit scoring models seems to be inevitable for the banks and other credit institutions.There are several approaches employed by financial institutions over the past decades to model the credit risk which are mainly classified into two groups of statistical and Artificial Intelligence (AI) techniques.Generally, the statistical methods include Logistic Regression (LR) and Linear Discriminant Analysis (LDA).On the other hand, AI approaches mainly comprise of machine learning techniques such as Support Vector Machines (SVM), Artificial Neural Networks (ANN), Decision Trees (DT) and so many other machine learning (classification and clustering) algorithms.There are some pros and cons associated with these methods.For instance, LDA assumes a normal distribution of the variables and a linear relationship between explanatory variables but it is unable to verify fulfillment of these assumptions [4,5].LR is used for forecasting on a dataset with binary outcomes.Although, the normality assumption is not required by LR, but linear relationship among variables is a basic assumption for both models.Therefore, some researchers [4][5][6][7] are having doubts about predictive performance of these models for credit scoring.In contrast, artificial intelligence techniques recently draw attention from many scholars for coping with credit scoring problems.These techniques are best known for their higher predictive accuracy compared to statistical models and usually do not require abovementioned assumptions.For example, ANN which simulates the human brain's mechanism on the computer environment does not need any assumptions and in the field of credit scoring, it performs much better than its classical rivals including LR and LDA [8][9][10][11][12][13].In general, it can be said that AI methods are superior to traditional ones [14][15][16].In recent years, many researchers have focused on the development of machine learning techniques for credit scoring applications.One of the methods that they are using to improve the performance machine learning algorithms is the hybridization.These researchers believe that the credit scoring models which are built by combing classification (supervised learning) and clustering (unsupervised learning) techniques have the ability to outperform sole machine learning methods [12,[17][18][19][20].
In this study, a new hybrid method is introduced for credit scoring which is based on a combination of traditional hybrid and stacking ensemble methods.The idea comes from the traditional hybrid model of classification plus clustering.This is because clustering is considered as an unsupervised learning method and it cannot differentiate data precisely like supervised methods.Accordingly, a classifier or set of classifiers can be trained first, and then its output is used as the input for the clustering method to enhance the clustering outcomes [21].In this model, instead of using a single classification algorithm in the first part of the hybrid model, we adopt a stacking ensemble method and in the second part several clustering algorithms will be interchangeably used.Also, this model benefits from a deep learning algorithm as the meta-learner classifier.It is believed that the superior learning capacity of deep learning can improve the predictive accuracy of the new hybrid credit scoring model.This study has chosen various types of classifiers and clusterers to be used in this hybrid model.In the relevant literature, many studies have developed hybrid credit rating models only by choosing single learning algorithms as the baselines (traditional hybrid models) but this study adopts the stacking ensemble method as the baseline of the hybrid model.Moreover, this paper tries to compare several versions of the proposed hybrid model by using various combinations of classification and clustering algorithms.Thus, it helps us to identify which hybrid model can achieve the best prediction accuracy for credit scoring purposes.The structure of the paper is as follows: Section II reviews the literature in terms of different hybrid and ensemble credit scoring models.Section III explains the data and methodology of the study and Section IV presents the experimental results and analysis.Finally, Section V concludes the study and discusses future work opportunities.

II. LITERATURE REVIEW
In this section, the literature of machine learning in the field of credit scoring will be reviewed.When banks want to grant credit to their customers, they evaluate their credit.Through adopting a good credit scoring system, banks can classify their customers in terms of risk (probability of default), so offer them risk adjusted loans with different interest rates and collateral conditions.Therefore, optimal credit decisions can be made based on the outputs of the credit scoring models.Since the emergence of AI systems like neural networks, genetic algorithm and expert systems, these methods have been increasingly used in financial researches and also implemented by many financial institutions specially banks.Authors in [22] adopt the four different types of traditional hybrid machine learning techniques to identify which method can achieve the best predictive results.They combine different classification and clustering algorithms such as Naïve Bayesian, Decision Tress, Logistic regression, Neural Network, K-means and Expectation Maximization.Then, they apply these hybrid models on a real credit dataset from Taiwan.Comparative results show that the "classification + classification" hybrid model outperforms the other hybrid models.This model utilizes Logistic regression and Neural Network as the first and second classifiers (LR + NN), respectively.They state that these hybrid credit scoring models can help financial institutions make more correct decisions for issuing consumer loans with high confidence in the future.Authors in [23] study the behavior of imbalanced credit scoring datasets by different machine learning methods.Data imbalances take place when the number of defaulting customers in a dataset is typically much lower than the number of non-defaulting ones.They tested various models on five real-world credit datasets.Finally, they showed that when datasets are imbalanced, machine learning methods like decision trees, KNN, linear discriminant analysis (LDA) do no perform well.On the other hand, models such as gradient boosting and random forests have much better predictive performance.Although, decision trees (DT) is one of the most popular algorithms used in machine learning and credit scoring, it suffers from two drawbacks: 1) it's very sensitive to noise and 2) redundant features may falsify the learning process.Hence, authors in [24] suggest two ensemble methods namely Bagging-RS DT and RS-Bagging DT to deal with these problems.In these models they adopt Random Subspace (RS) and Bootstrap Aggregating (Bagging) strategies.They test these two models on Australian and German credit datasets and results show that these two models perform better than other base models.
Author in [25] introduces a new solution for credit scoring problems which is based on a modified version of SVM.He mentions that since most of real credit datasets are pretty big, the use of conventional nonlinear SVMs even with high levels accuracy are computationally suboptimal.Consequently, he proposes a clustered support vector machine (CSVM) to cope with this problem.He concludes that the CSVM, despite gaining similar prediction performance, can stay relatively cheap from computational point of view.In another attempt to create an optimal credit scoring model, authors in [26] have proposed the Ensemble Classification based Supervised Clustering (ECSC) method.The main idea behind this model is that data samples from the identical class might have dissimilar characteristics or patterns.By means of supervised clustering, samples with similar characteristics or patterns are categorized into the same cluster.Hence, the training subsets, formed by mixture of clusters from diverse classes, could well express various patterns of samples, which is beneficial to enhance the variety and accuracy of base classifiers.In this paper they use base classifiers such as logistic regression, decision trees, SVM and also K-means for supervised clustering.They have applied this model along with random subspace bagging (RS-Bagging), Bagging-RS, and dynamic classifier ensemble using classification confidence (DCE-CC) on German and Australian credit datasets.Results show the ECSC is relatively more accurate than other models.
One of the heuristic methods used for credit scoring is fuzzy SVM based on Support Vector Data Description (SVDD) which is introduced in [27].SVDD is based on the SVM classifier, which looks for a spherical-shaped border around a dataset to identify outliers or unique data.This approach uses SVDD to mitigate the impact of outliers and noisy data in order to improve the Fuzzy SVM learning rate.Authors adopted this model to test against the ordinary linear and nonlinear fuzzy SVM on the Australian and German credit datasets.What can be drawn as a conclusion is that although the best result is obtained by the SVDD-FSVM its superiority is negligible.In a recent study, authors in [28] introduced a model based on the combination of hybrid and ensemble methods.They believe that merging filtering and feature selection methods can perform as an effective pre-processor for machine learning models.For this reason, they have combined Multivariate Adaptive Regression Splines (MARS) and Gabriel Neighborhood Graph editing (GNG) in the hybrid modeling stage.As base classifiers, they have selected decision trees, ANN, random forests, Bayesian network and SVM.They have applied these models on seven real world credit datasets.Results illustrate that the authors' proposed model relatively improves the predictive performance relative compared with base learners.

III. METHODOLOGY
This section describes the procedure of developing the credit scoring system introduced by this study.Generally, there are two ways to establish a hybrid machine learning model which are traditional and ensemble methods.The traditional hybridization method offers four different ways to combine machine learning algorithms.These options are (1) merging two classification algorithms, (2) merging one classification algorithm with one clustering algorithm, (3) merging one clustering algorithm with one classification algorithm, and (4) merging two clustering algorithms [22,29].On the other hand, ensemble methods offer sophisticated ways of hybridizing machine learning techniques.Employing ensembles is beneficial as they can overcome the three problems of base learning algorithms namely statistical, computational, and representational problems [30].When the size of a dataset is too small compared with the potential space of hypotheses, a learning algorithm may select to yield a hypothesis from a group having the equal predictive accuracy on the training data.Thus, the statistical problem emerges in such cases if the selected hypothesis is unable to forecast new data.When a learning algorithm is trapped in an incorrect local minimum rather than finding the best hypothesis within the hypotheses space, the computational problem will arise.Lastly, the representational problem occurs when no hypothesis inside the hypotheses space is a good estimate to the correct function [31].There are several forms of ensembles including bagging, boosting, and stacking.These techniques are frequently used in the literature of machine learning and credit scoring.Findings suggest that ensemble methods usually achieve superior predictive performance compared to other single algorithms or traditional hybrid models [26,28,[32][33][34][35][36].Unlike the bagging and boosting which are used in many papers, few researchers have employed the Stacking method.Stacking (stacked generalization) is designed to enhance predictive performance through combining the predictions of several machine learning algorithms [37].It consists of training a combiner algorithm to amalgamate the predictions of various learning algorithms.First, an ensemble of classifiers (base classifiers) is trained using the available data via bootstrapped sampling (Tier 1 classifiers).Then the output of base classifiers are used as an input to train a meta-classifier (Tier 2 classifier) [36].In other words, stacking trains a set of classifiers parallelly and then learning is done by a meta-learner.Author in [38] emphasizes that classifiers which are functioning in a collaborative way can significantly outpace those working separately, showing the importance of using such a model.
The meta-learner (meta-classifier) in the stacking algorithm generates a vector of weight distribution by assigning a weight to each base classifier that is proportional to their performances [31].Stacking can be considered as a fully customizable hybrid machine learning system as it hosts various types of base-and meta-classifiers.Also, it has been successfully employed on both supervised and unsupervised learning tasks [39][40][41].In a recent study [32] is showed that a hybrid ensemble machine learning system with stacking is superior to other types of ensemble methods.The proposed hybrid meta-learner model is built based on the combination of traditional hybrid and ensemble modeling of credit scoring systems.The foundation of the model is based on the traditional hybrid model of "classification+clustering" which uses a classification technique as a pre-processor for the clustering algorithm.The only difference is that this paper adopts the stacking ensemble method in the first part instead of using a single classification method.Also, several classification and clustering techniques are used in this study which are briefly described in following sub sections.

B. Clustering Techniques
Clustering (or unsupervised learning) methods can be viewed as the way toward combining similar examples into a cluster.Unlike the classification, labeled examples are not available in clustering.The main aim of clustering approach is to increase the resemblance between the group members.Moreover, the data between different clusters should have the highest dissimilarity.Conversely, the highest data similarity should exist within each cluster [42].There are two categories for clustering algorithms, which are partitional and hierarchical clustering algorithms but the former is much more popular [43].Partitional clustering has been widely implemented in many credit scoring problems.K-means and expectation maximization are two renowned partitional clustering algorithms.In contrast, hierarchical clustering generates clusters according to a hierarchy by means of the accumulation algorithm.Then, a different single cluster will be merged individually until fulfilling some rules.The outcome will create a series of branching partitions.This study uses five clustering algorithms namely Expectation Maximization (EM), K-means (KM), Fuzzy C-means (FCM), Density-based spatial clustering of applications with noise (DBSCAN), and Self-organizing Maps (SOM).

C. The Hybrid Meta-Learner Model
This study introduces a new hybrid method for credit scoring which is a mixture of traditional hybrid and stacking ensemble methods.The idea comes from the traditional hybrid model of "classification plus clustering" due to the fact that clustering is an unsupervised learning method and it is unable to distinguish data accurately like supervised approaches.Therefore, a classifier or set of classifiers can be trained first, and then the output can be used as the input for the clustering technique to improve the clustering results.In this process, instead of using a single classification algorithm in the first part of the hybrid model, a stacking ensemble method will be used.This stacking model utilizes three different base classifiers (level 0 generalizers) to train the meta-classifier (level 1 generalizer).In the second part, several clustering techniques will be interchangeably used in order to find which combination of algorithms yield the best results.One of the advantages on this hybrid system is placing a deep learning algorithm (DL) in the heart of the proposed model as the metalearner.Owing to the great learning ability of DL, the predictive performance of the model is expected to improve significantly.For the first time, the concept of DL is proposed in 2006.This algorithm is defined in the framework of deep belief networks (DBN).Later, DL has caused considerable amount of scientific researches in several fields [45][46][47].This algorithm as a feature selection technique, tries to get feature abstractions at the high-levels through learning various feature structures in the training process.Every DL iteration is an unsupervised learning process for feature extraction, and the mix of different layers has the ability to create a deep supervised predictor [48].DL has various theoretical frameworks, but this study utilizes the H2O version, which is based on the feedforward architecture.As shown in the Figure 1, the main part of the DL model is the neuron which is inspired by the human neural system.In this model, the weighted mix of input signals (α) is combined, and then an output signal f(α) conveyed by the connected neuron.The nonlinear activation function and neuron's activation threshold (bias) are denoted by f and b, respectively [49].The weights which are connecting neurons and biases with other neurons define the output of the whole network.The error on the labeled training data should be minimized through weight adjustment procedure in order to make learning possible.Specifically, the aim is to minimize the loss function of L(W; B | j) for every training example j [49].DL as the meta-learner in the stacking algorithm creates a vector of weight distribution by giving a weight to each base classifier that is proportional to their performances.Stacking can be seen as a completely customizable hybrid machine learning technique since it embraces various types of base-and metaclassifiers.This study employs different types of classifiers and clusterers to be used in this hybrid model.In the literature, many works have been done by developing hybrid credit rating models only by choosing single learning algorithms as the baselines but this study adopts the stacking ensemble method as the baseline of the hybrid model.Furthermore, the aim is to compare several versions of the proposed hybrid model by selecting various mix of classification and clustering algorithms in order to find the best model.Figure 2 illustrates the framework for the proposed hybrid meta-learner model.The hybrid meta-learner model

www.etasr.com Armaki et al.: A Hybrid Meta-Learner Technique for Credit Scoring of Banks' Customers
After collecting the desired datasets, the pre-process treatment should be applied on the data.In this stage, those data points which are considered as outliers or anomalies should be removed from datasets.For this reason, a distancebased outlier detection algorithm is used to identify n outliers in the given dataset based on the distance to their k nearest neighbors [50].Also, those examples with missing feature values are removed from datasets.Then, a combination of three classifier algorithms is interchangeably selected out of various classifiers as the base learners.As mentioned before, the stacking model is equipped with DL algorithm as the metalearner.The stacking model uses the results of base learners (level 0 generalizers) to train the meta-learner (level 1 generalizers).In the next step, the results of the DL as the processed data are fed into the clustering unit.Hence, it is expected to see significant improvement in the prediction accuracy of the model.The performances of models are compared after applying 10-fold cross-validation with stratified sampling during the training and testing stages.Therefore, each dataset is divided into 10 unique subsets (strata), in which any 9 of the 10 subsets are used for training and the leftover for testing.In other words, each model will be trained and tested 10 times.

D. Data
Financial institutions need to have a system for evaluating the credit risk of their customers when granting loans.But before that, they need to make sure that the accuracy of their system is at the acceptable level.Since, it is very difficult to assess the performance of a credit scoring system against private datasets, it is necessary to measure its performance against some benchmark datasets.As a result, we study four real-life datasets to evaluate the predictive power of our proposed hybrid meta-learner model.Table I displays the characteristics of the datasets.Particularly, the first three datasets are considered as benchmark datasets in the literature.They are related to consumer credit card loans from Australia, Germany, and Japan; and collected from the UCSD data repository.The last dataset is related to consumer loans in Iran and collected from Mellat Bank.

E. Evaluation Strategies
In order to assess the predictive power of the developed models, prediction accuracy rate and F-measure which is the harmonic mean of precision and recall are taken into account.Precision shows the accuracy degree of classification results and recall is the success rate of identifying classification results.Moreover, besides from these evaluation methods, type I and II errors are also shown for the best model in each dataset (Table II).These evaluation methods can be calculated as follows:

A. The Single Baseline Classifiers
Table III shows the prediction accuracy, F-measure and average rank of single baseline classifiers.In order to rank the performance of the models, we choose the average rank method as presented in [23].This method helps us compare various model performances and identify the top performers.As shown, the SVMPSO has shown the best predictive accuracy and F-measure and it is ranked first among the other baseline classifiers.SVMs are first introduced by authors in [51] as a form of linear classifiers.SVMs can be utilized for twofold classification with the aim of generating a best hyperplane (line) that sorts the input information into two classes (bad or good credit) [52].As a

www.etasr.com Armaki et al.: A Hybrid Meta-Learner Technique for Credit Scoring of Banks' Customers
population-based stochastic optimization approach, particle swarm optimization (PSO) works via simulating the birds' behavior inside a flock.This algorithm is introduced by authors in [53] and [54].PSO can be used to improve the accuracy of SVMs through identifying the best hyperplane which separates two classes.As can be seen from the results, the SVM-PSO is one of the highly accurate heuristic classifiers but only few studies have applied this method.According to the results, the second and third best performers are NN and AMLP.Conversely, the worst predictive performance on average belongs to NB.Here, F-measure shows the overall performance of the model via combining precision and recall values.

B. The Hybrid Meta-Learner Models
Table IV represents the prediction accuracy, F-measure and average rank of the best hybrid meta-learner models.As it is shown in this table, the (KNN-NN-SVMPSO)-(DL)-(DBSCAN) model has achieved the best accuracy rate and Fmeasure compared with other hybrid models on all credit datasets.It is interesting that other high performing models also have the same stacking combination as the best model.It is obvious that employing an optimized version of support vector machine, SVMPSO, has a significant role in improving the accuracy of the results.Moreover, among the clustering algorithms, DBSCAN has highest performance and followed by SOM and FCM.DBSCAN is a data clustering algorithm introduced in [55] and it is a density-based clustering algorithm that groups data points which are closely located together.DBSCAN is known as one of the top performing clustering algorithms in the literature.The advantages of DBSCAN algorithm can be enumerated as follows:  Unlike the K-means, DBSCAN does not need the number of clusters in the dataset to be specified a priori.
 It has the ability to detect clusters which are arbitrarily shaped.It can even discover a cluster entirely bounded by (but not linked to) a different cluster.

www.etasr.com Armaki et al.: A Hybrid Meta-Learner Technique for Credit Scoring of Banks' Customers
 It is robust to outliers so it can successfully deal with noises in datasets.
Table V compares the best baseline and hybrid models in terms of accuracy, F-measure, type I and II errors.The best hybrid model has improved the accuracy rate and F-measure results of the best baseline model by 2.68% and 5.97% on average, respectively.In contrast, the best hybrid model has reduced the type I and II errors by 79.84% and 95.16% on average, respectively.Furthermore, as we have included three public real-world credit datasets in this study, it is easy to compare the predictive performance of our hybrid models with other studies in the literature.
Table VI presents a comprehensive summary of credit scoring performances of various models which are used in the literature by several researchers.As the Australian and German credit data sets are the most frequently used datasets in the relevant literature, we also collected the results of the papers which have used these two datasets as their benchmarks.As shown, different authors have applied different models in their studies and they found various results in terms of predictive performance.It is interesting that the best performing hybrid model of this study is placed in the first rank based on the prediction accuracy measure.This hybrid credit scoring model has reached to an astonishing 99.71% and 99.80% accuracy on Australian and German datasets, respectively.

V. CONCLUSION
As financial institutions are exposed to credit risk when issuing consumer loans, developing reliable credit scoring systems is crucial for them.Since, machine learning methods have demonstrated their applicability and merit, this study develops and compares several hybrid machine learning approaches for the credit scoring problem.In this paper, a novel framework is proposed for hybrid meta-learning to improve the predictive performance of credit scoring models.Based on the selected datasets, the results show that the hybrid meta-learner model of (KNN-NN-SVMPSO)-(DL)-(DBSCAN) outpaces all the literature's baseline classifiers in terms of accuracy rate and type I/II errors.This model also outperforms the best models used in the relevant literature in terms of accuracy rate with a significant margin.The findings of this study give us the insight to realize which type of hybrid machine learning techniques is capable of achieving higher accuracy and lower error rates in the case of credit scoring.Also, it is clear that the optimized version of support vector machine, SVMPSO, and deep learning algorithms have significant roles in enhancing predictive power of the proposed models.As a result, it is believed that using the best credit scoring model identified by this study can help financial institutions to make more accurate and confident credit decisions in the future.For further studies, several issues can be studied in the future.One of them can be the reduction of feature dimensionality.To be precise, the pre-process treatment of selected datasets for dimensionality reduction or feature selection can be beneficial to achieve better prediction precision [72].Although, this paper is quite rich in terms of employing various machine learning algorithms, there are other techniques which can be applied for further comparisons especially those algorithms which are heuristically optimized.Lastly, since this paper specifically concentrates on the credit scoring problem, future studies can examine other problem areas such as corporate loans, house and car loans to identify which hybrid method has the best performance or if the empirical outcomes differ from the results of this paper.
Classification (or supervised learning) methods are capable of mapping input vectors into one of various preferred output classes through learning by examples.A classifier can be learned by computing the rough distance between input-output instances and correctly labeling outputs out of training set.This procedure is named as the model generation stage.After generating the model, the resulting classifier is able to classify an unidentified example based on the learned classes in the training set.Various classification techniques are employed in this paper which are Artificial Neural Network (NN), Automated Multilayer Perceptron (AMLP), Decision Tree (DT), K-nearest Neighbors (KNN), Logistic Regression (LR), Naïve Bayesian (NB), Support Vector Machines (SVM), and Support Vector Machines optimized by Particle Swarm Optimization (SVM-PSO).

Fig. 1 .
Fig. 1.The neuron architecture in the deep learning model

TABLE II .
CONFUSION MATRIX FOR A CREDIT SCORING PROBLEMNote: Positive class is Bad.

TABLE III .
ACCURACY, F-MEASURE AND AVERAGE RANK OF THE SINGLE BASELINE MODELS : All the numbers are in percentage form.The analyses are carried out in RapidMiner 7.2 program. Note

TABLE IV .
ACCURACY, F-MEASURE AND AVERAGE RANK OF THE BEST HYBRID MODELS Note: See TableIII

TABLE V .
AVERAGE PERFORMANCE COMPARISON OF THE BEST BASELINE AND HYBRID MODELS : "Improvement" shows the percentage growth rate for the accuracy and F-measure.It also show the type I and II error reduction rates in percentage. Note

TABLE VI .
PERFORMANCE COMPARISON OF VARIOUS CREDIT SCORING MODELS IN THE LITERATURE www.etasr.comArmaki et al.: A Hybrid Meta-Learner Technique for Credit Scoring of Banks' Customers