The credit scoring has been regarded as a critical topic. This study proposed four approaches combining with the NN (Neural Network) classifier for features selection that retains sufficient information for classification purpose. Two UCI data sets and different approaches combined with NN classifier were constructed by selecting features. NN classifier combines with conventional statistical LDA, Decision tree, Rough set and F-score approaches as features preprocessing step to optimize feature space by removing both irrelevant and redundant features. The procedure of the proposed algorithm is described first and then evaluated by their performances. The results are compared in combination with NN classifier and nonparametric Wilcoxon signed rank test will be held to show if there has any significant difference between these approaches. Our results suggest that hybrid credit scoring models are robust and effective in finding optimal subsets and the compound procedure is a promising method to the fields of data mining.