The purpose of this paper is to make a new trial to explore the influence factors of loan default in Internet finance loan business. A facial feature extraction and classification model is proposed. The optimal facial feature extraction algorithm is obtained by comparing four commonly used facial feature extraction algorithms and certain facial features based on physiognomy are selected and classified in 117, 507face images. An experimental study with the help of the proposed model is conducted to explore the correlations between loan defaults and Internet finance loan users’ classified facial features based on physiognomy. The findings are as follows: among male Internet finance loan users, short eyebrows are related to default and eye angle, nose height-to-width ratio (nHWR), lip thickness and facial width-to-height ratio (fWHR) are positively related to default behavior and the mouth length is negatively related to default; among female Internet finance loan users, eyebrows angle, eyes angle, lip thickness and facial width-to-height ratio are positively correlated to default and mouth length is negatively correlated with default. Additionally, the conclusion of that male fWHR is positively related to default of the proposed study is echoed with the research results of [1] and [2].