Financial institutions normally have limited applicants and small sample credit datasets in early stages of business. Machine learning models may get overfitted due to a lack of sufficient training samples, which will lower the models’ classification accuracy. This paper proposes a novel transfer learning model to tackle this challenge, via aligning the conditional probability distribution and the marginal probability distribution between traditional businesses and new businesses. We conduct experiments on two real credit datasets to validate the model. Experimental results show that the proposed model outperforms other benchmark algorithms in prediction accuracy. The proposed model could have the potential for various application scenarios, including the utilization of non-financial data such as legal documents for credit rating or related risk assessments.