Cleaning the data
The dataset needed work before any model could use it.
- Loan approval status, the target, had inconsistent labels: Approved, APPROVED, Declined, DECLINED, Accept and Reject. I standardised them to Approved and Declined.
- Sex was 99.4% missing, so I dropped it.
- Credit application acceptance overlapped with the approval outcome and would leak the answer into the model, so I dropped it.
- Age ranged from minus 30 to 180. I filled missing values with the mean and removed outliers with the IQR method.
- Payment default on file mixed Y, N, YES and NO. I standardised and encoded it.
- Loan amount had 2,045 outliers, removed with IQR.
Part 1: approved or declined
I trained three models on an 80/20 split, stratified so both sets kept the same approved to declined ratio, with a fixed random state so the results can be reproduced. Each model was tested on the same 10,845 unseen applications.
The bank wanted a model that catches as many declined applications as possible, and is right when it does. So accuracy was not the main measure. Recall, precision and F1 on the declined class were.
| Model | Accuracy | Precision | Recall | F1 | AUC |
|---|---|---|---|---|---|
| Naive Bayes | 0.87 | 0.95 | 0.04 | 0.07 | 0.86 |
| Logistic regression | 0.90 | 0.72 | 0.39 | 0.51 | 0.88 |
| kNN, k = 13 | 0.90 | 0.69 | 0.49 | 0.57 | 0.87 |
kNN was the best model. It had the highest recall and F1 for declined loans, so it was better at catching rejections than the other two, even though it still misses some.
Tuning kNN
GridSearchCV picked k = 29 with Manhattan distance.
| kNN | Accuracy | Precision | Recall | F1 |
|---|---|---|---|---|
| Before, k = 13 | 0.90 | 0.69 | 0.49 | 0.57 |
| After, k = 29 | 0.90 | 0.72 | 0.44 | 0.55 |
Precision went up, but recall and F1 went down. For a bank that wants to catch declined loans, tuning did not improve the model.
An ensemble
I combined Naive Bayes and logistic regression in a soft voting ensemble, since both output probabilities. Precision rose to 0.88, but recall dropped to 0.16. It missed too many declined applications, and its F1 was worse than either model on its own. For the bank's goal, logistic regression is the better choice than the ensemble.
Part 2: the maximum loan amount
For approved clients I compared two decision tree regressors: a fully grown tree, and a tree pruned to four levels.
| Model | MAE | MSE | R² |
|---|---|---|---|
| DT-1, fully grown | 559.93 | 1,641,015.56 | 0.9985 |
| DT-2, pruned to 4 levels | 8,116.98 | 122,045,545.50 | 0.8875 |
DT-1 was better on every metric. The pruned tree was too simple and underfitted. But DT-1 is 24 levels deep, and an R² that close to 1 suggests it may be overfitting. A tuned tree, with depth 21 found by cross-validated grid search, scored almost the same (R² 0.9984) and is the safer choice in practice.
One client
Client 60256 was 56, earned 57,000 and asked for 25,700. The tuned tree predicted a maximum of 16,750. Several of this client's values were outside anything the model was trained on: their age, a 35 year credit history, and a 23% interest rate. For values out of range, the tree gave a lower amount.
Limitations
- The best classifier still misses about half of all declined applications.
- kNN treats every feature as equally important, slows down as the data grows, and depends heavily on scaling.
- Removing age outliers with IQR capped ages at around 40, so the model never saw older applicants. A simple filter for ages 18 to 100 would have been better.
Ethics
Because older age groups were removed from the training data, the model can be biased against older applicants. Age should not decide whether someone gets a loan.
Python, pandas, scikit-learn and Jupyter.