Socioeconomic status is at the heart of everything we do in life from healthcare access to education to housing. And historically it has been measured by surveys which are slow, expensive, and prone to self-report bias. For my MSc Data Science dissertation at the University of Hertfordshire, I wanted to explore whether machine learning could be a faster, more scalable alternative to census data.
I used Folktables, an open source Python library that pulls curated, cleaned data directly from the US Census Bureau's American Community Survey. This eliminates a lot of the noise you get with self reported survey data, and it's fully reproducible since anyone can pull the same dataset.
The goal was to predict if an individual's income is above or below 50000 dollars a year, based on age, education level, occupation, hours worked per week, marital status, and sex.
I trained and tuned three classifiers.
Logistic regression as a fast and interpretable baseline. Random forest for a stronger nonlinear model. Gradient Boosting for the best possible predictive performance.
Each was scored in accuracy, precision, recall, F1 score, and AUC ROC. The linear model used Recursive Feature Elimination for feature selection, while the tree based models used feature importance scores.
The Results
Gradient Boosting was the best at 80 percent accuracy and an AUC of 0.88. Random Forest was close with an AUC of 0.87 and Logistic Regression was third at 0.81, as it failed to capture the nonlinear relationships of the data.
The five most important features of all models were occupation, education level, age, hours worked per week, and sex in line with current sociological research on income inequality.
Deployment
I didn’t want this to be another notebook that never gets used again. So I built a Flask web app around the final Gradient Boosting model, with a clean interface where anyone can enter a few demographic details and get a real time prediction. It’s deployed on Replit.
The next step is to focus on fairness. Any model for predicting income from demographic data must be very sensitive to bias, especially around protected characteristics. That’s something I’d like to explore further, possibly using the same Folktables framework to test fairness metrics across different demographic groups.
The full paper is openly published with a DOI through Zenodo and the code is on GitHub if you want to see the methodology or build on it.
Paper: https://doi.org/10.5281/zenodo.21963185
Code: https://github.com/Mustaphayinka/Msc_Data_Science_Project
I would really like to talk to anyone working on fairness-aware ML or applied data science more generally. Feedback and critique are welcome.
