Authors : Stow, May; Stewart, Ashley Ajumoke

Volume/Issue : Volume 10 - 2025, Issue 8 - August

Google Scholar : https://tinyurl.com/2p2cebmn

Scribd : https://tinyurl.com/4hh4drf5

DOI : https://doi.org/10.38124/ijisrt/25aug970

Note : A published paper may take 4-5 working days from the publication date to appear in PlumX Metrics, Semantic Scholar, and ResearchGate.

Note : Google Scholar may take 30 to 40 days to display the article.

Abstract : The exponential growth in machine learning model complexity has led to substantial increases in computational requirements and associated carbon emissions, raising concerns about the environmental sustainability of artificial intelligence systems. While previous research has primarily focused on neural network compression for GPU accelerated environments, the environmental impact of classical machine learning algorithms deployed on CPU infrastructure remains underexplored. This study investigates the application of pruning and aggressive pruning techniques to Random Forest and Gradient Boosting classifiers, evaluating their effectiveness in reducing carbon emissions while maintaining acceptable predictive performance. The research employs structural compression methods including tree pruning and estimator reduction across three UCI benchmark datasets (Adult Income, Wine Quality, Heart Disease) with varying size and class distribution characteristics. Comprehensive evaluation encompasses performance metrics, computational efficiency, and lifecycle carbon footprint analysis. Results demonstrate that combined pruning achieves 97.6% reduction in carbon emissions while maintaining 94.5% of baseline accuracy. Notably, compressed Random Forest models exhibited improved F1 scores on imbalanced datasets, with up to 137% improvement on Wine Quality data, suggesting compression serves as implicit regularization. Model size reductions reached 54% with inference time improvements of 38%. These findings establish that aggressive compression of tree based ensembles can simultaneously address environmental concerns and computational constraints without prohibitive performance degradation, making sustainable machine learning accessible for resource constrained deployments

Keywords : Green AI, Model Compression, Ensemble Pruning, Carbon Footprint, Sustainable Computing, Tree-Based Models.

References :

1. Anthony, L. F. W., Kanding, B., & Selvan, R. (2020). Carbontracker: Tracking and predicting the carbon footprint of training deep learning models. arXiv preprint. https://doi.org/10.48550/arXiv.2007.03051
2. Breiman, L. (2001). Random forests. Machine Learning, 45(1), 5-32. https://doi.org/10.1023/A:1010933404324
3. Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D. M., Wu, J., Winter, C., et al. (2020). Language models are few-shot learners. Advances in Neural Information Processing Systems, 33, 1877-1901. https://doi.org/10.48550/arXiv.2005.14165
4. Chen, T., & Guestrin, C. (2016). XGBoost: A scalable tree boosting system. Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 785-794. https://doi.org/10.1145/2939672.2939785
5. Cheng, Y., Wang, D., Zhou, P., & Zhang, T. (2018). Model compression and acceleration for deep neural networks: The principles, progress, and challenges. IEEE Signal Processing Magazine, 35(1), 126-136. https://doi.org/10.1109/MSP.2017.2765695
6. Courbariaux, M., Hubara, I., Soudry, D., El-Yaniv, R., & Bengio, Y. (2016). Binarized neural networks: Training deep neural networks with weights and activations constrained to +1 or -1. arXiv preprint. https://doi.org/10.48550/arXiv.1602.02830
7. Fernández, A., García, S., Galar, M., Prati, R. C., Krawczyk, B., & Herrera, F. (2018). Learning from imbalanced data sets. Springer. https://doi.org/10.1007/978-3-319-98074-4
8. Frankle, J., & Carbin, M. (2018). The lottery ticket hypothesis: Finding sparse, trainable neural networks. arXiv preprint. https://doi.org/10.48550/arXiv.1803.03635
9. Friedman, J. H. (2001). Greedy function approximation: A gradient boosting machine. Annals of Statistics, 29(5), 1189-1232. https://doi.org/10.1214/aos/1013203451
10. Gholami, A., Kim, S., Dong, Z., Yao, Z., Mahoney, M. W., & Keutzer, K. (2018). A survey of aggressive pruning methods for efficient neural network inference. arXiv preprint. https://doi.org/10.48550/arXiv.2103.13630
11. Gupta, U., Kim, Y. G., Lee, S., Tse, J., Lee, H. H. S., Wei, G. Y., Brooks, D., & Wu, C. J. (2021). Chasing carbon: The elusive environmental footprint of computing. IEEE Micro, 41(5), 34-42. https://doi.org/10.1109/MM.2021.3094469
12. Han, S., Mao, H., & Dally, W. J. (2015). Deep compression: Compressing deep neural networks with pruning, trained aggressive pruning and Huffman coding. arXiv preprint. https://doi.org/10.48550/arXiv.1510.00149
13. He, H., & Garcia, E. A. (2009). Learning from imbalanced data. IEEE Transactions on Knowledge and Data Engineering, 21(9), 1263-1284. https://doi.org/10.1109/TKDE.2008.239
14. Henderson, P., Hu, J., Romoff, J., Brunskill, E., Jurafsky, D., & Pineau, J. (2020). Towards the systematic reporting of the energy and carbon footprints of machine learning. Journal of Machine Learning Research, 21(248), 1-43. http://jmlr.org/papers/v21/20-312.html
15. Hernández-Lobato, D., Martínez-Muñoz, G., & Suárez, A. (2009). Statistical instance-based pruning in ensembles of independent classifiers. IEEE Transactions on Pattern Analysis and Machine Intelligence, 31(2), 364-369. https://doi.org/10.1109/TPAMI.2008.204
16. Hinton, G., Vinyals, O., & Dean, J. (2015). Distilling the knowledge in a neural network. arXiv preprint. https://doi.org/10.48550/arXiv.1503.02531
17. Hooker, S., Courville, A., Clark, G., Dauphin, Y., & Frome, A. (2020). What do compressed deep neural networks forget? arXiv preprint. https://doi.org/10.48550/arXiv.1911.05248
18. Hooker, S., Moorosi, N., Clark, G., Bengio, S., & Denton, E. (2021). Characterising bias in compressed models. arXiv preprint. https://doi.org/10.48550/arXiv.2010.03058
19. Jacob, B., Kligys, S., Chen, B., Zhu, M., Tang, M., Howard, A., Adam, H., & Kalenichenko, D. (2018). Aggressive pruning and training of neural networks for efficient integer-arithmetic-only inference. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2704-2713. https://doi.org/10.1109/CVPR.2018.00286
20. Ke, G., Meng, Q., Finley, T., Wang, T., Chen, W., Ma, W., Ye, Q., & Liu, T. Y. (2017). LightGBM: A highly efficient gradient boosting decision tree. Advances in Neural Information Processing Systems, 30, 3146-3154. https://doi.org/10.5555/3294996.3295074
21. Lacoste, A., Luccioni, A., Schmidt, V., & Dandres, T. (2019). Quantifying the carbon emissions of machine learning. arXiv preprint. https://doi.org/10.48550/arXiv.1910.09700
22. Lane, N. D., Bhattacharya, S., Georgiev, P., Forlivesi, C., Jiao, L., Qendro, L., & Kawsar, F. (2016). DeepX: A software accelerator for low-power deep learning inference on mobile devices. Proceedings of the 15th International Conference on Information Processing in Sensor Networks, 1-12. https://doi.org/10.1109/IPSN.2016.7460664
23. LeCun, Y., Denker, J. S., & Solla, S. A. (1990). Optimal brain damage. Advances in Neural Information Processing Systems, 2, 598-605. https://doi.org/10.5555/109230.109298
24. Louizos, C., Reisser, M., Blankevoort, T., Gavves, E., & Welling, M. (2018). Relaxed aggressive pruning for discretized neural networks. arXiv preprint. https://doi.org/10.48550/arXiv.1810.01875
25. Martínez-Muñoz, G., & Suárez, A. (2006). Pruning in ordered bagging ensembles. Proceedings of the 23rd International Conference on Machine Learning, 609-616. https://doi.org/10.1145/1143844.1143921
26. Painsky, A., & Rosset, S. (2016). Cross-validated variable selection in tree-based methods improves predictive performance. IEEE Transactions on Pattern Analysis and Machine Intelligence, 39(11), 2142-2153. https://doi.org/10.1109/TPAMI.2016.2636831
27. Patterson, D., Gonzalez, J., Le, Q., Liang, C., Munguia, L. M., Rothchild, D., So, D., Texier, M., & Dean, J. (2021). Carbon emissions and large neural network training. arXiv preprint. https://doi.org/10.48550/arXiv.2104.10350
28. Polino, A., Pascanu, R., & Alistarh, D. (2018). Model compression via distillation and aggressive pruning. arXiv preprint. https://doi.org/10.48550/arXiv.1802.05668
29. Prokhorenkova, L., Gusev, G., Vorobev, A., Dorogush, A. V., & Gulin, A. (2018). CatBoost: Unbiased boosting with categorical features. Advances in Neural Information Processing Systems, 31, 6638-6648. https://doi.org/10.48550/arXiv.1706.09516
30. Samie, F., Bauer, L., & Henkel, J. (2016). IoT technologies for embedded computing: A survey. Proceedings of the Eleventh IEEE/ACM/IFIP International Conference on Hardware/Software Codesign and System Synthesis, 1-10. https://doi.org/10.1145/2968456.2974004
31. Schwartz, R., Dodge, J., Smith, N. A., & Etzioni, O. (2020). Green AI. Communications of the ACM, 63(12), 54-63. https://doi.org/10.1145/3381831
32. Strubell, E., Ganesh, A., & McCallum, A. (2019). Energy and policy considerations for deep learning in NLP. Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, 3645-3650. https://doi.org/10.18653/v1/P19-1355
33. Wu, C. J., Raghavendra, R., Gupta, U., Acun, B., Ardalani, N., Maeng, K., Chang, G., Aga, F., Huang, J., Bai, C., Gschwind, M., Gupta, A., Ott, M., Melnikov, A., Candido, S., Brooks, D., Chauhan, G., Lee, B., Lee, H. H. S., ... Hazelwood, K. (2022). Sustainable AI: Environmental implications, challenges and opportunities. Proceedings of Machine Learning and Systems, 4, 795-813. https://doi.org/10.48550/arXiv.2111.00364
34. Zhang, L., & Wang, G. (2019). Fast training of random forests. Proceedings of the AAAI Conference on Artificial Intelligence, 33(1), 5910-5917. https://doi.org/10.1609/aaai.v33i01.33015910
35. Zhou, Z. H., Wu, J., & Tang, W. (2002). Ensembling neural networks: Many could be better than all. Artificial Intelligence, 137(1-2), 239-263. https://doi.org/10.1016/S0004-3702(02)00190-X