Ensemble-Based Predictive Framework for Computational Workload Forecasting Using Google Borg Traces

Abstract

This paper introduces a powerful framework based on data regarding future predatory workloads using Google Borg Traces, which is one among the largest examinations of cluster-extensive resource management figures. In the study, the researchers are going to create a predictive model in the form of an accurate and interpretable prediction of CPU utilization using systematic data preparation, advanced preprocessing, feature engineering, and ensemble-based machine learning methods. The data, which was comprehensive in terms of scheduling of tasks, CPU and memory consumption, system performance indicators, were completely cleaned, standardized and converted into formats that are analytical reliable. To select the features, the Mutual Information analysis was utilized, and the Quantile, Power and Z-score transformations were used to normalize the features to improve the stability of the model. Complex feature engineering procedures were used to capture both temporal, frequency and statistical dynamics, and Truncated Singular Value Decomposition (SVD) was used to dimensionality reduce computational efficiency. Three ensemble regression algorithms were used: Random Forest, Light Gradient Boosting Machine (LightGBM), and CatBoost and tested on Mean Absolute Error (MAE), Root Mean Squared Error (RMSE), and Coefficient of Determination (R 2). The best of them was the Random Forest Regressor (MAE = 0.000003, RMSE = 0.000109, R 2 = 0.999915), with LightGBM and CatBoost trailing behind it with very close predictive accuracy. To further improve interpretability, SHAP analysis was used to determine the most significant predictors, including the average and maximum use of CPU, which has a significant effect on the outputs of the model. The findings substantiate that ensemble-based models are effective in modeling non-linear relationships that are very complex, and exist amongst system metrics and resource utilization. Altogether, the work provides a scalable, interpretable, and high-performance model of the intelligent workload prediction and optimization model within large-scale computing systems that can be useful in enhancing efficiency, reliability, and predictive control in recent cloud and cluster computing systems.