Safety-Oriented Air Quality Index Classification for Imbalanced Data Using Optimized Boosting Models with Optuna and Oversampling

Abstract

Air Quality Index (AQI) classification is essential for communicating environmental health risks. However, hazardous air conditions occur far less frequently than normal conditions, challenging conventional classification models. This study investigates multi-class AQI classification using the "Global Air Quality 2025" dataset, comprising 52,704 observations with an extreme class imbalance ratio of approximately 1:173. Under such conditions, conventional accuracy metrics often mask systemic failures in detecting critical minority classes. To address potential data leakage present in previous approaches, this research implements a rigorous cross-validation architecture combined with an independent 20% hold-out test set. The methodology employs an Ablation Study to systematically isolate the impacts of Optuna hyperparameter tuning guided by Macro F1-Score and oversampling techniques (SMOTE and ADASYN). The results demonstrate that the proposed Hybrid-SMOTE LightGBM configuration successfully balances hazard detection sensitivity with global stability. On the unseen hold-out set, the optimal model achieved a Macro F1-Score of 0.8079, an accuracy of 92.80%, and a ROC-AUC of 0.9847. Crucially, the model delivered a 65.12% recall for the critical Unhealthy minority class, a nearly 40% improvement over the baseline. Error profile analysis confirmed the model's safety-oriented robustness, as 97.6% of peak hazardous events were either accurately classified or safely constrained to the adjacent warning category, minimizing catastrophic misclassifications. These findings prove that reliable detection of environmental hazards requires safety-oriented per-class evaluation and strict validation frameworks, as reliance on aggregate global metrics leads to dangerously misleading performance assessments.