Comparative Analysis of Ensemble Methods for Non-Linear Classification: A Data Mining Approach in Education

Resumen

Anticipating students orientation is a real challenge that defies simple linar logic. Educational data is apparently noisy , micsellaneous , and challenging to model using classical approaches. In this study, we test the hypothesis that Ensemble Methods offer a far superior path
to convergence when dealing with this complexity compared to traditional single-model classifiers.
Specifically, we pitted Bagging (Random Forest) and Boosting (Gradient Boosting - GBM) algorithms against standard baselines (SVM, Logistic Regression) using a real-world dataset of 798 high school students from the Souss-Massa region (Morocco). Therefore, not only was the objective to increase the level of accuracy, but to analyze how these models stabilize the loss function across 12 socio-academic variables.
The statistical verdict is clear: ensemble architectures handle non-linearity far more effectively. \textbf{Random Forest} secured the top spot with an Accuracy of 76.6% and an AUC of 0.865, significantly outperforming standalone Decision Trees (73.4%). The GBM concurrently showed exceptional robustness with a stable AUC of 0.859. Most importantly, our feature analysis proves that orientation isn't driven by a simple average grade; instead, Mathematics and Languages act as dominant, non-linear filters. These findings implies that for modeling complex human choices, ensemble learning is technically more reliable than linear approximations.