Time-Aware Cross-Validation for Dynamic Hotel No-Show Classification

I Gusti Ngurah Agung Krishna  Aditya; Syadia Nabilah Binti Mohd  Safuan; I Nyoman Gede Arya  Astawa

Authors

I Gusti Ngurah Agung Krishna Aditya School of Graduate Studies, Management Science and University, Malaysia
Syadia Nabilah Binti Mohd Safuan Faculty of Information Science & Engineering, Management Science and University, Malaysia
I Nyoman Gede Arya Astawa Department of Information and Technology, Bali State Polytechnic, Indonesia

Keywords:

Time-Aware Cross-Validation, Temporal Leakage, Rolling-Origin Validation, Purged K-Fold, Hotel No-Shows, Hospitality Analytics

Abstract

Time-aware cross-validation strategies are examined for dynamic classification of hotel reservation no-shows. Standard random resampling methods, which assume independent observations, introduce temporal leakage when applied to booking data and consequently yield inflated performance estimates. Temporally consistent validation approaches, including rolling-origin, expanding-window, and purged cross-validation with temporal embargo, are evaluated using hotel reservation timelines as the empirical context. Results indicate that time-aware validation produces more conservative and stable performance estimates across forecasting horizons, reveals sensitivity of temporally driven features, and exposes generalization degradation obscured by random cross-validation. Validation strategy is shown to materially influence perceived model reliability and downstream operational decisions such as overbooking control. A practical framework and implementation guidelines are provided for temporally consistent evaluation in hospitality analytics, with broader relevance to event-driven classification problems.

References

C. Fraisse, “Introducing Concept Drifts,” OCTO Talks. Accessed: Jan. 16, 2026. [Online]. Available: https://blog.octo.com/introducing-concept-drifts

J. Xiao, S. Z. Abidin, V. V. Vermol, and B. Gong, “Dynamic temporal reinforcement learning and policy-enhanced LSTM for hotel booking cancellation prediction,” PeerJ Comput. Sci., vol. 10, p. e2442, 2024, doi: 10.7717/peerj-cs.2442.

C. Bergmeir, R. J. Hyndman, and B. Koo, “A note on the validity of cross-validation for evaluating autoregressive time series prediction,” Comput. Stat. Data Anal., vol. 120, pp. 70–83, 2018.

D. Liu et al., “Machine learning approaches to predicting no-shows in pediatric medical appointment,” npj Digital Medicine 2022 5:1, vol. 5, no. 1, pp. 50-, Apr. 2022, doi: 10.1038/s41746-022-00594-w.

F. O. Osorio et al., “Predicting No-Shows at Outpatient Appointments in Internal Medicine Using Machine Learning Models,” PeerJ Comput. Sci., vol. 11, pp. 1–29, 2025, doi: 10.7717/PEERJ-CS.2762/SUPP-3.

M. Adil et al., “Solving the Problem of Class Imbalance in the Prediction of Hotel Cancelations: A Hybridized Machine Learning Approach,” Processes 2021, Vol. 9, Page 1713, vol. 9, no. 10, p. 1713, Sep. 2021, doi: 10.3390/PR9101713.

C. Deina, F. S. Fogliatto, G. J. C. da Silveira, and M. J. Anzanello, “Decision analysis framework for predicting no-shows to appointments using machine learning algorithms,” BMC Health Serv. Res., vol. 24, no. 1, p. 37, Dec. 2024, doi: 10.1186/S12913-023-10418-6.

S. Albelali and M. Ahmed, “Hidden Leaks in Time Series Forecasting: How Data Leakage Affects LSTM Evaluation Across Configurations and Validation Strategies,” 2025.

R. P. Fraga, Z. Kang, and C. M. Axthelm, “Effect of Machine Learning Cross-validation Algorithms Considering Human Participants and Time-series: Application on Biometric Data Obtained from a Virtual Reality Experiment,” Proceedings of the Human Factors and Ergonomics Society Annual Meeting, vol. 67, no. 1, pp. 2162–2167, 2023, doi: 10.1177/21695067231192258.

O. Gulaydin and M. Mourshed, “Machine learning for subnational residential electricity demand forecasting to 2050 under shared socioeconomic pathways: Comparing tree-based, neural and kernel methods,” 2025, doi: 10.1016/j.energy.2025.138195.

J. Palet, “Context-based predictive models for medical emergencies,” 2022.

P. Silvestre, N. Antonio, and P. Carrasco, “Navigating uncertainty: enhancing hotel cancellation predictions with adaptive machine learning,” Information Technology & Tourism 2025 28:1, vol. 28, no. 1, pp. 9-, Dec. 2025, doi: 10.1007/S40558-025-00349-9.

C. Bergmeir and J. M. Benítez, “On the use of cross-validation for time series predictor evaluation,” Inf. Sci. (N. Y)., vol. 191, pp. 192–213, May 2012, doi: 10.1016/J.INS.2011.12.028.

V. Cerqueira, L. Torgo, and I. Mozetič, “Evaluating time series forecasting models An empirical study on performance estimation methods,” 2019.

N. Antonio, A. de Almeida, and L. Nunes, “Hotel booking demand datasets,” Data Brief, vol. 22, pp. 41–49, Feb. 2019, doi: 10.1016/j.dib.2018.11.126.