Browsing by Author "Nguyen, Thang"

Machine learning (ML) systems are often deployed in complex, dynamic and uncertain environments, where their performance can degrade over time due to changes in data distribution or user behaviour. Monitoring and maintenance are essential processes to ensure the reliability and validity of ML systems in production. However, these processes pose many challenges for ML practitioners, such as choosing appropriate metrics, detecting anomalies, and deciding when to retrain models. In this thesis, we conduct a multivocal literature review to investigate the current state-of-the-art and best practices for ML model monitoring and maintenance. We aim to provide a comprehensive and systematic overview of the existing methods, tools, and challenges for ML model monitoring and maintenance, as well as to identify the gaps and future directions for research in this area. Through an examination of 28 sources, we made insights into common challenges of ML models post-deployment including but not all: concept drift, data pipeline and input data integrity, and complex models feedback loop. The corresponding monitoring and maintenance methods to mitigate some of those challenges were also extracted. In addition, we explored overall trends and demographics of the topic which show an increasing interest, as well as demonstrating how grey literature can provide practical and diverse experiences and opinions from industry practitioners.