Browsing by Subject "confidence calibration"

In this thesis, we give an overview of current methodology in the field of uncertainty estimation in machine learning, with focus on confidence scores and their calibration. We also present a case study, where we propose a novel method to improve uncertainty estimates of an in-production machine learning model operating in an industrial setting with real-life data. This model is used by a Finnish company Basware to extract information from invoices in the form of machine-readable PDFs. The solution we propose is shown to produce confidence estimates, which outperform the legacy estimates on several relevant metrics, increasing coverage of automated invoices from 65.6% to 73.2% with no increase in error rate.