Browsing by Subject "Simulation"

Most of the standard statistical inference methods rely on the evaluating so called likelihood functions. But in some cases the phenomenon of interest is too complex or the relevant data inapplicable and as a result the likelihood function cannot be evaluated. Such a situation blocks frequentist methods based on e.g. maximum likelihood estimation and Bayesian inference based on estimating posterior probabilities. Often still, the phenomenon of interest can be modeled with a generative model that describes supposed underlying processes and variables of interest. In such scenarios, likelihood-free inference, such as Approximate Bayesian Computation (ABC), can provide an option for overcoming the roadblock. Creating a simulator that implements such a generative model provides a way to explore the parameter space and approximate the likelihood function based on similarity between real world data and the data simulated with various parameter values. ABC provides well defined and studied framework for carrying out such simulation-based inference with Bayesian approach. ABC has been found useful for example in ecology, finance and astronomy, in situations where likelihood function is not practically computable but models and simulators for generating simulated data are available. One such problem is the estimation of recombination rates of bacterial populations from genetic data, which often is unsuitable for typical statistical methods due to infeasibly massive modeling and computation requirements. Overcoming these hindrances should provide valuable insight into evolution of bacteria and possibly aid in tackling significant challenges such as antimicrobial resistance. Still, ABC inference is not without its limitations either. Often considerable effort in defining distance functions, summary statistics and threshold for similarity is required to make the comparison mechanism successful. High computational costs can also be a hindrance in ABC inference; As increasingly complex phenomena and thus models are studied, the computations that are needed for sufficient exploration of parameter space with the simulation-comparison cycles can get too time- and resource-consuming. Thus efforts have been made to improve the efficiency of ABC inference. One improvement here has been the Bayesian Optimization for Likelihood-Free Inference algorithm (BOLFI), which provides efficient method to optimize the exploration of parameter space, reducing the amount of needed simulation-comparison cycles by up to several magnitudes. This thesis aims to describe some of the theoretical and applied aspects of the complete likelihood-free inference pipelines using both Rejection ABC and BOLFI methods. The thesis presents also use case where the neutral evolution recombination rate in Streptococcus pneumoniae population is inferred from well-studied real world genome data set. This inference task is used to provide context and concrete examples for the theoretical aspects, and demonstrations for numerous applied aspects. The implementations, experiments and acquired results are also discussed in some detail.

Comparison of amphetamine profiles is a task in forensic chemistry and its goal is to make decisions on whether two samples of amphetamine originate from the same source or not. These decisions help identifying and prosecuting the suppliers of amphetamine, which is an illicit drug in Finland. The traditional approach of comparing amphetamine samples involves computation of the Pearson correlation coefficient between two real-valued sample vectors obtained by gas chromatography-mass spectrometry analysis. A two-sample problem, such as the problem of comparing drug samples, can also be tackled with methods such as a t-test or Bayes factors. Recently, a newer method called predictive agreement (PA) has been applied in the comparison of amphetamine profiles, comparing the posterior predictive distributions induced by two samples. In this thesis, we did a statistical validation of the use of this newer method in amphetamine profile comparison. In this thesis, we compared the performance of the predictive agreement method to the traditional method involving computation of the Pearson correlation coefficient. Techniques such as simulation and cross-validation were used in the validation. In the simulation part, we simulated enough data to compute 10 000 PA and correlation values between sample pairs. Cross-validation was used in a case-study, where a repeated 5-fold group cross-validation was used to study the effect of changes in the data used in training of the model. In the cross-validation, performance of the models was measured with area under curve (AUC) values of receiver operating characteristics (ROC) and precision-recall (PR) curves. For the validation, two separate datasets collected by the National Bureau of Investigation of Finland (NBI), were available. One of the datasets was a larger collection of amphetamine samples, whereas the other dataset was a more curated group of samples, of which we also know which samples are somehow linked to each other. On top of these datasets, we simulated data representing amphetamine samples that were either from different or same source. The results showed that with the simulated data, predictive agreement outperformed the traditional method in terms of distinguishing sample pairs consisting of samples from different sources, from sample pairs consisting of samples from the same source. The case-study showed that changes in the training data have quite a marginal effect on the performance of the predictive agreement method, and also that with real world data, the PA method outperformed the traditional method in terms of AUC-ROC and AUC-PR values. Additionally, we concluded that the PA method has the benefit of interpretation, where the PA value between two samples can be interpreted as the probability of these samples originating from the same source.