Browsing by master's degree program "Datatieteen maisteriohjelma"

When studying galaxy formation and evolution, the relationship between galaxy properties and dark matter halo properties are important, since galaxies form and evolve within these halos. This relationship can be figured out using numerical simulations, but unfortunately, they are computationally expensive and require vast amounts of computational resources. This provides incentive to use machine learning instead, since training a machine learning model requires significantly less time and resources. If machine learning could be used to predict galaxy properties from halo properties, numerical simulations would still be needed to find the halo population, but the more expensive hydrodynamical simulations would no longer be necessary. In this thesis, we use data from the IllustrisTNG hydrodynamical simulation to train five different types of machine learning models. The goal is to predict four different galaxy properties from multiple halo properties, and measure how accurate and reliable the predictions are. We also compare the different types of models with each other to find out which ones have the best performance. Additionally, we calculate confidence intervals for the predictions to evaluate the uncertainty of the models. We find that out of the four galaxy properties, stellar mass is the easiest to predict, whereas color is the most difficult one. From the five different types of models, light gradient boosting is in all cases either the best performing model, or its performance is almost as good as that of the best performing model. This, combined with the fact that training this type of model is extremely fast, light gradient boosting has good potential to be utilized in practice.

In little over a decade, cryptocurrencies have become a highly speculative asset class in global financial markets, with Bitcoin leading the way. Throughout its relatively brief history, the price of bitcoin has gone through multiple cycles of growth and decline. As a consequence, Bitcoin has become a widely discussed – and polarizing – topic on Twitter. This work studies whether the sentiment of popular Bitcoin-related tweets can be used to predict the future price movements of bitcoin. In total, seven different algorithms are evaluated: Vector Autoregression, Vector Autoregression Moving-Average, Random Forest, XGBoost, LightGBM, Long Short-Term Memory, and Gated Recurrent Unit. By applying lexicon-based sentiment analysis, and heuristic filtering of tweets, it was discovered that sentiment-based features of popular tweets improve the prediction accuracy over baseline features (open-high-low-close data) in five of the seven algorithms tested. The tree-based algorithms (Random Forest, XGBoost, LightGBM) generally had the lowest prediction errors, while the neural network algorithms (Light Short-Term Memory and Gated Recurrent Unit) had the poorest performance. The findings suggest that the sentiment of popular Bitcoin-related tweets can be an important feature in predicting the future price movements of bitcoin.

Recently, 3D reconstruction has become a popular topic due to applications in Virtual Reality, Augmented Reality, and historical heritage preservation. Yet, high-quality reconstruction is not available to the general public because of the cost of laser scanners. The goal of this thesis is to make the democratization of 3D reconstruction closer with the use of photogrammetry (reconstruction from multi-view images.) However, current approaches are very slow and tend to oversmooth the geometry. Our method involves learning the scene via a neural representation by taking posed multi-view images as input. We note that state-of-the-art (SOTA) approaches rely on traditional Structure-from-Motion (SfM) algorithms to extract camera poses. We also observe that SfM can generate a coarsely correct mesh for the underlying object. Nevertheless, SOTA techniques start training the neural representation from a sphere. Therefore, we propose a novel initialization method that takes the mesh obtained from SfM and initializes the neural representation from it. We validate our approach through extensive experiments on a widely used multi-view stereo DTU dataset. We show that our method outperforms both traditional and SOTA neural techniques in terms of reconstruction quality. It manages to learn the underlying geometry and recover small details like cracks and dents. We also show that it speeds up convergence by 4 times. All the datasets, reconstructed meshes, and learned model weights are available at this link.

Quantum computing has an enormous potential in machine learning, where problems can quickly scale to be intractable for classical computation. Quantum machine learning is research area that combines the interplay of ideas from quantum computing and machine learning. Powerful and useful machine learning is dependent on having large-scale datasets used to train the models to be able to solve real-life problems. Currently, quantum machine learning lacks a plethora of large-scale quantum datasets required to further develop the models and test the quantum machine learning algorithms. Lack of large datasets is currently limiting the quantum advantage in the field of quantum machine learning. In this thesis, the concept of quantum data and different types of applied quantum datasets used to develop quantum machine learning models is studied. The research methodology is based on a systematic and comparative literature review of the state of the art articles in quantum computing and quantum machine learning in the recent years. We classify datasets into inherent and non-inherent quantum data based on the nature of the data. The preliminary literature review addresses patterns in the applied quantum machine learning. Testing and benchmarking QML models primarily uses non-inherent quantum data, or classical data encoded into the quantum system, while separate research is focused on generating inherent quantum datasets.

Most machine learning projects consist of four distinct phases: data preparation, model training, model validation, and inference serving. Even though all of these phases are vital components of a successful machine learning project, the focus of most machine learning work is solely on the training of models. The other phases often need to be implemented with ad-hoc solutions, which can easily lead to technical debt. Technical debt is a metaphor for describing the quality of a software project. It describes the state of a project by comparing it to a financial loan. During software development, a loan can be taken to add value to the present state of the system. However, the loan comes with interest and has to be payed back. A loan can be taken, for example, by writing low quality code to meet a deadline. The loan has to be payed back by rewriting the code later, or else it will start to grow interest. The interest can be seen in the code functioning poorly or requiring substantial amounts of time to be understood. If a loan is not payed back, the interest keeps increasing, making it more and more difficult to pay the loan back later. In this thesis, we study the effect machine learning frameworks have on technical debt. We describe the machine learning project lifecycle and the various sources of technical debt associated with it. We review available machine learning frameworks and their mitigation strategies for the technical debt in machine learning projects. Our insights demonstrate how frameworks can be used to reduce the overall technical debt in machine learning projects.

Acidic sulfate soils (a.s. soils) decrease the soil pH and generate an acidic environment that is toxic for vegetation and aquatic life. Therefore, knowledge of potentially dangerous soils is necessary during, e.g. construction planning to reduce ecosystem service loss. Identifying a.s. soils accurately is a difficult task and require expensive and laborious field and lab tests. In order to identify larger areas cost-effectively, remote sensing methods could bring benefits to the construction planning field. This work shows that point clouds acquired with aerial LiDAR scanning can be used to remote sense acid sulfate soils with PointNet and PointNet++ deep learning models. However, additional research with larger data sets is required to improve the accuracy of the models for real-world applications. Therefore, work also suggests further research and improvement ideas for collecting data for such models.

Developing reliable, regulatory compliant and customer-oriented credit risk models requires thorough knowledge of credit risk phenomenon. Tight collaboration between stakeholders is necessary and hence models need to be transparent, interpretable and explainable as well as accurate, for experts without statistical background. In the context of credit risk, one can speak of explainable artificial intelligence (XAI). Hence, practice and market standards are also underlined in this study. So far, credit risk research has mainly focused on the estimation of the probability of default parameter. However, as systems and processes have evolved to comply with regulation in the last decade, recovery data has improved, which has raised loss given default (LGD) up to the heart of credit risk. In the context of LGD, most of the studies have emphasized estimation of one-stage models. However, in practice, market standards support a multi-stage approach which follows the institution's simplified recovery processes. Generally, multi-stage models are more transparent and have better predictive power and compliant status with the regulation. This thesis presents a framework to analyze and execute sensitivity analysis for multi-stage LGD model. The main contribution of the study is to increase the knowledge of LGD modelling by giving insights to the sensitivity of discriminatory power between risk drivers, model components and LGD score. The study aims to answer two questions. Firstly, how sensitive the predictive power of multi-stage LGD model is on the correlation of risk drivers and individual components? Secondly, how to identify the most driving risk factors that need to be considered in multi-stage LGD modelling to achieve adequate level LGD score? The experimental part of this thesis is divided into two parts. The first one presents the motivation, study design and experimental setup used in this thesis to execute the study. The second part focuses on the sensitivity analysis of risk drivers, components and LGD score. Sensitivity analysis presented in this study gives important knowledge of behavior of multi-stage LGD and dependencies between independent risk drivers, components and LGD score with regards to the correlations and model performance metrics. Introduced sensitivity framework can be utilised in assessing the need and schedule for model calibrations with related to the changes in application portfolio. In addition, framework and results can be used in recognizing the needs for monthly performed IFRS 9 ECL calculation updates. The study also gives input for model stress testing where different scenarios and impacts are analyzed regarding the changes in macroeconomic conditions. Even though the focus of this study is in credit risk, the methods presented are also applicable in the different fields outside the financial sector.

Improving employee well-being is a key part of pension agency Keva’s mission statement. Recently, Keva has launched a tool for conducting repeated small-scale employee well-being surveys called ”Pulssi”. With the number of responses reaching thousands Keva has identified processing and organizing this data as a part of this process that could be improved using machine learning methods. In this thesis, we conducted a comprehensive investigation into using language models and sentiment classifications as a solution. We tested three different methodologies for this purpose, traditional machine learning with learned embeddings, generative language methods, and fine-tuned BERT models. To our knowledge, this is the first study evaluating the use of language models on the Finnish sentiment analysis task. Additionally, we evaluated the feasibility of implementing these methods based on their operating costs and the time it took to create classifications. We found that the traditional machine learning trained on learned embeddings performed surprisingly well, achieving an accuracy of 91%. These models offer a fast and cost-effective alternative to the more cumbersome language models. Our fine-tuned BERT model the ”KevaBERT” achieved an impressive accuracy of 93.6%, when trained on GPT-4 generated predictions, suggesting a potential pathway for training data creation. Overall our best performance was achieved by the ”GPT-4 few-shot with context” model at 93.9% accuracy. Our accuracies rival or even surpass the state-of-the-art accuracies achieved on other datasets. Despite the near human-level performance, this model was slow and expensive to operate. Based on these findings we recommend the use of our ”KevaBERT” model for sentiment classifications and a separate GPT-4 based model for text summarization.

SQL kuuluu suositeltujen oppiaineiden joukkoon tietojenkäsittelytieteestä. Se on tehokas tapa varastoida dataa kontekstista riippumatta. SQL on kuitenkin opittavana aiheena opiskelijoilleen vaikea, ja tämän vuoksi SQL-opetuksen rinnalla käytetään opetusohjelmistoja. Opetusohjelmistojen avulla SQL:ää päästään opettelemaan käytännössä, paikataan suurta oppilaiden määrää opettajien määrään nähden, ja kerätään aineistoa opiskelijoiden suoriutumisesta. Oppimisohjelmistojen keräämä aineisto oppilaiden suoriutumisesta tarjoaa mahdollisuuden ennustaa opiskelijoiden suoriutumista kurssilla koneoppimismenetelmin. Tämä tutkielma kouluttaa SQL-opetusohjelmiston aineistoilla hyväksi todettuja koneoppimisalgoritmeja malleiksi, jotka osaavat ennustaa osaako opiskelija seuraavalla yrityksellään SQL-harjoitustehtävän oikein. Kyseessä ei ole tehdä mallia joka osaisi tarkastaa SQL-tehtäviä, vaan tarkoituksena on antaa koneoppimisalgoritmien tarkkailla opiskelijoilta muita kerättyjä tilastoja tehtäväyrityksen oikeellisuuden arvioimiseen ilman itse oppilaan antamaa ratkaisua. Tutkielmassa huomataan useiden koneoppimismallien olevan toimivia tämän tavoitteen saavuttamiseksi. Vastaavia koneoppimismalleja voidaan hyödyntää oppilaiden löytämisessä, joilla on vaikeuksia tehtävien tekemisessä. Tämä tieto on arvokasta esimerkiksi opetusohjelmistoille, jotka pyrkivät antamaan SQL-tehtävien tekijöille vihjeitä hyödylliseen aikaan.

As privacy gains consensus in the field of machine learning, numerous algorithms, such as differentially private stochastic gradient descent (DPSGD), have emerged to ensure privacy guarantees. Concurrently, fairness is garnering increasing attention, prompting research aimed at achieving fairness within the constraints of differential privacy. This thesis delves into algorithms designed to enhance fairness in the realm of differentially private deep learning and explores their mechanisms. It examines the role of normalization, a technique applied to these algorithms in practice, to elucidate its impact on fairness. Additionally, this thesis formalizes a hyperparameter tuning protocol to accurately assess the performance of these algorithms. Experiments across various datasets and neural network architectures were conducted to test our hypotheses under this tuning protocol. The decoupling of hyperparameters, allowing each to independently control specific properties of the algorithm, has proven to enhance performance. However, certain mechanisms, such as discarding samples with large norms and allowing unbounded hyperparameter adaptation, may significantly compromise fairness. Our experiments also confirm the critical role of hyperparameter values in influencing fairness, emphasizing the necessity of precise tuning to ensure equitable outcomes. Additionally, we observed differential convergence rates across algorithms, which affect the number of trials needed to identify optimal hyperparameter settings. This thesis aims to offer detailed perspectives on understanding fairness in differentially private deep learning and provides insights into designing algorithms that can more effectively enhance fairness.