Browsing by discipline "Algorithms and Machine Learning"

Traditional flat classification methods (e.g., binary, multiclass, and multi-label classification) seek to associate each example with a single class label or a set of labels without any structural dependence among them. Although, there are some problems in which classes can be divided or grouped into subclasses or superclasses respectively. Such a scenario demands the application of methods prepared to deal with hierarchical classification. An algorithm for hierarchical classification uses the information related to structure present in the class hierarchy and then improves the predictive performance . The freedom to perform a more generic classification, but with higher reliability, gives the process a greater versatility. Several studies have shown that, in solving a hierarchical classification problem, flat models are mostly overcome by hierarchical ones, regardless of the approach – local (including its derivations) or global – chosen. This thesis aims to compare the most popular hierarchical classification methods (local and global) empirically, reporting their performance – measured using hierarchical evaluation indexes. To do so, we had to adapt the global hierarchical models to conduct single path predictions, starting from the root class and moving towards a leaf class within the hierarchical structure. Further, we applied hierarchical classification on data streams by detecting concept drift. We first study data streams, various types of concept drifts, and state-of-the-art concept drift detection methods. Then we implement Global-Model Hierarchical Classification Naive Bayes (GNB) with three concept drift detectors: (i) Kolmogorov-Smirnov test, (ii) Wilcoxon test, and (iii) Drift Detection Method (DDM). A fixed-size sliding window was used to estimate the performance of GNB online. Finally, we must highlight that this thesis contributes to the task of automatic insect recognition.

Data compression is one way to gain better performance from a database. Compression is typically achieved with a compression algorithm, an encoding or both. Effective compression directly lowers the physical storage requirements translating to reduced storage costs. Additionally, in case of a data transfer bottleneck where CPU is data starved, compression can yield improved query performance through increased transfer bandwidth and better CPU utilization. However, obtaining better query performance is not trivial since many factors affect the viability of compression. Compression has been found especially successful in column oriented databases where similar data is stored closely in physical media. This thesis studies the effect of compression on a columnar storage format Apache Parquet through a micro benchmark that is based on the TPC-H benchmark. Compression is found to have positive effects on simple queries. However, with complex queries, where data scanning is relatively small portion of the query, no performance gains were observed. Furthermore, this thesis examines the decoding performance of the encoding layer that belongs to a case database, Fastorm. The goal is to determine its efficiency among other encodings and whether it could be improved upon. Fastorm's encoding is compared against various encodings of Apache Parquet in a setting where data is from a real world business. Fastorm's encoding is deemed to perform well enough coupled with strong evidence to consider adding delta encoding to its repertoire of encoding techniques.

Interpretability in machine learning aims to provide explanations on the behaviors of complex predictive models, widely refer as black-boxes. Generally, interpretability means the understanding of how the models work internally, whereas, explanations are the one way to make machine learning models interpretable, e.g., using transparent and simple models. Numerous approaches have been proposed as explanation methods which strive to interpret black-box models. These explanation methods mainly try to approximate the local behavior of a model, and then explain it in a human-understandable way. The primary reason to explain the local-behavior is that explaining the global behavior of a black-box is difficult, and it remains an unsolved challenge. Moreover, there is another challenge which argues on the quality and stability of the generated explanations. One way to evaluate the quality of explanations is by using robustness as a property. In this work, we define the explanation evaluation framework, which attempts to measure the robustness of explanations. The framework consists of two distance-based measures stability and separability. We explore and use stability measure from existing literature and introduce our new separability measure, which goes along with stability measure in order to quantify the robustness of explanations. We examine model-agnostic (LIME, SHAP) and model-dependent (DeepExplain) explanation methods to interpret the predictions for various supervised predictive models, especially classifiers. We build classifiers by using UCI classification benchmark datasets and MNIST handwritten digits dataset. Our results illustrate that current model-agnostic and model-dependent explanation methods do not perform adequately with respect to our explanation evaluation framework. Our results show that these explanation methods are not robust to variations in features values and often produce different explanations for similar values and similar explanations for different values, which leads to unstable explanations. Our results and outcomes demonstrate that the developed explanation evaluation framework is useful to assess the robustness of explanations and inspire further exploration and work.

Anomaly detection is an important task in many domains such as maritime where it is used to detect, for example, unsafe, unexpected or criminal behaviour. This thesis studies the use of deep autoencoders for anomaly detection on high dimensional data in an unsupervised manner. The study is performed on a benchmark data set and a real-life AIS (Automatic Tracking System) data set containing actual ship trajectories. The ships’ trajectories in the AIS data set are a form of time-series data, and therefore recurrent layers are used in an autoencoder to allow the model to capture temporal dependencies in the data. An autoencoder is a neural network architecture where an encoder network produces an encoding and decoder network takes the encoding intending to produce the original input. An encoding is a compressed fixed-sized vector presentation of the original input. Since the encoding is used by the decoder to construct the original input, the model learns during the training process to store essential information of the input sequence to the encoding. Autoencoders can be used to detect anomalies using reconstruction error by assuming that a trained autoencoder is able to reconstruct non-anomalius data points more accurately than anomalous data points, and therefore data points with high reconstruction error can be considered anomalies. In addition to reconstruction error, the autoencoders produce encodings. The research of this thesis studies the possibility of calculating an outlier score for the encodings and combining the score with resconstruction error to form a combined outlier score. OPTICS-OF (Ordering Points to Identify the ClusteringStructure with Outlier Factors) is a density based anomaly detection technique which can be used to calculate outlier scores for the encodings. The outlier score of OPTICS-OF for a data point is based on how isolated it is within its neighbourhood. The proposed method is evaluated on a benchmark Musk data set for which anomalies are known. A data set with labelled anomalies provides a setting for analyzing the performance of the method and its properties. The method is then put to the test on the AIS data set where it is used to find new anomalies in the data set from two derived distinct feature sets. The AIS data set contains one known anomaly which is presented both as an example of a maritime anomaly and for which more detailed analysis of the produced outlier scores are presented. The results of the study show potential for the proposed combined score method, and the analysis identifies multiple areas for further research. Deep autoencoders are successfully used to find new anomalies from the AIS data set which show actual behaviour deviating from normal ship movement.

Time series analysis has been a popular research topic in the last few decades. In this thesis, we develop time series models to investigate short time series of count data. We first begin with Poisson autoregressive model and extend it to capture day effects explicitly. Then we propose hierarchical Poisson tensor factorization model as an alternative to the traditional count time series models. Furthermore, we suggest acontext-based model as an improvement over hierarchical Poisson tensor factorization model. We implement the models in an open-source probabilistic programming framework Edward. This tool enables us to express the models in form of executable program code and allows us to rapidly prototype models without the need of derivation of model specificupdaterules. We also explore strategies for selecting the best model out of alternatives. We study the proposed models on a dataset containing media consumption data. Our experimental findings demonstrate that the hierarchical Poisson tensor factorization model significantly outperforms the Poisson autoregressive models in predicting event counts. We also visualize the key results of our exploratory data analysis.

Spam detection techniques have made our lives easier by unclogging our inboxes and keeping unsafe messages from being opened. With the automation of text messaging solutions and the increase in telecommunication companies and message providers, the volume of text messages has been on the rise. With this growth came along malicious traffic which users had little control over. In this thesis, we present an implementation of a spam detection system in a real-world text messaging platform. Using well-established machine learning algorithms, we make an in-depth analysis on the performance of the models using two different datasets: one publicly available (N=5,574) and the other gathered from actual traffic of the platform (N=1,477). Making use of the empirical results, we outline the models and hyperparameters which can be used in the platform and in which scenarios they produce optimal performance. The results indicate that our dataset poses a great challenge at accurate classification, most likely due to the small sample size and unbalanced dataset, along with nuances in the dataset. Nevertheless, there were models that were found to have a good all-around performance and they can be trained and used in the platform.

Most online fraud involves identity thief, especially in financial services such as banking, commercial services, or home security. Passwords have always been one of the most reliable and common way to protect user identities. However, passwords can be guessed or breached. Biometric authentications have emerged to be a compliment way to improve the security. Nevertheless, biometric factors such as fingerprint or face recognition can also be spoofed. Additionally, those factors require either user interaction (touch to unlock) or additional hardware (surveillance camera). Therefore, the next level of security with lower risk of attack and less user friction is essentially needed. gait authentication is one of the viable solutions since gait is the signature of the way humans walk, and the analysis can be done passively without any user interactions. Several breakthroughs in terms of model accuracy and efficiency were reported across several state-of-the-art papers. For example, DeepSense reported the accuracy of 0.942±0.032 in Human Activity Recognition and 0.997±0.001 in User Identification. Although there have been research focusing on gait-analysis recently, there has not been a stan- dardized way to define proper testing workflow and techniques that are required to ensure the correctness and efficiency of gait application system, especially when it is done in production scale. This thesis will present a general workflow of Machine Learning (ML) system testing in gait au- thentication using V-model, as well as identifying the areas and components that requires testing, including data testing and performance testing in each ML-related components. This thesis will also suggest some adversarial cases that the model can fail to predict. Traditional testing technique such as differential testing will also be introduced as a testing candidate for gait segmentation. In addition, several metrics and testing ideas will also be suggested and experimented. At last, some interesting findings will be reported in the experimental results section, and some areas for further future work will also be mentioned.

Tropes are storytelling devices or conventions that can be found in storytelling media, for example in movies. DBTropes is an RDF-dataset with media-trope connections extracted from Tv Tropes, which is a community-edited wiki listing tropes for various creative works. This study investigates whether the tropes of films can be used to cluster similar movies. First, we extracted film-trope connections from the DBTropes dataset. We then took four samples from the dataset, three for clustering films, and one for clustering tropes. We used the film–trope connections to calculate euclidean and cosine distances between movies and for the last sample between tropes. Then we clustered the samples with hierarchical clustering using complete and ward linkage. Our results show that hierarchical clustering can group similar films together using this dataset. For calculating distances the cosine distance method works significantly better than euclidean distance. Both hierarchical clustering methods complete and ward work well. It depends on the chosen sample, which of them results in a clearer and more interpretable output. We conclude that the data works well for clustering similar films or similar tropes.