Skip to main content
Login | Suomeksi | På svenska | In English

Browsing by Subject "reinforcement learning"

Sort by: Order: Results:

  • A Reinforcement Learning Application for Portfolio Optimization in the Stock Market 
    Huertas, Andres (2020)
    Investment funds are continuously looking for new technologies and ideas to enhance their results. Lately, with the success observed in other fields, wealth managers are taking a closes look at machine learning methods. Even if the use of ML is not entirely new in finance, leveraging new techniques has proved to be challenging and few funds succeed in doing so. The present work explores de usage of reinforcement learning algorithms for portfolio management for the stock market. It is well known the stochastic nature of stock and aiming to predict the market is unrealistic; nevertheless, the question of how to use machine learning to find useful patterns in the data that enable small market edges, remains open. Based on the ideas of reinforcement learning, a portfolio optimization approach is proposed. RL agents are trained to trade in a stock exchange, using portfolio returns as rewards for their RL optimization problem, thus seeking optimal resource allocation. For this purpose, a set of 68 stock tickers in the Frankfurt exchange market was selected, and two RL methods applied, namely Advantage Actor-Critic(A2C) and Proximal Policy Optimization (PPO). Their performance was compared against three commonly traded ETFs (exchange-traded funds) to asses the algorithm's ability to generate returns compared to real-life investments. Both algorithms were able to achieve positive returns in a year of testing( 5.4\% and 9.3\% for A2C and PPO respectively, a European ETF (VGK, Vanguard FTSE Europe Index Fund) for the same period, reported 9.0\% returns) as well as healthy risk-to-returns ratios. The results do not aim to be financial advice or trading strategies, but rather explore the potential of RL for studying small to medium size stock portfolios.
  • Simulation Environment for Reinforcement Learning Control of Acoustic Levitation 
    Kinnunen, Anniina (2023)
    Acoustic levitation refers to the levitation of particles using sound waves. It can be performed on a phased array of transducers (levitator) where the transducers create the sound waves. The levitator device can be controlled by altering the values of the control parameters of the transducers. In this thesis, we present an automatic approach for finding the control parameter values using a branch of machine learning called reinforcement learning. The main goal is to make specifying the control parameter values for complex levitation tasks easier. We first build a simulation environment for the learning task, and then perform several experiments in the environment and compare two model-based reinforcement learning algorithms: Covariance Matrix Adaptation Evolution Strategy and a baseline strategy based on random actions. The experiments are related to optimizing the hyperparameters of reinforcement learning, testing the algorithm with limitations that using a real levitator would bring, and solving different levitation tasks. The results of the experiments show that the simulation environment enables controlling levitators with model-based algorithms. Furthermore, both of the algorithms that were used were able to solve various control problems, such as lifting a particle and moving a particle in circles.
  • Using Reinforcement Learning and Task Decomposition for Learning to Play Doom 
    Kropotov, Ivan (2020)
    Reinforcement learning (RL) is a basic machine learning method, which has recently gained in popularity. As the field matures, RL methods are being applied on progressively more complex problems. This leads to need to design increasingly more complicated models, which are difficult to train and apply in practice. This thesis explores one potential way of solving the problem with large and slow RL models, which is using a modular approach to build the models. The idea behind this approach is to decompose the main task into smaller subtasks and have separate modules each of which concentrates on solving a single subtask. In more detail, the proposed agent will be built using the Q-decomposition algorithm, which provides a simple and robust algorithm for building modular RL agents. The problem we use as an example of usefulness of the modular approach is a simplified version of the video game Doom and we design a RL agent that learns to play it. The empirical results indicate that the proposed model is able to learn to play the simplified version of Doom on a reasonable level, but not perfectly. Additionally, we show that the proposed model might suffer from usage of too simple models for solving the subtasks. Nevertheless, taken as a whole the results and the experience of designing the agent show that the modular approach for RL is a promising way forward and warrants further exploration.

Yhteystiedot

HELSINGIN YLIOPISTO