Browsing by Subject "Operator recurrent neural network"

In my mathematics master's thesis we dive into the wave equation and its inverse problem and try to solve it with neural networks we create in Python. There are different types of artificial neural networks. The basic structure is that there are several layers and each layer contains neurons. The input goes to all the neurons in the first layer, the neurons do calculations and send the output to all the neurons in the next layer. In this way, the input data goes through all the neurons and changes and the last layer outputs this changed data. In our code we use operator recurrent neural network. The biggest difference between the standard neural network and the operator recurrent neural network is, that instead of matrix-vector multiplications we use matrix-matrix multiplications in the neurons. We teach the neural networks for a certain number of times with training data and then we check how well they learned with test data. It is up to us how long and how far we teach the networks. Easy criterion would be when a neural network has learned the inversion completely, but it takes a lot of time and might never happen. So we settle for a situation when the error, the difference between the actual inverse and the inverse calculated by the neural network, is as small as we wanted. We start the coding by studying the matrix inversion. The idea is to teach the neural networks to do the inversion of a given 2-by-2 real valued matrix. First we deal with networks that don't have the activation function ReLU in their layers. We seek a learning rate, a small constant, that speeds up the learning of a neural network the most. After this we start comparing networks that don't have ReLU layers to networks that do have ReLU layers. The hypothesis is that ReLU assists neural networks to learn quicker. After this we study the one-dimensional wave equation and we calculate its general form of solution. The inverse problem of the wave equation is to recover wave speed c(x) when we have boundary terms. Inverse problems in general do not often have a unique solution, but in real life if we have measured data and some additional a priori information, it is possible to find a unique solution. In our case we do know that the inverse problem of the wave equation has a unique solution. When coding the inverse problem of the wave equation we use the same approach as with the matrix inversion. First we seek the best learning rate and then start to compare neural networks with and without ReLU layers. The hypothesis once again is that ReLU supports the learning of the neural networks. This turns out to be true and happens more clearly with wave equation than with matrix inversion. All the teaching was run on one computer. There is a chance to get even better results if a more powerful computer is used.