Browsing by Author "Ahvenharju, Panu"

The topic of the thesis is integrating Natural Language Processing (NLP) and Computer-assisted Language Learning (CALL) into teacher-led Spanish instruction. The aim is to present a development process and a CALL application to be used to study learning results. The study seeks answers to questions on how an NLP-based CALL application can be used to investigate learning, and how its usage rate and usage affect learning outcomes. Also, the focus is on usability, asking how usable the students evaluate the application to be, and what kind of open feedback they give for it. 108 secondary school students and four teachers from the Helsinki Metropolitan region participated in the study, where a gamified application creates a competitive setting between five teaching groups. The students use the application to solve textbook-based cloze exercises that are generated using a combination of a neural language model and a rule-based exercise creation. The vocabulary tests measure learning by selecting test words according to the usage analytics so that they are from outside the cloze fields of exercise sentences. The students who used the application were divided into two groups: those (N=26) who encountered the test words in the application and those (N=31) who did not. The results are being compared to those in the control group (N=8) who did not use the application. The results show that the group encountering the test words performed 11.39 percentage points better than the control group. Interestingly, the students who did not encounter the words performed 25.21 percentage points better in tests than the control group. Despite the positive results, statistical analysis revealed a significant relationship only between usage rate and encountering the test words, not between the test words and the vocabulary test results. This may be explained by the different sizes of the groups, the random way how the application selected exercises, and the fact that the students did not encounter the words often enough. The method requires many enhancements before utilising it on a larger scale. The students evaluated the application's usability to be good, and they left 18 open feedback responses, which were mostly positive.