Skip to main content
Login | Suomeksi | På svenska | In English

Assessing text readability and quality with language models

Show simple item record

dc.date.accessioned 2020-03-19T14:32:29Z
dc.date.available 2020-03-19T14:32:29Z
dc.date.issued 2020-03-19
dc.identifier.uri http://hdl.handle.net/123456789/27581
dc.title Assessing text readability and quality with language models en
ethesis.discipline none und
ethesis.department none und
ethesis.faculty Matemaattis-luonnontieteellinen tiedekunta fi
ethesis.faculty Faculty of Science en
ethesis.faculty Matematisk-naturvetenskapliga fakulteten sv
ethesis.faculty.URI http://data.hulib.helsinki.fi/id/8d59209f-6614-4edd-9744-1ebdaf1d13ca
ethesis.university.URI http://data.hulib.helsinki.fi/id/50ae46d8-7ba9-4821-877c-c994c78b0d97
ethesis.university Helsingin yliopisto fi
ethesis.university University of Helsinki en
ethesis.university Helsingfors universitet sv
dct.creator Liu, Yang
dct.issued 2020
dct.language.ISO639-2 eng
dct.abstract Automatic readability assessment is considered as a challenging task in NLP due to its high degree of subjectivity. The majority prior work in assessing readability has focused on identifying the level of education necessary for comprehension without the consideration of text quality, i.e., how naturally the text flows from the perspective of a native speaker. Therefore, in this thesis, we aim to use language models, trained on well-written prose, to measure not only text readability in terms of comprehension but text quality. In this thesis, we developed two word-level metrics based on the concordance of article text with predictions made using language models to assess text readability and quality. We evaluate both metrics on a set of corpora used for readability assessment or automated essay scoring (AES) by measuring the correlation between scores assigned by our metrics and human raters. According to the experimental results, our metrics are strongly correlated with text quality, which achieve 0.4-0.6 correlations on 7 out of 9 datasets. We demonstrate that GPT-2 surpasses other language models, including the bigram model, LSTM, and bidirectional LSTM, on the task of estimating text quality in a zero-shot setting, and GPT-2 perplexity-based measure is a reasonable indicator for text quality evaluation. en
dct.language en
ethesis.isPublicationLicenseAccepted true
ethesis.language.URI http://data.hulib.helsinki.fi/id/languages/eng
ethesis.language English en
ethesis.language englanti fi
ethesis.language engelska sv
ethesis.thesistype pro gradu -tutkielmat fi
ethesis.thesistype master's thesis en
ethesis.thesistype pro gradu-avhandlingar sv
ethesis.thesistype.URI http://data.hulib.helsinki.fi/id/thesistypes/mastersthesis
dct.identifier.ethesis E-thesisID:c33d92af-b73e-4e31-89da-b5e379c8a216
dct.identifier.urn URN:NBN:fi:hulib-202003191584
dc.type.dcmitype Text
ethesis.facultystudyline Algoritmit fi
ethesis.facultystudyline Algorithms en
ethesis.facultystudyline Algoritmer sv
ethesis.facultystudyline.URI http://data.hulib.helsinki.fi/id/SH50_083
ethesis.mastersdegreeprogram Tietojenkäsittelytieteen maisteriohjelma fi
ethesis.mastersdegreeprogram Master's Programme in Computer Science en
ethesis.mastersdegreeprogram Magisterprogrammet i datavetenskap sv
ethesis.mastersdegreeprogram.URI http://data.hulib.helsinki.fi/id/MH50_009

Files in this item

Files Size Format View
Yang_s_MSc.pdf 1.179Mb PDF

This item appears in the following Collection(s)

Show simple item record