Browsing by Subject "reliabiliteetti"

Objectives. Early mathematical skills predict future academic success. The development of mathematical skills during primary school years tends to be weaker in children learning Finnish as a second language as compared to native speakers. However, these skills beginto develop early in life, with mathematical vocabulary playing a crucial role. Given that learning the Finnish language is a time-consuming process, there is a growing demand for valid and reliable assessment methods for mathematical skills and vocabulary that do not necessitate speech production in the second language. The SMST Picture Vocabulary Test aims to assess the recognition of words related to mathematics. The first objective of this study is to analyze the reliability and validity of this test, while the second is to examine the role of age, home language, gender, and the duration of participation in Early Childhood Education (ECE) in relation to knowledge of vocabulary. Methods. The data was gathered from children who were Finnish language learners aged 5–7 years (N=100) attending public ECE centers in the capital area. The reliability of the test was evaluated through internal consistency analyses. Additionally, known-group validity and concurrent validity assessments were conducted. Knowledge of mathematical vocabulary was examined through test results, with additional background data collected from teachers. Teachers evaluated the overall language and mathematical skills of the children. The study employed a quantitative approach with statistical analysis methods using the IBM SPSS program. Main results and conclusions. The study suggests that the SMST Picture Vocabulary Test demonstrates weak reliability, yet moderate validity, to be used for children learning Finnish as a second language. Concurrent validity analysis affirms the test to be primarily a vocabulary knowledge assessment tool. The SMST test demonstrated the ability to differentiate between two pre-established groups (Finnish language learners and native speakers) in mathematical skills through knowledge of mathematical vocabulary, thereby supporting the known-group validity of the test. The duration of participation in ECE emerged as the strongest indicator for mathematical vocabulary test results, followed by age, predicting 29 % of the vocabulary knowledge. Gender differences were found, with the results favoring girls. Home language showed no significant relation to competency in math vocabulary. The findings from this study pave the way for further research into the assessment of mathematical vocabulary knowledge among Finnish language learners, with a particular focus on understanding the factors influencing this knowledge.

Objectives. Progressive Feedback in Early Childhood Education is a project for development of early childhood education in 12 Finnish municipalities. The data in the project is collected through child observation and staff and manager self-evaluation. The objective of this study was to assess the inter-rater agreement of the project’s observation method. Inter-rater agreement is a common way of assessing the reliability and validity of a method that is based on observational data collection. Methods. This study used paired observation data that was collected through the project. The data consisted of observations conducted by 9 different pairs of observers, generating 411 paired observations in total. The data was coded as quantitative data and quantitative methods were used for the reliability analysis. Cohen’s kappa was used to calculate the level of agreement for the rating system scales and for all the paired observers using the rating system. Cohen’s kappa calculates the level of agreement among raters and accounts for agreement based on chance. Cross-tabulation was used to examine the distribution of agreement and disagreement of the observational data on the rating scales. Results. Based on labels assigned by Landis and Koch (1977) the level of agreement for one rating scale was almost perfect, for two rating scales substantial, for three rating scales moderate and for three rating scales fair. The disagreement was notable on certain scales and there were clear differences between scales and raters. Based on the results and sources the reliability of the observational method should be improved in the future. The improvement should focus on increasing the training of the observers, evaluating the objectives of the observation and calibrating the observation rating scales. Using Cohen’s kappa was not problem-free, so in future research multiple methods should be used to ensure balanced results. Further studies include ways to follow up on the reliability of the project’s observation method and the possibilities of generalizability theory and argument based validity as part of the research on reliability and validity of observation based quality assessment methods.