Browsing by Subject "mittari"

Objectives. The factors of early childhood education (ECE) quality and indicators describing them were published by Finnish Education Evaluation Centre (FINEEC) in October 2019. Act on Early Childhood Education and Care (540/2018) obligates organizer to evaluate ECE. Indicators themselves are not assessment tools. The organizers should draw up more detailed criteria for evaluation based on the indicators. In Finnish ECE, previous research has shown a noticeable variation in evaluation practices. Progressive Feedback (PF) is a method for evaluation and development of early childhood education (blogs.helsinki.fi/reunamo/). One of the instruments in PF is a learning environment evaluation form. The objective of this study is first to examine how well does the learning environment evaluation form cover the evaluation of the indicators of process-related quality (FINEEC). Based on the examination the second objective is to create new criteria that covers all the indicators. The third objective is to assess the reliability of the developed learning environment evaluation form. Methods. The method used in the first objectives was theory-based content analysis. Data for the reliability analysis was collected form Vantaa, Kouvola and Sipoo, from groups with two teachers. The data consisted of the evaluations of 42 teachers, 21 pairs. This study used paired evaluation data to evaluate inter-rater reliability with Intraclass correlation coefficient (ICC). Results. The learning environment evaluation form (PF) covered eight indicators well and ten mostly or partly. The evaluation of nine indicators was not covered. Based on Koo & Li (2016) the ICC values of 59.6% of the criteria indicated poor reliability. The ICC values of the remaining 40.4% of the criteria indicated moderate, good or excellent reliability. Based on the results and sources the criteria of the evaluation form should be critically examined and developed. Rather than concrete matters, most of the criteria measure subjective experiences which can be difficult to assess. Criteria measuring more than one thing as well as linguistically complicated criteria had more often ICC values indicating poor reliability. Relatively short answering time might have affected on poor reliability. Based on this study the evaluation of process-related factors and indicators describing them with one evaluation form like this is challenging or impossible. Multimethod evaluation could be a more reliable way.