Multiple case study in seven European countries regarding culture-sensitive classroom quality assessment
- Publisher: EU CARE project
This report presents the findings of a multiple case study, conducted in seven European countries to examine common and culturally differing aspects of curriculum, pedagogy, and quality of Early Childhood Education and Care (ECEC) provisions in Europe. This multiple case study involved intensive data collection on structural characteristics, process quality, implemented curricula and pedagogical approaches in four ECEC centers in each of the seven countries that were considered examples of ‘good practice’ by national experts. A multi-method approach was used to obtain a comprehensive overview of the different aspects of quality in classrooms for 0-3 and 3-6-years-old children. Video recordings were made of four common situations in ECEC centers, i.e. play, mealtime, creative activities and educational/emerging academic activities, which were used to evaluate process quality with a standard observational tool, namely the CLASS Toddler and CLASS Pre-K and to analyse occurring educational dialogues in-depth. The CLASS was chosen as an example of a well- developed, theory-based standard observation instrument that is currently widely used in several countries in different continents. In addition, educator reports were used to collect information on structural educator, classroom and center characteristics as well as information on the curriculum of the provision of different types of activities focusing on (pretend) play, self-regulation and pre- academic activities, including language, literacy, math, and science activities. Finally, information on educator’s beliefs and perspectives on classroom process quality was collected through personal interviews and focus group discussions with professionals in all participating countries. A total of 28 ECEC centers (14 centers for 0-3-year-olds, 14 centers for 3-6-year-olds) participated in the case study, involving in total 77 educators (of whom 41 worked in 0–3 classrooms). Videos were made of four common activity settings in ECEC (1) play, (2) mealtime, (3) educational/emerging academic activities, and (4) creative activities to increase comparability across countries, resulting in a total number of 62 videos for 0–3 classrooms and 62 videos for 3–6 classrooms (total number of 124 videos). The videos were coded using the CLASS Toddler and Pre-K versions by two experienced coders (from Finland and Portugal) and 25% of the data (i.e. one video per center) was double coded by an experienced coder from another country (the Netherlands) revealing good inter-observer reliability. The results based on the video data showed that the emotional support and classroom organization was in the high range, whereas the instructional support was in the mid range in this selective sample of good centers. This pattern reflects the general pattern found in ECEC classrooms, but with somewhat higher average scores than previous studies have found that used the CLASS, reflecting that, indeed, ‘good practices’ were selected for this study. The overall high level of process quality also indicated that what was thought good practice in one country was by-and-large also considered good practice in another country. However, there was also considerable variation in the quality assessments that could be attributed to the type of activity setting, group size (small vs. large group) and arrangement and to constellations of structural characteristics of the participating centres. In 0-3 classrooms play and educational/emerging academic activities provided the best opportunities for children to be engaged in higher quality processes, both with regard to emotional support and support for learning and development from educators. In 3-6 classrooms educational/emerging academic activities also showed the highest quality in both domains, but play situations now showed somewhat lower quality in instructional aspects. The difference might emerge from the different role of educators in children’s play: in 0-3 classrooms play was more often actively guided and facilitated by educators, whereas in 3-6 classrooms educators tended to take a monitoring role or not to be present in play situations. This finding might reflect in general an increased reliance on children’s play skills and putting more emphasis on developing children’s authonomy and peer relations via play as children grow older. Moreover, process quality was higher during small group activities compared to whole group activities, which was particularly evident for the dimensions regard for children’s perspectives, quality of feedback and language modelling. The content of the activity was also associated with process quality. Process quality was rated higher for example during science activities than during other educational/emerging academic activities. It appeared that science activities mostly concerned hands-on activities which, on average, were provided in smaller groups compared to language and literacy activities that were more often provided in the whole group and included activities such as circle time talk, shared reading and singing songs. Educators reported on the curriculum activities and children’s behavior that are seen as important for children’s development, in particular pretend play and self-regulation, and different types of pre- academic activities, including language literacy, math, and science activities. There appeared to be different patterns for 0-3 and 3-6 classrooms, with an emphasis on the provision of self-regulation and pre-academic activities for older children. However, there appeared to be differences between centers in different countries as well, likely reflecting variation in pedagogical traditions. On average, there seemed to be a stronger focus on language and math activities than on literacy and science activities, in both 0–3 and 3–6 classrooms. When distinguishing between different types of curricula it appeared that a balanced curriculum with roughly equal emphasis on play, self-regulation and pre- academic activities was related to the highest observed process quality. A predominant orientation on play in 3-6-years-old classrooms, at the expense of other types of activities, appeared to be related to lower instructional support for children’s learning although emotional support and classroom organization were in high level also in these classrooms. This point to the importance of having a curriculum with a good balance between different types of activities to support children’s holistic development. There was considerable variation in structural quality (groups size, ration) across centers, but different combinations of characteristics together with children’s age range, rather than single aspects, appeared to be related to higher observed process quality and to the implementation of a balanced curriculum. Moreover, both a favourable group size and a favourable children-to-staff ratio were found to be related to higher process quality, although not in combination, which can be explained by the choices educators make in preparing and organizing the day and the activities they provide to children. Based on our field notes in larger classrooms educators provided more activities in smaller groups throughout the day. Altogether, the findings indicate that a smaller group size with fewer educators or a larger group size with more educators were both related to higher quality and a more balanced curriculum. Other quality aspects included opportunities for additional in-service training, professional development activities provided at the center and the overall organizational climate in the center, which were all found to be important for process quality and curriculum emphasis. Additional in-service training with longer work experience was related to higher process quality in 0–3 classrooms and to a balanced implemented curriculum, which in turn was related to the highest process quality. Also opportunities for continuous professional development in the center with high organizational climate, including team meetings to discuss the developmental and educational goals of working with children, coaching, and using collegial observation and feedback to improve practice, was related to high observed process quality and a stronger emphasis on the provision of self-regulation and educational/emergent academic activities compared to other centers. These results were strongest when educators also evaluated the overall organizational climate of their center higher in terms of collegiality, supportive supervision, joint decision-making and clearly defined goals based on a shared mission and orientation. For the in-depth investigation of educational dialogues, the recorded play and educational/emerging academic activities of the 3-6 classrooms were analysed using a qualitative content analysis. Educational dialogues are considered a specific form of collective, reciprocal, and purposeful interactions in which there are extended verbal exchanges between the educator and children involving questioning, listening to each other and sharing of different ideas and points of view. In total, 8 episodes of educational dialogues were identified out of 28 video recordings, which mostly concerned educational/emerging academic activities in both small and large group settings, mainly addressing topics of science and math (5 out of 8). The remaining educational dialogues were identified in play situations. The educational dialogues that were identified in educational/emerging academic activities were more likely to be educator-initiated whereas the educational dialogues that emerged in play were initiated by children. Not all children in the group were equally actively taking part on the educational dialogue. The number of children actively contributing to educational dialogues ranged from 2 to 8 children per episode and the proportion of actively engaged children was higher in small groups compared to large groups. The educator’s role in the dialogues varied from a more leading role to a role as facilitator. Children were more likely to engage in a dialogue when the topic was familiar, related to their personal experiences and when hands-on materials or concrete examples were used. The videos in which educational dialogues were identified were also rated higher on the CLASS Pre-K dimensions Concept development, Quality of feedback, and Language modelling, attesting to the validity of the concepts measured with the CLASS as a process quality assessment instrument chosen to use in this multiple case study. The in-depth analysis of educational dialogues provided more detailed information on how back-and-forth exchanges between educators and children evolve, and on the specific strategies educators use to initiate or maintain the educational dialogue: The verbal interaction was often structured around educators asking questions and children providing answers, but during educational dialogues children were adding actively new themes to the topic and on few occasions building a chain of reasoning independently. Educators enhanced educational dialogue by validating children’s comments and by allowing the discussion to follow their initiations. By asking for children’s opinions and by using open questions, children were better able to contribute to educational dialogue. In our culture sensitive analysis there appeared to be a high level of agreement among professionals across countries about what constitutes high process quality. A group of 84 professionals from at least one center in each country participated in focus group discussions or in personal interviews to investigate their values and beliefs regarding classroom quality and to discuss their reflections on their own practices within a video cued situations. The professionals from 6 European countries mentioned three main goals of ECEC: (1) supporting children’s autonomy, (2) creating a sense of belonging, and (3) fostering children’s learning. There was wide consensus about the importance of a warm, positive classroom with sensitive educators adopting a child-centered approach which can support children’s learning. These aspects of quality were, generally, found to be well reflected in the standard assessment tool used in the current study for evaluating process quality, that is, the CLASS. However, the European professionals also strongly valued belonging to a group and being part of a community, the possibility to establish and develop peer relations, and a focus on broad developmental goals by striking a balance between ‘soft’ and ‘hard’ skills. These aspects were found to be less well reflected in the CLASS, which is more focused on dyadic adult-child relationships and puts less emphasis on peer relations and peer learning. Promoting a critical cultural approach to evaluation tools means also ceasing to consider the relationship between the tools and the services they evaluate only in a top-down, unidirectional way. Assessment and validation-adaptation processes can benefit from a reversed perspective that involves professionals in a reflective experience and an intercultural dialogue supported by and with the instruments. It offers educators an enriching opportunity to express the definitions of quality underlying their practices; to acquire a deeper awareness of them; to compare and even intentionally contaminate their local theories with values embedded in the instrument. It can therefore foster professional development and reflection and, consequently, improve quality. Altogether, the findings indicate a European perspective on classroom process quality that is not fully captured by standard quality assessment tools that were developed in other cultural, more individualistic, contexts, such as the United States. This calls for extension of existing tools or for development of new tools that can capture the European perspective. Recommendations 1) A balanced curriculum which focus on broad developmental goals by striking a balance between ‘soft’ and ‘hard’ skills in child development can be considered to be basis for the high process quality in classroom practices that influence children’s holistic development and learning. 2) Providing more small group activities can be an effective way to combine a more child- centered approach with stimulation of children’s deeper learning and development. Incorporating small group activities into the daily routines can be beneficial in terms of emotional and instructional process quality, meaning that if the overall group size is not favourable, as long as educators use opportunities for the provision of activities in smaller groups balancing whole group and small group activities during the day can support the process quality of activities. 3) In line with a stronger focus on collaborative and peer learning, the use of educational dialogues seems a good way to integrate child-centeredness with the stimulation of children’s cognitive and language development from a collective, group-based perspective. Increasing educator’s knowledge on educational dialogues and how to incorporate them into daily activities can enhance process quality and increase children’s involvement in activities, thus making these experiences more meaningful. 4) The provision of science activities turned out to be related to the highest process quality, yet given the least emphasis in current ECEC curricula according to educators’ self-reports. Science activities, including exploration and discovery while using hands-on materials, provide ample opportunities for reflection and discussion and educational dialogues, and can facilitate deeper understanding, promote children’s reasoning and thinking skills, and elicit complex language use, while allowing children initiative and self-determination. 5) Current widely-used standard observation instruments to assess quality in ECEC, for 0-3 and 3-6-years-old , such as the CLASS Toddler and CLASS Pre-K, provide a framework to assess quality, but need to be complemented by observation tools that (a) address educators’ group-sensitivity and strategies to strengthen group-belongingness, peer-interaction and peer-learning, (b) assess the flexible use of subgroup arrangements within the larger group to provide more guided small group work, (c) focus more specifically on the occurrence of educational dialogues, (d) evaluate to what extent social-emotional and personal ‘soft’ skills are fostered, such as self-regulation, problem-solving, creativity, collaboration and citizenship next to traditional ‘hard’ academic skills, and (e) determine to what extent inclusiveness and positive attitudes towards diversity are promoted. It is recommended to initiate the development of additional observation and self-evaluation tools that build on instruments such as the CLASS, but are extended as outlined here to serve the goals of European ECEC better. 6) In view of enhancing process quality in ECEC centers a promising way seems to be providing possibilities for continuous professional development for educators in the centers. This seems especially effective when embedded within an overall supportive organizational climate. To create opportunities for educators’ continuous professional development together with the policy level support for centers will benefit the quality of ECEC in Europe.