Introduction

The UN Sustainable Development Goal 4 Target 4.2 represents a global commitment to increasing access to quality early childhood education (ECE) globally (United Nations, 2015). Scholars have reached some consensus on critical elements of ECE classrooms needed to improve learning, including emotional support, instructional support, and positive classroom management (Hamre et al., 2013). Yet these measures don’t explicitly consider children’s engagement in the learning process. Indeed, theories of school engagement and self-regulated learning emphasize the importance of children’s motivation and active engagement in the learning process (Pintrich, 2000; Wang & Eccles, 2012), yet these theories have been applied primarily to older children.

As global efforts grow to increase quality education and learning, it has been posited that poor learning levels may be due in part to the prevalence of models of education in much of the world that emphasize rote instruction and memorization (Banerjee et al., 2017; Hirsh-Pasek et al. 2020; Mehta & Fine, 2019a; Rogoff, 2003). Such pedagogical approaches have been also extended downward from primary to preprimary settings (Agbenyega, 2018). An alternate pedagogical model promotes engaged and interactive learning, positing that when children are engaged, they learn more deeply and academic achievement improves; as learning is self-sustained, children are intrinsically motivated (Reyes et al., 2012).

There is overlap between engaged learning and play. In early childhood, there is a longstanding consensus that play contributes to learning and development in early childhood and beyond (e.g., Bodrova et al., 2013; Elkind, 2007; Montessori, 1973), partly because it promotes children’s engagement (Hirsch-Pasek et al., 2015). The concept of engagement can be more precisely defined than the related concept of learning through play and has a stronger prima facie case for being critical for learning; further, it is recognized as important by teachers and caregivers across cultures and contexts (Mehta & Fine, 2019). Understanding how engagement can be supported in classrooms may be critical for operationalizing the promise of playful learning. However, there are currently no systematic approaches to measure the opportunities of support for engagement in learning that could inform classroom practice. This is particularly true in low- and middle-income countries, where discussions about the nature of quality ECE have not included an emphasis on children’s engagement (Chen & Wolf, 2021).

In this study, we introduce a classroom observation tool designed to assess support for engagement in learning opportunities in preprimary classrooms. We expand on existing observation tools by focusing all items specifically on teacher–child interactions that promote engagement in learning. We take an exploratory approach to examine: (1) the underlying domains of classrooms measured by the observation tool; and (2) its construct validity in Ghanaian preprimary school classrooms. We consider the utility of the emergent constructs in ECE classrooms and the results inform further refinement and development of the concepts and the tool.

Current Conceptualizations of Teacher–Child Interactions

Classroom quality is a complex and multifaceted construct, but it is generally separated into two components: structural and process quality. Structural quality indicators, such as infrastructure, staff qualifications, and materials, are important to the extent that they enable better processes within classrooms (Slot, 2018). Elements of process quality—that is, the daily interactions that children experience that encompass academic, social, and emotional supports—most directly drive learning and development. Domains of process quality across diverse contexts generally focus on emotional support, instructional support, and classroom management (e.g., Hamre et al., 2013; Raikes et al., 2020; Wolf et al., 2018). Behind the concept of process quality is a long-standing theoretical base on the ways social-settings affect individuals (i.e., through daily interactions in those settings; Tseng & Seidman, 2007), with a particular focus on classroom settings (Cohen et al., 2003; Pianta & Hamre, 2009) and students’ daily experiences with teachers and peers (Seidman & Tseng, 2011), and that these interactions are culturally bound (Stigler et al., 2000).

Despite tens of studies affirming the structure of classroom process quality in preprimary and primary classrooms (e.g., Hamre et al., 2013; Leyva et al., 2015), our understanding of classroom experiences that causally improve learning is quite limited. Correlational research shows positive associations between process quality and child outcomes in ECE (e.g., Hamre et al., 2014; Mashburn et al., 2008). Yet findings are mixed regarding the consistency of these results across diverse settings, outcomes, and study designs, with many high-quality studies and meta-analyses showing weak—or even null—associations between quality and outcomes (Burchinal, 2018; McDoniel et al., 2022; Watts et al., 2021). These findings suggest that potentially important domains of process quality are not currently well-measured. An explicit emphasis on engagement in learning may be one critical missing aspect.

Engagement in Learning

Children learn through hands-on experiences, particularly in the preschool years. Hands-on play-based learning includes active, socially engaged, meaningful, iterative, and joyful forms of learning (Zosh et al., 2018); sustained engagement is critical for all of these and results in children engaging more deeply for longer (Zosh et al., 2018). The concept of sustained engagement maps onto existing characterizations of engagement in the literature. It is a type of engagement in learning, which refers to “students’ psychological state of activity that affords them to feel activated…and be absorbed during learning activities” (Wong and Liem, (2022). Learning engagement is distinct from school engagement, which refers to a students’ state of connection with the school-community. Engagement, as we define it, includes two of the three dimensions commonly identified in the literature (Fredricks et al., 2004; Wang & Eccles, 2013): cognitive—“the extent to which students are absorbed during learning activities” (Wong & Liem, 2022)—and affective—“the extent to which students feel activated during learning activities” (Wong & Liem, 2022). The third dimension of learning engagement—behavioural engagement, defined as “the extent to which students intentionally exert effort during learning activities” (Wong & Liem, 2022)—is not included in its definition, as we consider engagement to be effortless and sustained. Learning engagement has been studied primarily in middle childhood and adolescence (e.g., Wang & Eccles, 2012). In this study, we investigate how the concept can be applied in early childhood.

Engagement in early childhood may help explain how experiences can result in deeper and wider learning. Experts have suggested that instruction that facilitates such engaged learning advances a breadth of “twenty-first century skills” critical for learning that have been called the “6Cs”: collaboration, communication, content, critical thinking, creative innovation, and confidence (Golinkoff & Hirsh-Pasek, 2016; Hirsh-Pasek et al., 2020). It is plausible that engagement is critical for learning in all contexts, although little research has been conducted on the concept in LMICs. Based on the limited uptake of learner-centered pedagogy in sub-Saharan Africa (Schweisfurth, 2011), we might expect intrinsically motivated engagement to be less well supported in schools on the continent.

The PLAY ECE Observational Tool

The Playful Learning Across the Years ECE observational tool (hereafter “PLAY ECE”) aims to measure opportunities that foster engagement in early learning classrooms. The tool captures the interactions between a caregiver and students as initiated by the caregiver (similar to other classroom observation tools) that support engagement. With a focus on the setting-level, the tool does not directly assess child behavior.

Based on a global literature review of measurement of teacher and caregiver-child interactions, review of frameworks for learning through play, and qualitative data collection in two LMICs (Jukes et al., 2022), teacher–child interactions lead to children’s engagement by supporting the following: connection to experience (personal and community experiences outside the classroom), exploration (of materials and concepts), problem-solving (supporting generation and iterative testing of informal hypotheses about the world), agency (autonomy in learning activity engagement), social connectedness (with peers and between students and teacher), and positive emotional climate (a dimension that we define similarly to existing general quality measures; Hamre et al., 2013).

Qualitative data was used to probe the understanding and experience of ‘play,’ ‘learning,’ and ‘engagement’ in some typical LMIC ECE settings. For example, we gauged what kinds of situations were common in some typical LMIC ECE settings through focus groups, surveys, and naturalistic observations which resulted in brief descriptions, accounts or episodes that teachers/carers would have been familiar with from these contexts. The data were used to refine the definition of six constructs derived from the literature and to generate items for the tool. We examine items of PLAY ECE across these six dimensions.

PLAY ECE relates to other measures of quality in several ways. First, it has a domain-general focus and is not subject specific. Second, PLAY ECE measures teacher–child interactions. However, unlike existing quality measures, PLAY ECE focuses only on specific teacher–child interactions that support engagement in learning. While existing classroom observation tools (e.g., TEACH ECE, Measuring Early Learning Environments (MELE)) do have indicators of general quality in teacher–child interactions, they do not have the specificity of constructs that the PLAY ECE does, such as multi-item scales for support for exploration or support for social connectedness. While there is overlap with previous tools (e.g., elements of emotional support which are central to engagement), PLAY ECE has an expanded focus on the domains specified in our theoretical framework (e.g., inclusion of new elements for social connectedness and positive climate designed explicitly for measuring interactions that are more culturally relevant in non-Western contexts). For example, items include a focus on how teachers promote children’s interest in each other’s lives and encouraging friendship and social acceptance amongst students.

Finally, PLAY ECE aims to measure interactions that promote a broad range of outcomes, including social-emotional and academic outcomes. While other domain-general classroom observation tools include some aspects of children’s engagement in learning, these items are usually not examined as a separate construct and generally provide stronger measures of aspects of the classroom related to emotional support, classroom management, and instructional quality. Interestingly, measures of instructional quality have at times shown less variation and be less predictive of child outcomes (Berlinski & Schady, 2015; Leyva et al., 2015). Understanding the utility of items that are solely focused on engagement in learning may be key to capturing variation in instructional support.

The Ghanaian Preprimary School Context

Ghana is a lower-middle-income country in West Africa that was one of the first on the continent to expand free, universal preprimary education for all children. In 2007, two years of preprimary education—called Kindergarten 1 (KG1; 4-year-olds) and Kindergarten 2 (KG2; 5-year-olds)—were added to the universal basic education system that had previously begun in grade 1. Ghana has among the highest ECE enrollment rates on the African continent, though the country still struggles with issues of poor educational quality as well as primary school-aged children being overenrolled in preprimary school (Ghana Ministry of Education, 2016; United Nations Children’s Fund, 2019). The KG curriculum emphasizes play and joy of learning, as well as creativity, prosocial skills, and active engagement (Ministry of Women & Children’s Affairs, 2004).

In the 2015–2016 school year, the Ghanaian Ministry of Education partnered with academic researchers and two local organizations to develop and evaluate a training model to improve the quality of preprimary school, focusing on the Greater Accra Region. The primary component of the Quality Preschool for Ghana (QP4G) school-randomized study was an in-service teacher training (TT) that aimed to increase child-centered activity-based learning opportunities, positive behavior management, and the quality of teacher–child interactions guided by the national preprimary school curriculum. The training focused on five areas: how children learn and developing a child-friendly environment, positive classroom management, integrating play and hands-on activities into language and literacy instruction, integrating play and hands-on activities into early numeracy instruction, and assessment and planning.

The first target was to transform classroom instruction from rote memorization of academic concepts to an activity-based approach that engages students in developing critical thinking skills. The training included concrete examples in which teachers could integrate activities and opportunities for exploration into teaching academic content, including making explicit connections to children’s daily lives. The second target was to reduce harsh and corporal punishment while improving positive behavior management by creating a child-friendly classroom environment, practicing proactive behavior management, and encouraging students to follow class rules and norms. The goal was to enhance pathways for learning by increasing emotional support and play-based learning to help children learn to self-regulate and develop executive function and social-emotional skills (Diamond et al., 2007).

A second treatment condition targeted parents through three Parent–Teacher Association meetings offered to all KG parents in schools participating the TT program. At each meeting, videos developed for the intervention were viewed followed by discussions led by district educational coordinators. The content included play-based learning, parents' roles in children's learning, and encouraging parent–teacher communication. QP4G was evaluated through a school-randomized trial where schools were randomly assigned to: (a) TT, (b) TT plus parental awareness (TTPA), and (c) control.

In the original evaluation study, both treatment conditions led to medium-sized reductions in teacher burnout and improved several dimensions of observed classroom quality (increased emotional support and positive behavior management). TT (but not TTPA) also increased support for student expression (Wolf et al., 2019b) as measured by a classroom observation measure assessing general process quality (in contrast to the PLAY ECE tool). Furthermore, both treatment conditions increased the number of developmentally appropriate activities teachers used. In addition, the TT condition, but not TTPA condition, led to small and sustained improvements in child outcomes one- and two-years (Wolf, 2019) and five years after treatment ended (Wolf et al., 2022).

Concurrent Validity of Classroom Quality Measures and Sensitivity to Intervention

In the development of a new observation tool, examining validity in terms of associations with a broad range of concepts and measures is critical. First, understanding how a newly developed tool to assess classroom quality and playful interactions is associated with previous, more widely used measures of quality is an important step. Even if a tool aims to measure a new domain of process quality, it is likely that there would be some correlations with other measures of quality (e.g., an overall Responsive Teaching latent factor; Hamre et al., 2014). The extent to which measures of support for engagement may bolster our understanding of the instructional quality of a classroom, and how it would correlate with other domains of process quality, is an open question, but small positive correlations are hypothesized.

Second, associations of a quality measure with teacher educational characteristics serves to establish a link between process and structural quality. Although links between structural and process quality appear in high-income country contexts (e.g., Wysłowska & Slot, 2020), they have generally found to be weak and somewhat inconsistent (Wolf et al., 2018); this link has not been extensively investigated in LMIC contexts. In Ghana, public-sector kindergarten teachers are required to have a minimum of a Diploma in Basic Education (2 years of coursework and one year as a student–teacher) obtained from an approved college of education (Asare & Nti, 2014), while there are no minimum requirements for private sector teachers. this Diploma used to require two years of coursework followed by one year of student teachingFootnote 1; thus, the level of required credentials is less than in high-income countries. It is possible that this training is critical to preparing teachers; but it is equally possible that it is not sufficient to strongly affect teaching quality. Previous research in Ghana has shown that education level and training in early childhood development do predict some elements of preprimary classroom quality (Wolf et al., 2018). Given the lack of directional evidence on training and education (e.g., Early et al., 2007), this analysis is considered exploratory and contributing to the scarce evidence-base on how ECE teacher characteristics predict elements of process quality.

There is also a literature on the stress of teaching and the role of professional well-being in shaping learning environments and teacher performance (see Darling-Hammond & Cook-Harvey, 2018, for a review). Challenging conditions for teachers in lower-income countries, including Ghana, can lead to a loss of motivation, and this may partially explain poor teaching performance and student learning outcomes, high rates of turnover and absenteeism, and misconduct (Bennell & Akyeampong, 2007; Lauwerier & Akkari, 2015). We thus also consider teachers’ psychological and professional well-being as potential predictors of practices supporting engaged learning, hypothesizing that better professional well-being would be associated with teachers providing more such opportunities.

Finally, a further form of validity of an observed quality measure is its sensitivity to impacts of an intervention focused on improving quality. We examine the experimental impacts of the QP4G quality improvement program for on the PLAY ECE measure.

The Current Study

We provide the first psychometric analysis of PLAY ECE administered in Ghanaian preprimary school classrooms. We draw on data from the QP4G project and re-analyze classroom observation data to answer the following questions:

  1. 1.

    What domains of support for engagement are measured by the PLAY ECE observational tool?

  2. 2.

    Are factors representing these domains sensitive to impacts of a teacher professional development program aimed at increasing child-centered, activity- and play-based learning opportunities for children?

  3. 3.

    What is the concurrent validity of these factors as measured by PLAY ECE? Specifically, do factors correlate with a) dimensions of a classroom observation tool used to assess general process quality, b) an index of teacher activity-based pedagogical practices, and c) teacher characteristics, including qualifications and well-being?

Our study provides the first rigorous analysis of a tool developed to assess support for engagement in learning in a low-resource ECE context.

Method

Sample and Procedures

Data for this project come from an impact evaluation conducted in the 2015–2016 school year in the Greater Accra Region of Ghana as part of the Quality Preschool for Ghana project (Wolf et al., 2019a, 2019b). The current study received approval from the [Blinded for Peer Review] (STUDY00021638). Six of the most disadvantaged districts (of 16) were selected in summer 2015. Within each district, schools were identified using the Ghana Education Service EMIS database, which listed all registered schools in the country. Schools were then randomly selected, stratified by district and public/private schools, resulting in 240 schools in total (118 public, 132 private). Every KG classroom was included, and 15 children were randomly selected from each classroom roster to participate in direct assessments (eight from KG1 and seven from KG2). Most schools had two KG teachers; two were randomly sampled for data collection for schools that had more than two teachers. Finally, schools were randomly assigned—stratified by district and public/private status—to one of three treatment conditions: control, TT, and TTPA.

In May–June of 2016 (end of the academic year), all teachers were videotaped teaching a lesson in their classroom for 30–60 min in the first half of the six-hour school day. Videos were recorded by trained enumerators who also administered surveys to the teachers. The content of the lessons varied and were reported by teachers to included: creative activities, environmental studies, literacy and language, mathematics, movement and dance/physical education, and “other.” At both fall and spring waves, direct assessments with children randomly selected from class rosters were also conducted by trained enumerators. There were 420 classrooms video-taped in the end-of-year wave that also included teacher survey data.

Measures

Classroom Observation Tool

The PLAY ECE observation tool is part of a suite of tools to measure support for engagement across home, center-based, and school-based settings, for children ranging in ages 0–12. In this study, we focus on the classroom-based observation tool developed for preprimary school children. There are 49 items developed to assess six underlying constructs: connection to experience, problem solving, exploration, agency, positive emotional climate, and social connectedness. Each item was scored on a 4-point scale with response options incorporating frequency, inclusivity, effectiveness, or time of each practice observed during the observation (0 = not observed, 1 = low quality…3 = high quality; see Table 1). Descriptive statistics of all PLAY items are shown in Appendix Table A1.

Table 1 Description of rating scale for the PLAY ECE observation tool
Full size table

Eleven local enumerators in Ghana participated in a fully remote observation training. The group generally consisted of individuals who had experience conducting data collection for education research studies. Training took place over 10 days and included bias training, training on the six hypothesized constructs, including item-by-item review with small/large group discussion using contextualized video examples, and finally, calibration. For calibration, each participant independently coded 3 videos that had been coded by master coders. Absolute agreement and ± 1 agreement rates with master codes were calculated with adequate results: average absolute agreement ranged from 64.1 to 72.7%; average ± 1 agreement ranged from 87.7–94.7%. Four enumerators reached an average absolute agreement of 70% or higher, six enumerators had an average absolute agreement of 68% or higher, and a final enumerator had an average absolute agreement of 64%. This final enumerator was not included in the study due to lower levels of inter-rater reliability.

TIPPS

The Teacher Instructional Practices and Processes System (TIPPS; Seidman et al., 2018; Wolf et al., 2018) is a classroom observation tool designed to assess teacher–child interactions reflecting general process quality in LMICs, designed specifically for low-resourced contexts. We used the TIPPS-ECE version that included minor adaptations for use in Ghana. Following analyses from Wolf et al. (2018), we used 15 of the 19 TIPPS items that measure three domains: Facilitating Deeper Learning (3 items; connecting lesson to teaching objectives, provides specific, high quality feedback, and uses scaffolding; α = 0.62), Supporting Student Expression (SSE; 4 items; considers student ideas and interests, encourages students to reason and problem solve, connects lesson to students’ daily lives, and models complex language; α = 0.70), and Emotional Support and Behavior Management (ESBM; 7 items, positive climate, negative climate, sensitivity and responsiveness, tone of voice, positive behavior management, provides consistent routines, student engagement in class activities; α = 0.70). See Wolf et al. (2018) for details on the analysis and concurrent validity of the three factors in this sample. Item scores were averaged within factors to create domain scores that ranged from 1 to 4, with higher scores indicating greater quality.

Developmentally Appropriate Practices Checklist

We used a researcher-developed curriculum checklist of instructional practices delivered in the classroom during the observation period (Wolf et al., 2019b). The checklist consisted of 13 activities that are described in Ghana’s national curriculum and were explicitly emphasized in the parent study’s teacher training intervention. Items targeted behavior management and instructional practice, and included: “Teacher praises children for positive behavior,” “Teacher threatens children with or used a cane on children at least once (reverse coded),” “Teacher explicitly reminds children of the class rules,” “Teacher uses a signal to gain children’s attention (e.g., drum beat, song, bell),” “Children are seated in a way that children can see each other’s faces (e.g., in a circle, or tables together in groups),” “Teacher uses one or multiple songs to facilitate learning at some point in the lesson,” and “There is an activity that facilitated the lesson objectives that involved manipulation of materials.” Each item was coded as either present in the video (a score of 1) or absent in the video (a score of 0), and overall scores represented item sums with a potential range of 0 to 13.

Teacher Characteristics

Through a directly administered survey, teachers reported on their educational background and professional well-being. Items used in this study included: educational level (none (0), some primary (1), completed primary school (2), completed junior high school (3), completed secondary high school (4), and completed any bachelor’s or postgraduate degree (5)); number of years as an ECE teacher (M = 6.5, SD = 6.8); whether teachers had any specific training in ECD (87.8%); and whether they teach at a public (46.3%) or private school.

In addition, teachers responded to three scales assessing their professional well-being, including Burnout (two subscales from the Maslach Burnout Inventory (Maslach et al., 1996): emotional exhaustion, 9 items, α = 0.78, and personal accomplishment, 6 items, α = 0.72; as validated in Lee & Wolf, 2019), and two scales adapted from Bennell and Akyeampong (2007) as reported in Wolf et al. (2015): Motivation (5 items, M = 4.6, SD = 0.59, α = 0.77) and job satisfaction (6 items; M = 3.09, SD = 0.69, α = 0.73).

Covariates

We include classroom- and school-level covariates in our regression models. Classroom-level covariates included dummy indicators for whether the classroom was KG1, KG2, or a mixed KG1/KG2 classroom. School-level covariates include district fixed effects and private school status (the variables included in the stratified random sampling of schools).

Analytic Plan

Our analysis included three steps described in detail.

Factor Analysis

Descriptive statistics for all 49 PLAY items were examined. While many items had low variation (e.g., over 90% of classrooms with a score of zero), all items were maintained for the factor analysis given the novel nature of the tool. Given the exploratory nature of the analysis and the large number of items, we first conducted an exploratory factor analysis examining multiple variations of the items on initial models to be fitted in order to arrive at a proposed model, testing solutions from 1 to 10 factors using varimax rotation. Analyses were conducted using MPlus V8.1. In selecting a model, we considered (1) goodness-of-fit statistics and (2) conceptual meaning of the factors using a 0.4 cut-off for interpreting item loadings (Tabachnik & Fidell, 2019). Once the final model was selected, a confirmatory analysis was conducted to test the model fit in a confirmatory framework.

Sensitivity as a Measure of Impact of an Intervention

We tested whether domains derived from the factor analysis were sensitive to the impacts of the interventions implemented as part of the randomized trial. We conducted two-level hierarchical linear models with classrooms nested in schools. Classroom factor scores were standardized relative to the control group mean; coefficients can be interpreted as effect sizes.

Concurrent Validity

In the final analytic step, we examined concurrent validity. We first examined bivariate correlations to assess how the PLAY factors correlated with two other measures of classroom quality and practices (i.e., TIPPS and developmentally appropriate practice checklist). Second, we examined correlations between teacher characteristics and scores on each PLAY factor.

Results

We describe our approach to examine the characteristics and measurement properties of the PLAY ECE tool in three steps based on our research questions examining (1) what domains of support for engagement are captured by the tool; (2) whether these factors are sensitive to impacts of a teacher professional program aimed at increasing child-centered, activity- and play-based learning opportunities for children; and (3) examining the concurrent validity of these factors.

Exploratory and Confirmatory Factor Analysis

Exploratory factor analyses were conducted with all 49 items, with solutions between one to ten factors tested; 0.4 cut-off was used for item loadings (Guadagnoli & Velicer, 1998; Tabachnick & Fidell, 2019). Neither the eigenvalue plot nor the model fit statistics pointed to a clear preferable number of factors. Several of the factor structures tested yielded unstable solutions (i.e., at least one of the factors had only two items that loaded onto it; this included solutions with 4, 5, 7, 8, 9 and 10 factors), and the scree plot pointed to potential 2-, 3-, and 5-factor models. Balancing the lack of clear guidance from the model fit statistics with parsimony and conceptual considerations, the 3-factor model was selected. This provided the best solution in terms of stable factors that had clear constructs represented and were distinct from one another.

We then conducted a confirmatory factor analysis with the 3-factor solution and to be guided by the modification indices to improve model fit. Using suggested modification indices, five inter-item correlations were added to the final model (both inter- and intra-factor item correlations). The fit statistics were acceptable for this first iteration of the tool, with the RMSEA and SRMR meeting more rigorous generally accepted model fit critera and the CFI and TLI below this criteria (Hu & Bentler, 1999): χ2 (df) = 226.605 (111), p < 0.001, CFI = 0.909, TLI = 0.889, RMSEA = 0.050, SRMR = 0.057.

We named the three factors: Support for Exploration (e.g., supports children’s engagement by encouraging children to explore concepts and engage in inquiry and hypothesis generation; α = 0.68), Social Connectedness (e.g., supports children’s engagement in their learning by relating it to events and experiences which are personal to the child); α = 0.71, and Positive Emotional Climate (e.g., supports children’s engagement by building a positive emotional climate through teacher interactions and by encouraging behaviors of friendship, social acceptance, and active listening among peers; α = 0.55). The final factor loadings for the retained items for the exploratory model are displayed in Table 2.

Table 2 Factor loadings for 3-factor exploratory model
Full size table

The factors had weak- to moderate-sized positive inter-correlations (i.e., Support for Exploration with Social Connectedness, r = 0.43; Support for Exploration with Positive Emotional Climate, r = 0.21; Social Connectedness with Positive Emotional Support, r = 0.64).

Sensitivity to Intervention Impact

Intervention impacts of the QP4G programs on all three PLAY factors are presented in Table 3. Results show that TT had moderate to large impacts on all three PLAY factors, including Support for Exploration (d = 0.696), Social Connectedness (d = 1.005), and Positive Emotional Climate (d = 0.549). The TTPA treatment arm had positive but smaller impacts on all three dimensions (d = 0.330, 0.522, and 0.446, respectively), consistent with previous findings of this treatment arm being less effective at improving classroom quality and child outcomes than the teacher training alone (Wolf et al., 2019a, 2019b).

Table 3 Impacts estimates of QP4G treatment arms on PLAY factors
Full size table

Given the large impacts of treatment status, we re-ran our CFA with classrooms in the two treatment conditions only to examine if the first statistics improved in a sample with greater item variability. We did find some improvements in model fit: CFI = 0.939, TLI = 0.921, RMSEA = 0.045, SRMR = 0.051.

Concurrent Validity

Relations to TIPPS

We examined correlations between PLAY scores with classroom quality scores from the TIPPS and the curricular checklist focused on positive behavior management, child-friendly classroom practices, and the use of instructional materials (Table 4). There was some evidence of convergent validity using these measures, and all associations were positive. Results suggest that the PLAY factors do measure distinct domains of classroom quality that show modest overlap with another quality measure.

Table 4 Bivariate correlations with classroom quality measures
Full size table

Support for Exploration had small, positive, and statistically significant correlations with all three TIPPS factors, including Facilitating Deeper Learning (a measure of scaffolding and instructional support; r = 0.11), Supporting Student Expression (a measure related to helping students develop reasoning and problem-solving skills; r = 0.09), and Emotional Support and Behavior Management; r = 0.34). Further, this factor had small, positive, and significant correlations with the observational checklist of developmentally appropriate practices (r = 0.19).

Social Connectedness had small, statistically significant correlations with Supporting Student Expression and Emotional Support and Behavior Management (r = 0.10 and 0.29, respectively), but not with Facilitating Deeper Learning. The factor also had small statistically significant correlations with the checklist (r = 0.17).

Positive Emotional Climate had small, statistically significant correlations with Supporting Student Expression and Emotional Support and Behavior Management and the checklist (r = 0.14, 0.13, and 0.16, respectively), but not with Facilitating Deeper Learning.

As a point of reference, the three TIPPS factors positively correlated with each other and the checklist ranging from r = 0.16–0.40; the PLAY factors correlated with each other ranging from r = 0.21–0.64. The factors measured by PLAY have smaller correlations to the TIPPS than the TIPPS factors do amongst themselves.

Relations to Teacher Characteristics

Exploratory analyses were conducted with a set of teacher characteristics, private vs. public sector status, and professional well-being indicators (Table 5). Few teacher characteristics were correlated with PLAY scores. Teacher education was positively correlated with Positive Emotional Climate, but no other education or training characteristics were related to PLAY scores. Interestingly, the number of years worked as a preschool teacher was negatively correlated with all three factors (though not statistically significant). It is possible that older teachers are more set in their ways, and less likely to implement progressive teaching practices that might be introduced in more current teacher trainings. There were no differences in public versus private schools.

Table 5 Bivariate correlations with teacher characteristics
Full size table

Associations with teacher professional well-being were mixed. On one hand, burnout was positively correlated with all three PLAY factors. Motivation was positively correlated with Positive Emotional Climate. Teachers’ self-reported job satisfaction was not correlated with any of the three factors. Notably, correlations were small (r < 0.10).

Discussion

We assessed the factor structure of a new pilot tool developed to measure classroom-based opportunities for engagement in learning, as well as construct validity of the factors, in Ghanaian preprimary classrooms. Capitalizing on a unique dataset of a school-randomized trial with video-based classroom observations and teacher surveys, we were further able to assess whether the observational PLAY ECE tool was sensitive to the experimental impact of a teacher professional development program aimed at improving the quality of child-centered, play-based pedagogical practices as well as to explore associations with another measure of classroom process quality and with teacher characteristics. Our findings advance several areas of literature related to early childhood educational quality, playful learning, and global early childhood education and advance the discourse and interest in “play-based” pedagogy and learning by clearly defining and articulating key processes that support engaged learning and can be measured at the classroom-level.

We provide evidence that engaged learning opportunities can be meaningfully measured. Our conceptual framework proposed six dimensions: connection to experience, exploration, problem-solving, agency, social connectedness, and positive emotional climate. Using exploratory and confirmatory factor analyses, we identified three domains of opportunites for engagement as measured by PLAY ECE: Support for Exploration, which includes the ways teachers engage children in self-expression and critical thinking (combining exploration, problem-solving, and connection to experience); Social Connectedness, which includes connections to children’s everyday lives explaining different experiences, actions, and intentions (combining connection to experience with social connectedness); and Positive Emotional Climate, which focuses on social acceptance, peer active listening, and emotional responsivity. The latter factor perhaps has more conceptual overlap with previous observation tools, focusing on the emotional climate of the classroom, but interestingly only had a small correlation with a measure of emotional support from the TIPPS (r = 0.13). The items of positive emotional climate in the PLAY may be a subset of or only have partial overlap with the broader domain, given their focus on elements of emotional climate that support engagement (e.g., peer active listening). The three factors had moderate-sized correlations (r = 0.21–0.64), suggesting that they each measure unique but related elements of teacher practice.

Interestingly, all three factors were impacted by the intervention that was implemented in the study from which this dataset is drawn, with larger effects for the teacher training intervention alone (d = 0.55–1.01) than the combined teacher training and parental awareness program (d = 0.33–0.52). These differences are consistent with previous published findings of the two intervention programs (Wolf et al., 2019a, 2019b). These findings suggest the PLAY ECE observation tool holds promise in detecting improvements in teacher practice and may serve useful in future program evaluation efforts related to increasing playful learning opportunities in classroom settings. This suggests that the PLAY measurement tool may have implications for teacher professional development in SSA and other contexts where traditional teacher-directed pedagogies are common. Numerous attempts have been made to introduce child-centered or play-based pedagogies into African schools. Many such attempts have been unsuccessful because they have not been adopted by teachers (Schweisfurth, 2011; Vavrus, 2009) or have been opposed by parents (Wolf, 2020). The PLAY measurement tool has been culturally adapted to measure pedagogical approaches that promote student engagement and yet are achievable and acceptable in contexts where teacher-directed instruction is the norm. In this way, the PLAY tool—and the instructional items it includes—represents a realistic path for teacher professional development in Africa.

Importantly, these practices were not widely observed in this set of classrooms despite their alignment with the aspirations and national pre-primary curriculum put forth by the Ghana Education Service (Ghana Education Service, 2012, 2013). The PLAY tools were designed to capture the kind of learner engagement that may only be evident in high-performing classrooms. Given global aspirations for modernizing and improving education systems (e.g., Hirsh-Pasek et al., 2020a, 2020b), we recommend exploring the use of the tool in conjuction with existing measures in order to capture the full spectrum of quality in ECE classrooms and to highlight areas where teachers may need additional support to ultimately implement such aspirational pedagogy. Increasing access to high-quality instruction has implications for equity, as access to and enrollment in ECE continues to be inequitable across and within LMICs (Akkari, 2022; UNICEF, 2019), and quality of services have been generally lower for marginalized communities (Neuman et al., 2015).

Interestingly, items measuring agency did not materialize as a salient set of items in the factor analysis, suggesting that additional refinement of this concept is needed to understand how, or if, teachers support children’s agency in Ghana and other preprimary settings. In developing PLAY ECE, cross-country qualitative work was conducted to understand how educators viewed children’s autonomy and engagement in the classroom. In two African contexts studied, including in Ghana, teachers endorsed autonomy as a goal of their instruction but felt that children needed direction from teachers to achieve this goal. Instruction in classrooms is consequently teacher-led with limited focus on child engagement. Our analysis suggests that this pilot tool holds promise—particularly the improved fit statistics in treatment classrooms where these practices were more prevalent—but also imply that further refinement is necessary, and these plans are underway.

We then explored construct validity through two separate analyses. First, we examined whether the three PLAY factors correlated with process quality factors previously identified in these same classrooms using the Teacher Instructional Practices and Processes System (TIPPS; Wolf et al., 2018). The TIPPS measures dimensions of classroom process quality grounded in the teacher–child interactions framework and has already been shown to measure domains of facilitation of deeper learning, emotional support and behavior management, and support for student expression (Wolf et al., 2018). We found small but significant correlations among the factors (r = 0.03–0.33), suggesting that the PLAY observation tool measures unique dimensions of quality teaching practice that is not captured in the TIPPS. We also examined correlations with a checklist of developmentally appropriate practices including practices related to positive behavior management and to a child-friendly classroom. These checklists also showed small but meaningful associations with the three PLAY factors, with correlations around 0.17. Given that the PLAY factors focused more on learning opportunities and less on behavior management, these correlations provide further support for the validity of the identified factors.

Second, we examined correlations with a select set of teacher characteristics measured in the fall, guided by the literature, with PLAY scores in the spring. All three factors were predicted by structural elements of teacher quality, including education level and training in early childhood development predicting higher scores, and years as an ECE teacher predicting lower scores (suggesting that newer teachers are more likely to provide playful learning opportunities to students). Regarding teacher professional well-being, it is perhaps counter-intuitive that elements of teacher burnout predicted higher scores of the PLAY factors. While teacher well-being was measured in the fall, and PLAY scores measured in the spring, our analysis is still correlational. Thus it is not known whether implementing these types of learning opportunities leads to higher levels of burnout (e.g., given the discouragement of such practices from school leaders; Wolf, 2018), or if teachers who have high levels of burnout implement such learning opportunities for their students for some reason.

Limitations and Conclusions

This study has important limitations that should be noted when interpreting the results. First, establishing both validity and reliability is important for any new tool. In this study, we only assessed the validity of the tool in terms of its factor structure and concurrent validity. Additional research is needed to examine inter-rater and test–retest reliability of the tool. Second, analyses are correlational in nature and causal links can not be concluded between any of the associations tested except the experimental impacts on the PLAY tool factors. Third, our sample of classrooms is limited to one region in Ghana—the Greater Accra Region, which is the most diverse, fastest growing and urbanizing region in the country (Tuholske et al., 2020). The results are not generalizable to other parts of Ghana or to other countries. This is important, as teachers in rural regions of Ghana face additional unique challenges such as large class sizes, higher student poverty rates, and lower levels of training (Cooke et al., 2016). have baseline information on PLAY scores; this addition would further strengthen the intervention impact estimates. Fourth, the items in the PLAY ECE tool were highly skewed, with most being observed in less than one-fifth of classrooms. The tool may provide guidance to help fulfill curricular aspirations for providing children with these types of learning experiences. Notably, in Jordan and Colombia, where the tool has also been used, greater variation has been identified (Jukes et al., 2022), suggesting that the tool may be useful to document differences across contexts in the quantity of engaged learning opportunities that exist within ECE settings. Fifth, we only included measures of PLAY ECE post-treatment, and did not have measures of baseline classroom scores, which would have been helpful in increasing the precision of the treatment impact estimates. Finally, given the number of items in the PLAY ECE, we did not have a large enough sample of classrooms to split the sample for exploratory and confirmatory analyses. As a result, the CFA results cannot be used to confirm model fit from the exploratory analyses, but instead can only be used to better understand cross-loadings and error terms. The correlated errors across factors suggested in the CFA, for example, suggest that future versions of the tool would benefit from further delineation of the constructs and associated factors. The current analysis is therefore relatively early in the development and refinement of this measure. This is an important area for future research to build on to better understand the underlying domains measured by the tool.

Two criticisms have been levelled against existing tools. The first is that existing tools do not pay enough attention to the content of ECE activities, and the second that they do not capture the level of scaffolded learning (Burchinal, 2018). The PLAY ECE observational tool is a domain-general measure, and thus does not address the first criticism of lack of attention to ECE content. The tool does partially address this second criticism, and the first factor identified (Support for Exploration) contains several elements of scaffolded learning, including eliciting relevant background knowledge, rephrasing answers and asking questions, and engaging children in hypothesis generation and generating explanations. It is possible that additional elements of scaffolded learning are needed in future tools, and that measures of the content being taught are needed to complement measures of guided play to be able to identify the types of classroom environments that do predict children’s learning outcomes. Bringing together these two bodies of recommendations may yield important advancements in the field of ECE classroom quality and provide a more holistic picture of classroom quality that more powerfully explains child outcomes.

Nonetheless, our study provides a significant advancement to the field of early childhood education and playful learning by advancing the theoretical framework of how playful learning opportunities can lead to engagement in the learning process, and systematically operationalizing such opportunities at the classroom-level in an under-studied context. The large intervention impacts hold promise, and future research is needed to understand if and how these constructs related to children’s learning and development. The PLAY ECE tool is currently undergoing contextualization and revision in a range of new countries and that factor structure, as well as the relationship between factors and student learning outcomes, will be re-examined in these new contexts. Thus, this study provides a foundation for future research to continue advancing the measurement of playful learning opportunities and engagement in learning across diverse contexts.