Keywords Indigenous peoples · Early childhood education and care · Program evaluation · Abecedarian approach · Language development
Early childhood conditions inﬂuence future health and well- being (National Scientiﬁc Council on the Developing Child, 2007, 2010; Shonkoﬀ, 2012). Children who experience high levels of disadvantage, neglect and trauma during their early years have a markedly increased risk of disruption to their development of cognitive, language, social, and emotional skills (Child Welfare Information Gateway, 2015; Perry, 2013; Schore, 2001).
Globally, Indigenous children face some of the greatest barriers to positive developmental outcomes and experi- ence greater educational, health, social, and economic dis- advantage following centuries of colonisation, assimilation, discrimination, and racism (Trent et al., 2019; Anderson et al., 2016; Australian Government Productivity Commis- sion, 2016; Best Start Resource Centre, 2010). In Australia,
Sharon Goldfeld and Sandra Eades are joint senior authors. * Catriona Elek email@example.com 1 Murdoch Children’s Research Institute (Centre for Community Child Health), Melbourne, VIC, Australia 2 Curtin Medical School, Curtin University, Perth, WA, Australia 3 The University of Melbourne, Melbourne, VIC, Australia 4 Baker Heart and Diabetes Institute, Alice Springs, NT, Australia 5 Department of Paediatrics, The University of Melbourne, Melbourne, VIC, Australia 6 Central Australian Aboriginal Congress, Alice Springs, NT, Australia 7 The Royal Children’s Hospital, Melbourne, VIC, Australia
Early Childhood Education Journal
Aboriginal and Torres Strait Islander1 children in the North- ern Territory are among the children who are most disad- vantaged (Australian Institute of Health and Welfare, 2017), particularly those in Central Australia (De Vincentiis et al., 2019). As a result of two and a half centuries of discrimina- tory policies, Aboriginal and Torres Strait Islander children are more likely to experience disadvantage and neglect, contributing to disruptions of their development in early childhood (Australian Government Productivity Commis- sion, 2016). One in ﬁve Australian children who commenced school in 2018 were developmentally vulnerable in one or more domains of child development; for Aboriginal children, it was two in ﬁve (Department of Education and Training, 2019). In the Northern Territory, Aboriginal children were even more disadvantaged on each domain: 7% of non-Indig- enous children were vulnerable in the language and cogni- tive skills domain, compared to 47% of Aboriginal children (Australian Government Productivity Commission, 2016).
Investment in the early years has been prioritised in Aus- tralia, especially for Aboriginal children; one of the current ‘Closing the Gap’ targets is that by 2025, the proportion of Aboriginal and Torres Strait Islander children enrolled in early childhood education in the year before schooling will be at least 95% (Closing the Gap Partnership, 2020).
Attendance at high-quality early childhood education and care (ECEC) services can have positive impacts on children’s cognitive development and learning, both in the short- and long-term (Apps et al., 2013; Goldfeld et al., 2016; Melhuish et al., 2015; Vandell et al., 2010), particularly for children who experience disadvantage at home (Australian Institute of Health and Welfare, 2015; Cloney et al., 2016; Page and Tayler, 2016; Torii et al., 2017). For Aboriginal children, attendance at ECEC services has been associated with bet- ter short-term cognitive outcomes, as well as longer-term cognitive and developmental improvements (Arcos Hol- zinger & Biddle, 2015), and greater ‘school readiness’ in preparation for transition to compulsory schooling (Kellard & Paddon, 2016). Parents have also identiﬁed that these ser- vices allowed them to undertake courses, return to work, or have time to themselves (Kellard & Paddon, 2016). For Aboriginal children, it is important that ECEC services sup- port children’s cultural identity and connections, for exam- ple, through incorporating cultural content, Aboriginal lan- guages, and a focus on family engagement (Guenther et al., 2019; Matiasz, 1989; Secretariat of National Aboriginal and Islander Child Care, 2012; Trudgett & Grace, 2011).
The quality of ECEC varies (Torii et al., 2017) and there is more to learn about how to implement high-quality ECEC programs with ﬁdelity, particularly in the Aboriginal
context. Only a few studies have investigated the implemen- tation, appropriateness, and acceptability of evidence-based programs in ECEC that meet the needs of Aboriginal chil- dren and their families (e.g., Brookes & Tayler, 2016; Elek et al., 2020; Moss et al., 2019; Mughal et al., 2016; Smith et al., 2018).
Many young Aboriginal children miss out on ECEC; in 2018, only 76% of Aboriginal children aged 4 years were enrolled in ECEC, well below the national target of 95% by 2025 (Australian Government, 2020). To increase access in Central Australia, an ECEC service, Arrwekele Akaltye- Irretyeke Apmere (hereafter referred to as ‘the Centre’), was established in 2017 by the local Aboriginal community-con- trolled primary health care organisation, Central Austral- ian Aboriginal Congress (‘Congress’). The Centre aimed to provide a service for young Aboriginal children in Alice Springs who did not already access ECEC. The Centre could support up to 45 children at any one time from the age of 6 months to approximately 3 years. The Centre included both indoor and outdoor areas. The group of children in attendance were treated as one ‘class’, and children of all age groups mixed within the learning spaces and moved freely between the main indoor and the outdoor space. The Centre maintained a 1:4 staﬀ:child ratio in acknowledgment of the range of ages of the children.
The Centre adopted the ‘Abecedarian’ approach to early childhood education which has been widely utilised interna- tionally and adapted for the Australian and Aboriginal com- munity context (‘Abecedarian Approach Australia’, referred to as the ‘3a’ approach). The 3a approach uses evidence-based teaching and learning strategies which emphasise the role of young children as active learners and places a priority on chil- dren’s language acquisition. It encourages educators to have frequent, intentional interactions with individual or small groups of children. Within the overarching focus on language development, the 3a approach has three main elements: Learn- ing Games®, conversational reading and enriched caregiving (“3a Abecedarian Approach Australia”, 2017; Ramey et al., 2012). This approach was chosen by the Centre, as interna- tional studies have demonstrated improvements in participants’ education, social, economic, employment and health outcomes as a result of the Abecedarian approach (Campbell et al., 2008; Ramey et al., 2012; Ramey & Ramey, 2005). Studies have also demonstrated the eﬀectiveness of the 3a strategies in support- ing Aboriginal educators, families and children in Australia, including in the Northern Territory (Brookes & Tayler, 2016; Page et al., 2019). Within the Centre, the plan for implement- ing the 3a approach included educators using the 3a strate- gies to plan activities for individual children, adopting the language-rich interaction style speciﬁed in the 3a approach during all aspects of their provision of education and care, and engaging in frequent one-on-one activities with children using the Learning Games.
1 Henceforth referred to as Aboriginal when referring to Indigenous children in Central Australia.
Early Childhood Education Journal
In order to support the adoption of the 3a approach, 3a training was provided by the Centre for staﬀ immediately prior to the evaluation period and twice during the evalu- ation period. Staﬀ were encouraged to plan 3a activities for each child, to oﬀer frequent (ideally, daily) one-on-one 3a interactions with each child, and to adopt the language- rich interaction style speciﬁed in the 3a approach across all aspects of their provision of education and care. Follow- ing the training, individual coaching was oﬀered to many educational staﬀ to support implementation. Training and coaching were provided by the Centre and supported by the University of Melbourne Graduate School of Education independently of the research team.
Few studies have evaluated ECEC services for Aborigi- nal children or reported on challenges in conducting such evaluations. This evaluation involved a process and impact evaluation of the Centre after its ﬁrst year of operation, spe- ciﬁcally on the ﬁdelity and quality of the service, the 3a approach, its cultural safety, acceptability to the community and the children’s learning and development. This paper also comments on lessons learned by the authors to provide insights for future evaluations of such programs.
Materials and Methods Evaluation Design
The evaluation was initiated by the Aboriginal community- controlled governing body for the Centre and used a mix of qualitative and quantitative methods to undertake a process and impact evaluation. The process evaluation examined the Centre’s educational quality, cultural safety and acceptability to parents/carers and the community. The impact evalua- tion assessed changes over time in children’s language and development.
The evaluation team had strong Aboriginal leadership (SE) with experience in Indigenous health and early child- hood development and an Aboriginal researcher from the local community (RH) was appointed to play a key role in the evaluation. Ethical approval for the conduct of the evaluation was provided by the Central Australian Human Research Ethics Committee (CA-17-2824) and the Royal Children’s Hospital Human Research Ethics Committee (37045).
Measures and Analysis Interviews and Surveys
We conducted structured interviews and surveys with parents/carers, staﬀ, and community stakeholders on per- ceived quality of the service, cultural safety, observed changes in their child’s behaviour and/or development,
and their thoughts on improving the service. Interview and survey data was analysed drawing from approaches to qualitative analysis described by Miles et al. (2014), Neale (2016) and Thorne et al. (1997). A priori codes were iden- tiﬁed by identifying key constructs related to the evalua- tion questions such as cultural safety. These codes were supplemented by inductive codes when the a priori codes were insuﬃcient. Themes were then identiﬁed by noting patterns, inconsistencies and relationships between codes.
Our deﬁnition of a culturally safe service drew from that presented by Williams (1999) and Congress’s Cultural Safety Framework, which describes what cultural safety means for the Centre, why it is important, the underpin- ning principles and processes for monitoring and evaluat- ing cultural safety. Cultural safety was thus deﬁned as one in which children (and Aboriginal staﬀ) feel valued and are comfortable with being themselves and expressing their Aboriginal identity and culture.
The Centre was observed using the CLASS observation tools (Toddler version) (La Paro et al. 2012) by trained and certiﬁed CLASS observers. The CLASS observation tools are reliable, valid tools for measuring eﬀective class- room interactions (Pianta et al., 2008) and have been used widely in Australian ECEC services (e.g., Eadie et al., 2017; Tayler et al., 2013). Toddler CLASS observation tools are scored on a 7-point scale (maximum score: 7) across 8 dimensions grouped into 2 domains. Composite scores for each dimension are generated by averaging indi- vidual observation cycle scores. Domain scores are pro- duced by following a standard formula for each domain.
Scores of 1–2 are classed as ‘low’, scores of 3–5 as ‘mid- range’, and scores of 6–7 as ‘high’ (Pianta et al., 2008).
Between September 2017 and December 2018, the Centre was observed on ten occasions. Each of these observa- tions consisted of ﬁve or six observation cycles of approxi- mately 20 min’ duration each, taken over the course of a week. The Centre was treated as one ‘classroom’ and the main activity taking place at the time was the focus of each observation cycle.
To measure fidelity of implementation of the 3a approach, four additional unannounced drop-in observa- tions were conducted between baseline and follow up, in early April and mid-June 2018. For these observations, a checklist based on one developed by Pilsworth et al. (2017) was used to record the type of talk and activities undertaken by educators during these periods of obser- vation. This captured, for example, whether staﬀ were engaging actively with children in accordance with the
Early Childhood Education Journal
3a approach, and whether this was with individual, small groups or large groups of children.
Measures of Language and Child Development
The Preschool Language Scales, 5th Edition (PLS-5) was used to measure each child’s auditory comprehension and expressive communication. The PLS-5 is an interactive, play-based comprehensive developmental language assess- ment for children from birth to age 7 years and 11 months (Zimmerman et al., 2012). This scale was chosen as the most appropriate measure of language development because it covers the full age range of children in this study, it covers children’s language development from birth, and it includes both direct observation and parent/carer report. Language assessments of children were conducted by child health nurses employed by Congress when they enrolled in the Centre as part of routine care. PLS-5 scores (range 50–150) were analysed for all participating children in relation to their exposure (as measured by hours of attendance). We also compared the language scores of participating children from their ﬁrst assessment to a reference group of children from the USA, upon which the standardised scoring system is based. For those children assessed more than once, results from the ﬁrst assessment were used.
The ASQ-TRAK was used to assess children’s develop- ment at baseline. The ASQ-TRAK is a culturally appropri- ate developmental screening tool for children from 1 month to 66 months of age (D’Aprano et al., 2016). The tool was adapted from the Ages and Stages Questionnaire, 3rd edition (ASQ-3) (Squires et al., 2009) for use in remote Australian Aboriginal contexts (D’Aprano et al., 2016). The ASQ-3 is widely used and has been culturally adapted and/or trans- lated for use in many international settings (e.g., Dionne et al., 2014; Filgueiras et al., 2013). The ASQ-TRAK is shorter, written in modiﬁed English, and includes other modiﬁcations to make items more culturally appropriate for Aboriginal children. ASQ-TRAK assessments were con- ducted by Congress health service staﬀ. Children’s scores were categorised as ‘below cut-oﬀ’, ‘close to cut-oﬀ’ and ‘above cut-oﬀ’ using cut-oﬀ scores from the ASQ-3 based on the scores of a sample of children from the USA (Squires et al., 2009).
A selection of the Centre’s administrative documents were qualitatively reviewed in order to examine the quality of service delivery. These included the Centre’s vision, mis- sion and philosophy statements, training documentation, minutes of staﬀ meetings, enrolment forms, individual child learning plans, records of 3a sessions, fortnightly program records, and documents relating to the independent quality
assessment and rating report for the Centre. Documents were treated as qualitative data (Bowen, 2009) and analysed using the same methods used for the analysis of interview and survey data. In addition, Congress provided anonymous data about staﬃng levels at the Centre during the evalua- tion period.
Child Attendance, Health and Demographic Information
In addition to the above sources of information, the Centre routinely collected information about the children’s attend- ance at the centre and exposure to the 3a program. Chil- dren’s health and demographic information was obtained from Congress’s primary health care records.
Results Participant Characteristics
Parents/carers of forty-six children attending the Centre pro- vided informed consent for their children to participate in the evaluation. Additionally, 21 staﬀ members (11 Aboriginal), fourteen community stakeholders (7 Aboriginal) and 8 Abo- riginal parents/carers were interviewed. In total, attempts were made by an Aboriginal researcher (RH) to contact 128 parents/carers. Many could not be contacted as their con- tact details had changed or because they regularly moved between Alice Springs and surrounding areas. Among those who could be contacted, some were busy during data col- lection periods with cultural activities, including ‘sorry business’. Others chose not to participate due to language barriers or for cultural reasons.
The participating staﬀ members ranged in experience from less than 6 months’ experience in early childhood education to more than 10 years. The majority of staﬀ had a qualiﬁcation in early childhood education: Six had received a Certiﬁcate III in Early Childhood; four had a diploma-level qualiﬁcation in early childhood education; and, three were qualiﬁed teachers with a university degree in education. At the beginning of the evaluation period, nine out of fourteen staﬀ had attended 3a training; at the end of the evaluation period, four new staﬀ had not yet completed the training.
Participating children commenced at the Centre at a range of ages. Many had a poor start to life; approximately a quarter (24%) were born with low birth weight and approx- imately one-third (32%) were born preterm. When they joined the Centre, 30% had low haemoglobin and around a quarter (26%) were diagnosed with otitis media within 6 months before or after joining the Centre (Table 1).
Thirty-four participating children had the ASQ-TRAK administered during the evaluation to determine their devel- opment at baseline. Twenty-four of the 34 children were