· Explain how these environmental structures have impacted your (1) personal, (2) educational, and (3) career goals.

Executive summary

This literature review provides an in-depth analysis of the concept of education policy implementation, its definitions, processes and determinants and proposes a framework for analysis and action. It aims to clarify what education policy implementation entails in complex education systems and support policy work building on the literature and country examples. The paper centres on school education. It specifically focuses on answering two questions: what does education policy implementation entail in theory and in practice? What are the determinants involved in the process of education policy implementation?

Education policy implementation is a complex, evolving process that involves many stakeholders and can result in failure if not well targeted. In fact, a range of reasons can prevent implementation from being effective, such as a lack of focus on the implementation processes when defining policies at the system level; a lack of recognition that the core of change processes require engaging people; and the fact that implementation processes need to be revised to adapt to new complex governance systems. It is therefore crucial to understand it, clarify its determinants and explore ways in which it can be more transparent and effective.

Education policy implementation does not only refer to the strict implementation process but needs to be seen in its broader context. Following an analysis of the range of definitions and frameworks on the topic, this paper defines education policy implementation as a purposeful and multidirectional change process aiming to put a specific policy into practice and which may affect an education system on several levels:

* Implementation is purposeful to the extent that the process is supposed to change education according to some policy objectives.

* It is multidirectional because it can be influenced by actors at various points of the education system.

* It is contextualised in that institutions and societal shocks and trends -i.e.in culture, demography, politics and economy- affect the education system and the ways in which a policy is shaped and translates in the education sector.

But to make it more concrete and valuable for policy makers, it is necessary to take this definition and make it more actionable, by analysing the range of determinants that hinder or facilitate the implementation process. From the analysis, this paper proposes a generic framework (Figure 1) shaping determinants around four dimensions, each of which should be taken into account for education policy implementation effectiveness:

* Smart policy design: a policy that is well justified, offers a logical and feasible solution to the policy problem, will determine to a great extent whether it can be implemented and how. For instance, if a new curriculum requires the use of high technology equipment which schools cannot afford, the policy may fail to be implemented unless some budget is available at the national or local level.

* Inclusive stakeholder engagement: Whether and how key stakeholders are recognised and included in the implementation process is crucial to its effectiveness. For example, engaging teacher unions in discussions early on in the policy process will have long-term benefits.

* A conducive institutional, policy and societal context: An effective policy implementation process recognises the existing policy environment, the educational governance and institutional settings and external context.

* A coherent implementation strategy to reach schools: The strategy outlines concrete measures that bring all the determinants together in a coherent manner to make the policy operational at the school level.

This framework is translated into a set of questions and principles for action (Table 4.1) to guide policy makers to think through, design and analyse their education policy implementation process.

1.Introduction: Why study education policy implementation?

While pressures on education systems grow to deliver high-quality education and the number of reforms increase, policy makers do not necessarily grant much attention to their implementation. Education policy implementation is a complex, evolving process that involves many stakeholders and can result in failure if not well targeted. It is therefore crucial to understand it, clarify its determinants and explore ways in which it can be more transparent and effective.

This paper builds on the literature to provide a definition of education policy implementation and its determinants, and proposes a framework to support education policy makers in the implementation process. This introductory Section sets the scene to explain why it is important to analyse this topic now, what are some of the main challenges, and it develops the two questions that underpin the study: what does education policy implementation entail in theory and in practice? What are the determinants involved in the process of education policy implementation? The Section then describes the methodology used for the study, and presents the different types of frameworks for policy implementation that exist and are used in this paper’s analysis.

1.1.Education policy implementation: Setting the scene

OECD countries adopted no fewer than 450 education reforms between 2008 and 2014 (OECD, 2015[1]). Considering the fast-paced economic, social and demographic environments that surround education, efforts for education systems to adjust, improve and drive the future appear warranted. There is little evidence of whether education reforms have an effect, however, because educational impacts are challenging to assess and seldom evaluated. Even when reforms do have an impact, stakeholders are easily dissatisfied with the outcomes and they tend to hold policy makers accountable for them (Gallup, 2017pp Corbier, 2017[3]). Similarly, there is little knowledge about the actual processes that produce, or are supposed to produce the desired outcomes. These processes “between the establishment of a policy and its effects in the world of action” (O’Toole, 2000m) are commonly referred to as policy implementation, even if there is no consensus on the definition.

There is indeed a difference between passing a policy bill or a strategy and turning it into daily practices for teachers, school administrators and local communities. Implementation details may be left for administrations and educators to figure out, effectively leaving the reform process half-way through (Hess, 2013ra). Observing that policies often do not get implemented as planned, or not with the desired outcomes, governments, experts and international organisations have come to acknowledge the need to focus more on implementation processes (Gurría, 2015и; Wagstaff, 2013[7]; Pont, 2008[8]; OECD, 2016и).

Challenges to implementing education policy include co-ordination issues, inadequacy of organisational resources, actors’ capacity or reactions against reforms. But as the education sector has become more complex, the challenges of putting change into practice have also evolved. Education stakeholders are increasingly diverse and growing more vocal and ambitious about what education systems should look like. The use of technologies contributes as well to making education systems more complex. Interactions between actors and between the various levels of education systems (national, regional and local) weigh more in the policymaking process. New questions are emerging about who is responsible to do what in the systems, how to hold them accountable and how the implementation process itself can contribute to enhancing education (Burns, Köster and Fuster, 2016[10]).

In fact, “education policy implementation” refers to different realities for different people: educators and students may consider policy implementation as the changes they bring to their everyday practices of managing schools, teaching, and learning. For national policy makers, implementation may refer to what needs to be executed to bring their new policy down to districts and schools. For regional or local policy makers, it may mean making choices about changing priorities, and use of resources.

The different definitions of education policy implementation found in the literature convey specific perspectives on the policy process (Datnow and Park, 2009[11]). Traditionally, policy makers see education policy implementation as a technical stage of the policy process in which the decision they have taken gets executed by the administration and educators throughout the system. Although they observe implementation failures, their solution is often to instigate more rational public management practices and monitor the implementation processes more closely.

This top-down perspective on implementation is challenged, especially in the case of more complex education systems. Scholars now tend to define implementation rather as an iterative, political process wherein actors influence the outputs and outcomes of the policy. Literature that focuses on “bottom-up” perspectives does not look at implementation per se, but at the policymaking process as one intricate political game. Such perspectives are essential to understand the complexity of implementation, but are challenging to integrate in practical advice for education policy actors, especially those making policy at the national level.

If policy makers and stakeholders want policies to be effective and improve education, they need to share a common understanding of implementation to be able to work together on the process. Between top-down and bottom-up approaches to policy making, this paper shows that education policy implementation is actually quite a complex process but “thorough implementation of policy change in education is actually possible”, in spite of this complexity (Mason, 2016[12]).

This literature review aims to clarify the scope based on current theories and on the new complexity in education policy making at the primary and secondary levels. This section sets the scene by introducing the topic, presenting the challenges of implementation and an overview of existing research and frameworks. Section 2 elaborates a definition and section 3 analyses the key determinants of implementation processes in education. Lastly, section 4 proposes a framework that can serve policy makers to support their implementation processes.

1.2.Key challenges of implementation

As our economies and societies have evolved from industrial to becoming knowledge based, education has become crucial for individual and social progress. Education systems are now more than ever required to provide high-quality education and competencies, in addition to new demands for well-being and values, to enable young generations to design and contribute to our fast-paced, global economy. But education policies may not reach the classroom, failing to achieve their intended outcomes, because of weak implementation processes.

The literature reveals a range of reasons preventing implementation from being effective. Among others, we can highlight a lack of focus on the implementation processes when defining policies at the system level, the lack of recognition that these change processes require engaging people at the core and the need to revise implementation frameworks to adapt to new complex governance systems.

These challenges call for the need to review current implementation approaches to see if they are adapted to education policy making in the 21st century and especially, whether they are able to support the development of professional processes that can contribute to success in the policy process.

1.2.1.Insufficient focus on implementation

Viewing education as a driver to develop highly-skilled youth and meet the needs of the knowledge society represents a paradigm shift from the beginning of the 21st century (Lessard and Carpentier, 2015[13]). This shift has caused policy makers and other stakeholders to pay more attention to schools’ performance and to raise their expectations about the quality and the scope of the services delivered in schools.

Governments have undertaken reforms to respond to these expectations. The number of reforms for a given system can be impressive: in Australia for instance, 38 national reforms were introduced between 2008 and 2014 while Ireland led 23 reforms in the same period (OECD, 2015^). In one country, educators may for example have to deal simultaneously with enhancing the equity and quality of educational outcomes, reforming the way teachers are trained and changing the way students are evaluated.

Whether formulated policies take effect “in the world of action” (O’Toole, 2000и) is not clear, however. Few studies actually document reform impact or can specify what factors contribute the policy’s success. It is also challenging to measure policy outcomes in education because they take time to appear, and because it may be difficult to attribute learning performance outcomes to one specific policy.

For example, in the Czech Republic, reforming the school-leaving examination took 14 years of debating and testing various versions and ways to implement them, even after an initial policy was passed (OECD, 2016и). Analysing the effects of Comprehensive School Reforms in the United States in the 1990s, Borman et al. find that the strongest effects are found 8 to 14 years after a reform begins (2003[14]). Policy evaluation analysis in the United States in the 1990s showed nonetheless that assessing the outcomes without looking at the concrete processes that produce them did not provide a complete picture (Rist, 1995[15]).

In fact, it appeared that policy makers often do not give priority to implementation. In an article entitled “the missing half of school reform”, education scholar Frederick M. Hess underlines how decision makers tend to focus their efforts on formulating the policy, with little or no follow-up on how to make the policy take effect in education. “In education, there is often a vast distance between policy and practice” (Hess, 2013[5]): educational policies seem to be developed with little consideration for the practical mechanisms necessary to their implementation. Questions such as “do teachers have the skills to teach this new curriculum?” are often overlooked. As a result, expectations concerning schools’ capacity to implement often exceed reality (OECD, 2010[16]).

Although it is difficult to assess whether a policy failed because it was not effective or because it was not well carried out, this lack of focus on implementation can have serious implications. First, the public resources invested in that policy might have been wasted when they could have served for another project. Second, after a number of policies failing to be implemented, citizens may start losing confidence and patience with policy makers and other actors in the education system. Passing a policy that fails to be implemented is thus a risk for education policy makers. It is therefore necessary to ensure that when designing and introducing new education policies, policymakers focus and design strategies for the implementation process itself, taking into consideration that it is a complex change process rather than the execution phase of policymaking.

1.2.2.Implementation as a change process

Embedded within the concept of implementation is the idea that the policy that gets implemented brings about an effective change to the education sector. For example, implementing a new curriculum at the school level mostly implies changing schools and teachers’ practices, their beliefs, and the materials used. On the other hand, a policy introducing new school funding formulas requires district leaders and principals to change the way individual schools and local education systems are managed and funded (OECD, 2017^).

Reforming education is no easy task, however. As noted in Hess (2013и) about the American public schools, “schools and districts do not go out of business” and follow their everyday activities in teaching and learning. According to a study on public sector activities, there is an entrenched tradition for education to stick to the status quo and resist change in a number of countries (OECD, 2017[18]). Given the cost of reforms and the uncertainty about the outcomes, stakeholders may prefer sticking to the status quo rather than changing (OECD, 2016[9]).

As most policies aim to bring a change to how education works, implementing them requires facing multiple challenges in the process. These include among others, communication and co-ordination issues, problems with organisational resources, capacity and compliance of the policy operators and policy targets (Weaver, 2010[19]).

Different approaches to educational change or policy reform emphasise a range of challenges to implementation. Organisation theory and public administration literature for example emphasises the importance of overcoming resistance from stakeholders, to build support, to provide a plan and resources for change, and to find a way to embed the policy in daily routines to make the change sustainable (Fernandez and Rainey, 2006[20]). For instance, schools may lack capacity and resources to implement reforms -such as funding, training or technology. The political economy of reform looks at limited public budgets and resistance by interest groups, which policy makers must find a way to bypass in order to reform effectively. School change scholars suggest that unless teachers, school leaders and other actors in education understand and share the policy meaning, it is unlikely to get implemented (Fullan, 2015[21]).

Educational change cannot be reduced to the question of resistance to reform or the outcome of policy implementation, however. The process of implementation in itself is an opportunity to engage stakeholders, which can benefit them and the education system overall. For instance, during the implementation of the Race to the Top programme in Rhode Island between 2010 and 2014, district leaders developed problem-solving skills by learning from their peers while reporting to the State Agency (OECD, 2016и).

Studying education policy implementation is therefore closely linked with understanding what determines education systems’ ability and actors’ willingness to engage and change. Moreover, these change processes take place in education systems that are increasingly complex, and require more elaborate strategies than the traditional top-down policy making.

1.2.3.Enacting change in complex education systems

Recent developments in the literature have shown how education is taking shape in increasingly complex environments, which affects the way modern education systems are governed (Burns, Köster and Fuster, 2016[10]). Complex systems are characterised by new structures and new behaviours that emerge thanks to the interactions between multiple actors.

The number and type of actors that get involved with education policy have grown. Regional and local administrators, school representatives, principals, teachers, parents and other actors are keen to defend their own vision of education, based on deeplyrooted and largely personal belief systems. These actors engage in heated political debates about what priorities to give to education, and take initiatives to bring new policies into schools.

These evolutions have changed the relationships between the various levels of decision-making and execution. In some systems, decentralisation allows local and regional decision makers, and district and school leaders to weigh more in the policymaking process, and to adapt policies to certain local priorities. More generally, education systems are moving from essentially top-down structures to more horizontal interactions in which negotiation and co-construction are in order. These systems are non-linear; they rely on feedback to shape their own evolution. They operate on multiple time-scales and at several levels simultaneously (Burns and Köster, 2016[22]).

These new dynamics create more challenging situations for policy implementation. Change programmes in public organisations tend to fail for reasons such as a lack of vision, incapacity to communicate, or failure to strike the right balance between marginal changes and structural transformations (Kotter, 1995[23]; Keller and Price, 2011[24]). The issue of building and maintaining trust for instance, is crucial in complex systems (Cerna, 2014[25]). New mechanisms must make it possible to hold different actors accountable for their actions, since central governments do not necessarily control all aspects of the policy process. At the same time, strong accountability should not hinder education systems’ potential for innovation: OECD countries find ways to use accountability as a tool for improvement and innovation at the classroom, school, local and national levels (OECD, 2013[26]; Burns and Köster, 2016[22])

The complexity of education governance also affects a system’ s disposition for implementation. In decentralised systems, multiple levels of governance can result either in many layers in the implementation channels, or in different reforms or programmes to implement in similar places for instance. This crowded policy space may create reform fatigue and confusion in those that have to implement it (Honig, 2006[27]).

Implementing a new policy, bringing a change to the way education works in this environment is becoming more complex and challenging than in more hierarchical organisations (Van Der Voet, Kuipers and Groeneveld, 2015[28]). Reform initiatives and reactions to these changes no longer come from decisions made from the top down; rather, they result from more intricate interactions between multiple actors at various levels in the system. Central governments still play a decisive role in the policymaking process, if only to guarantee a coherent education system. Policy makers thus need to understand and to take into account the new challenges that complex education systems imply for policy implementation.

Overall, the research literature reveals a range of reasons preventing implementation from being effective, including a lack of focus on the implementation processes, the challenges of engaging people effectively in change and the new complexity in education governance. These call for the need to review current implementation approaches. Our main interest in analysing the existing frameworks for implementation is to assess whether they are adapted to education policymaking in the 21st century, and if they can support the development of professional practices that can contribute to effective implementation.

1.3.Methodology of the study

An analysis of the current situation and challenges in education policy implementation leads us to pose two central questions to guide our analysis: What does education policy implementation entail in theory and in practice? What are the determinants involved in the process of education policy implementation? Answering these will allow us to understand the field and explore the possibility of developing a framework to support education policy implementation.

To answer the questions, the paper follows a traditional literature review and qualitative research approach, which includes the review and preselection of the most relevant literature on the topic and its analysis based on a common analytical framework. To analyse the various theoretical approaches of education policy implementation, we compare the approaches and draw some conclusions on how to build a basic framework.

1.3.1. Research terms and process

To analyse education policy implementation, the search process focused on finding relevant literature that addresses issues in and determinants of the implementation of social policies (in education and healthcare, mainly). It encompassed the search for general theories of the policy process when they included specific reference to implementation; the search for OECD publications offering a conceptualisation of education policy implementation and documentation on country practices in education policy implementation and empirical studies. To ensure international coverage, the search was undertaken in English and French. Most of the sources found were in English, and from Western schools of thought.

The search terms used in the review of the literature were education policy implementation, education reform implementation and policy implementation. In French, the equivalent searches included: mise en oeuvre politique éducation/éducative, mise en œuvre reforme éducative/éducation, mise en œuvre politique. Following the initial search, further references were found in key articles’ bibliographies.

The literature selected was peer-reviewed or referenced by authorities in the field. If less reliable sources contributed interesting points, their qualified references were analysed, before being integrated to the literature review if relevant. The initial date for the literature search was 1970 to cover initial policy frameworks, but we analysed each approach taking into account recent updates of the different theories or frameworks, either by their original authors or by later contributors.

The literature on country practices was selected irrespective of whether there was information on their outcome. Countries’ operational documentation (e.g. action plan) was reviewed if the policy was considered as implemented, or in the process of being implemented. Empirical and qualitative studies were considered when they identified specific strategies and concrete measures used to implement policies

The search for references initially yielded several million results and was refined through the use of key words and concepts. After skimming around 2 500 abstracts, over 150 items of interest were selected based on the terms detailed above. These were analysed more thoroughly based on their contribution to the two main questions of this paper: what does education policy implementation entail in theory and in practice? What are the determinants involved in the process of education policy implementation?

1.3.2. Research results and framework for analysis

The search resulted in the identification of 18 frameworks or models related to education policy implementation, presented in a table in Annex 1. The frameworks aim to make sense of implementation by studying what factors influence the process and determine its outcomes. Some frameworks are focused on public policy implementation more generally -such as Sabatier’s Advocacy Coalition Framework (Jenkins-Smith et al., 2014[29]), while others focus on education policy implementation (Bell and Stevenson, 2015[30p Haddad and Demsky, 1995[31]). Fullan’s approach links implementation to the issue of educational change (2015[21]). But each adopts a specific perspective on implementation, which we analysed.

Six of the 18 frameworks come from OECD projects on education reform implementation (OECD, 2010[16|; OECD, 2011[32]) and system governance (Burns and Köster, 2016[22]). Their comparative perspective spans various types of education policies, governance models and implementation practices, which make them instrumental to this paper’s analysis. OECD studies also contribute a share of the country cases reviewed.

In addition to the 18 frameworks, the review included a range of theoretical approaches to education policy implementation. These were reviewed in terms of their definition of education policy implementation, the challenges and issues they emphasise, their explanations of specific determinants of implementation, and the policy and country cases they used (if any).

More than 20 publications reviewed present country practices and strategies. We reviewed the type of education policy implemented, the country or governance level concerned (e.g. a regional or local jurisdiction), whether the policy was effectively implemented (when known), the factors identified for success or failure of implementation, and whether an implementation strategy could be identified. The publications include mostly narrative studies, although empirical studies were reviewed whenever available.

Altogether, the frameworks and the complementary literature are the core knowledge which we explore to analyse how education policy implementation is defined, and to study its determinants and how they are organised. To undertake the analysis in a systematic fashion, after having reviewed different approaches, we select and adapt the methodology used by Nilsen (2015[33]) in a study of the various theoretical approaches to implementation in the healthcare sector.

More concretely, Nilsen reviews the different theories, models and frameworks in implementation science1, with the aim to translate research into practice. He suggests that while there is interest in the use of theories, models and frameworks to understand implementation mechanisms, there are difficulties in choosing the most appropriate ones. He concludes that many models do not identify or structure determinants of policy implementation associated to success and that one of the key issues is to find a clarifying taxonomy for the analysis.

To undertake his analysis, Nilsen provides a classification of determinants in the different health care policy …