EXECUTIVE SUMMARY

Parents, educators, government ministers and policymakers in all contexts and countries around the world are concerned with learning and how to improve it. There are many reasons for this, but none is more important than the fact that learning is at the heart of success at the individual, community and global levels. Learning First is the title of this report, with the strong implication that learning should be the foremost goal of education policies worldwide.

The present review seeks not only to explain why this is the case but also focuses on what we need to know—that is, what research is needed—in order to improve learning in the decades to come, particularly among those children most in need. This question is addressed in the following six sections.

1. Learning Goals and Research. The first section begins with a historical synopsis of international education goals put forward in 1990 at the World Conference on Education for All in Jomtien (Thailand), in 2000 at the Education for All conference in Dakar, and later in 2000 as a part of the UN Millennium Development Goals for 2015. In 2011, the Center for Universal Education at the Brookings Institution published A Global Compact on Learning: Taking Action on Education in Developing Countries, which stated that there is a “global learning crisis—which affects children and youth who are out of school with limited learning opportunities and those who are in school but not learning the skills they need for their futures.” The present review of learning research in low-income countries follows from that report. The overall purpose is to explore the most pressing learning issues today that require further research attention in the years to come.
    
2. Learning Definitions and Contexts. This section reviews how the field of education has defined learning over the years. Here, learning is defined as a modification of behavior due to experience—such as in knowledge, skills, attitudes and values. Three main principles of effective learning are suggested: individual active involvement, social participation, and meaningful engagement. As a way to emphasize the importance of learning contexts, three individual stories—Illa, a four-year-old Quechua-speaking girl in Peru; Pawan, an eight-year-old primary school student in urban India; and Rachida, a young illiterate woman in rural Morocco—are provided in order to better explain the importance of learning as a culturally specific phenomenon. These stories help to illustrate a more general learning framework, encompassing the relationship between two dimensions of learning—its processes and contexts. A discussion follows concerning the need to disaggregate learners and their learning contexts—between countries and within countries—as a way to overcome frequent and simplistic generalizations about how the “average” child learns.
    
3. Global Change and the Contexts of Learning. This section considers the issue of global change on how learning and learning contexts are being transformed around the world. For example, researchers need to pay more attention to the impact of migration on children’s learning and on educational systems more broadly. In each instance of translocation, children confront the challenges of adapting to a new environment that may include different languages, dialects or cultures within the nonformal learning contexts of daily life. Similarly, in formal education contexts, student migrants have to cope with contrasts in culture, lifestyle and language of schooling, and demonstrate skills and achievement that may vary dramatically with their culture of origin. Other changes due to globalization include increased multilingualism in schools, growing overcrowding in classrooms, inability to keep up with teacher training, changes in intergenerational learning, and the growing importance of 21st-century skills. Based on these observations, it is suggested that learning contexts and needs should be understood as a shifting target.

Downloads

• Download the full paper

INTRODUCTION

With fewer than three years until the planned end-date of the United Nations Millennium Development Goals (MDGs), attention is rapidly turning to what will follow. The elaboration of the next global development agenda is a complex, multi-pronged process that is academic, political and practical, involving experts from a myriad of social and economic sectors and representing a cross-section of constituencies. While the formal U.N. process is still in the early stages, the ongoing discourse (predominantly occurring in the global north, but not exclusively) has introduced several potential frameworks for this agenda. This paper describes the leading frameworks proposed for the post-2015 global development agenda and discusses how education and learning fit within each of those frameworks. While many within the education community are working to develop a cohesive movement to advance an “access plus learning” agenda, it remains equally important to engage proactively with the broader development community to ensure that education fits within the agreed upon overarching organizing framework.

The frameworks described below represent a snapshot of current thinking in 2012. On the road to 2015, the education community will need to refine and sharpen its thinking with respect to how learning is incorporated into the prevailing framework. The seven frameworks that will be addressed in this paper are:

1. Ending Absolute Poverty
2. Equity and Inclusion
3. Economic Growth and Jobs
4. Getting to Zero
5. Global Minimum Entitlements
6. Sustainable Development
7. Well-Being and Quality of Life

Downloads

• Download the full paper

INTRODUCTION

In an earlier policy brief, Where is the Learning? Measuring Schooling Efforts in Developing Countries, we drew attention to what was labeled “the global learning crisis.” While tremendous progress has been made over the past couple of decades to get tens of millions of additional children to enroll in school, progress in improving learning outcomes has been considerably less impressive. Although, shockingly, comprehensive learning outcome data are not available for most of the developing world, the many small scale, local or, in some cases, national studies that have been done show a dismal picture. For instance, Uwezo, an East African initiative, found that in Tanzania, only 44 percent of students in Grade 4 were able to read a basic story from Grade 2. Similarly, the Annual Status of Education Report (ASER) facilitated by Pratham found that in rural India, less than half of Grade 4 students were able to do basic subtraction. These examples demonstrate the gravity of “the global learning crisis” as students fail to master competencies appropriate for their grade level, hindering the development of life skills and success in further schooling, as well as performance in the labor market.

With about 61 million children in the developing world still not yet in school, it is too early to declare victory on the “enrollment agenda”. But we would do a disservice to the 250 million children around the world who fail to reach Grade 4 or attain minimum learning standards, if we don’t step up efforts to improve learning outcomes.

This policy brief is part of a larger effort to link resources in the education sector with outcome measures. As we have documented elsewhere, few countries systematically collect comprehensive financial data on education, although fortunately an increasing number of initiatives is trying to address this issue by producing, for instance, National Education Accounts (NEAs). When the focus of the sector changes from enrollment to enrollment plus learning, efforts to better grasp the size and use of financial resources should evolve accordingly. For instance, much learning takes place outside of the classroom, especially in the early years. For NEAs to be a useful tool for adjusting the allocation of scarce resources, the “learning” sector should be defined more broadly than the education or “schooling” sector. We will address this and related issues in a subsequent policy brief.

Once our focus becomes enrollment plus learning, we have to broaden our view and look at the entire environment in which a child develops skills, starting with the households in which children are born. It has beenknown for many decades and throughout the world, that among the best predictors of future school performance are some basic household characteristics, such as income and mother’s education level.

Data from international assessments also show a relationship between income and educational performance, exemplified by intra and intercountry results. In Colombia, average Trends in International Mathematics and Science Study (TIMSS) math scores at Grade 8 for the richest quintile of students were close to 100 points higher than those from the poorest quintile. On the other hand, the difference in average scores between the poorest quintile in the United States and the richest quintile in Colombia was about 50 points. Income is not the only predictor of success, as exemplified in Peru, where children whose mothers have completed primary school and whose maternal language is Spanish rather than an indigenous language, have a greater probability of reaching the appropriate school grade for their age. In Kenya, Uwezo found that the higher their father’s educational attainment, the more likely children were able to read a story at Grade 3 or attend extra tutoring sessions.

In addition, the larger environment (such as the village or the urban neighborhood) in which the young child grows up also has a major and lasting impact. In Tanzania, urban students in Grade 3 are three times more likely than their rural counterparts to meet standards in literacy and numeracy. Related to the impact of the larger environment, data from Nigeria suggest that girls are more disadvantaged in school attendance, as parents may be reluctant to send girls to school because of perceived fears for their safety while traveling and concerns about the physical strength required for walking the distance.

Clearly, especially in the early years, most learning takes place outside of the classroom. Consequently, children who grow up in deprived circumstances will start life with a disadvantage leading to a lack of learning in the early grades, which will have lifetime effects.

In the next section, we will summarize the evidence that the early years (ages 0 to 5) are crucial for subsequent learning achievements. From this evidence we conclude that many of the problems with learning outcomes in the developing world (and in many developed countries) need to be addressed well before school age. Before delving into what happens in schools, we explore the relationship between enrollment, learning and dropout. As the crux of this brief is to lay out the evidence on what contributes to learning, we must acknowledge the factors leading to low enrollment and dropout. Next, we turn our attention to what happens in schools and what can be done to improve these activities, as well as try to summarize the evidence about the relationship between specific school-based inputs and learning outcomes. As it turns out, this evidence is, in many cases, rather feeble. Therefore, we will first focus on school-level inputs that are necessary for a good learning environment, i.e. without which we cannot expect any learning to take place. Most of these inputs are rather obvious, but they are worth mentioning. Subsequently, we will discuss additional inputs that have proven to contribute to learning outcomes in some cases, but not in others. Clearly how these inputs are applied matters.

Next, we address factors that contribute to learning outside of a formal environment, after which we review issues in health and nutrition that are closely linked to learning outcomes. We then review the need for the collection and dissemination of learning assessments in order to impact further improvements in these areas and we try to answer the question: what are the building blocks for an education sector that promote learning? Finally we explore needs for future research in learning.

Downloads

• Download the full paper

Globalization has been hugely beneficial to Asia. Japan, South Korea, Taiwan, Malaysia, Singapore, Hong Kong, Thailand, and China have reaped lasting benefits from worldwide investment flows, knowledge exchanges, and rapid economic growth. And while globalization undoubtedly made the rich even richer, the poor also benefitted.

Several Asian economies saw the emergence of a large middle class and the virtual elimination of poverty. The rural poor received higher wages after finding better quality manufacturing jobs in urban centers. And with improvements in technology and expanded trade, there was optimism about job prospects. In ethnically diverse societies such as Malaysia, globalization contributed significantly to the reduction of racial tensions rather than exacerbate them, contrary to what is happening in the West.

There is no question that the West has also benefited from globalization. The United Kingdom and the United States have seen huge gains in the services sector, especially in financial services. However, the accompanying income inequality is of a different hue than in Asia. Younger, better educated workers located in cosmopolitan urban centers such as New York and London have seen a phenomenal increase in their income. On the other hand, older, less educated workers in the rusting industrial belts of northern England and America have lost their jobs to manufacturers overseas. Instead of jobs with good growth prospects enjoyed by several generations in the past, the quality of jobs has deteriorated and there is little hope among the rust belt’s working class that this situation can be turned around. Worst, there is a perception that politicians don’t care. The recent voting patterns in the U.S. and the U.K. are a clear reflection of this despondence.

How did Asia achieve a shared prosperity from globalization with consistent domestic political support while the rich countries have struggled and are suffering the political blowback? 

The answer may lie in the heavy investment made by Asian governments in human capital (education and health) to prepare the workforce to take advantage of the high wage manufacturing jobs created by globalized investment. This was complemented by public investment in infrastructure to continue to attract foreign investment. The fiscal deficits associated with large public investment in human capital and physical infrastructure were tolerated because the political and economic benefits of preparing the workforce for new jobs were considered worthwhile objectives.

Both the U.S. and the U.K, in contrast, have underinvested in infrastructure and in “skilling up” the labor force to make the transition to new and better jobs from the ones lost to lower wage workers in Asia.  In the U.K, it happened under the watch of the incumbent conservative government. While presenting to Parliament the result of the recent referendum to the European Union, Prime Minister David Cameron spoke proudly of leaving behind a sound economy resting on the pillar of a sharp fiscal retrenchment—low taxes and even lower public expenditure. One result of this “sound” economy is that a large number of people are stuck in dead-end jobs and are looking for opportunities to vent their frustration.

In the U.S., the Obama administration has been hemmed in by the recalcitrant Republican Congress. Badly needed public investment in health and education to prepare workers and an overdue upgrade of infrastructure to attract investment have been thwarted by a Congress wedded to fiscal austerity. This has prolonged the pain of transition to new jobs.

The long and painful transition to productive jobs has resulted in the clamor for reneging on globalization commitments. This is misplaced because protecting jobs that are best done elsewhere is not possible without putting curbs on investment. That would be moving towards a world that globalizes misery. There is thus no alternative to a proactive government that eases the transition to new jobs in rich countries.

Of course, Asia had the advantage of preparing its work force for known job streams. Rich countries, on the other hand, have to discover new productive jobs. However, we do know that discovery is more likely if education standards improve, physical infrastructure is cutting edge, and science and research are well-funded.  

Rich countries don’t have to give up on manufacturing as a source of employment. Germany has shown the way to creating high-end manufacturing jobs in a rich-country setting. It has a highly skilled work force that produces technology-intensive products which generate a large trade surplus. There is little support in Germany for reneging on global commitments.

Dying cities, dead-end jobs, and a seemingly uncaring government feed into the perception that living standards will continue to fall. Demagogues exploiting ethnicity point the finger at immigrants and have succeeded in directing rich-country worker ire at them. This is a far cry from the democratic vision rich democracies should aspire to and is not in any away a solution to these problems.  The protest should be aimed, instead, at elected governments to play their role in facilitating the transition to the next generation of jobs.     

The world has paid heavily for Europe’s nationalistic ambitions—colonial subjugation of Africa and Asia and the two world wars are the most egregious examples. The EU is an attempt to tame those impulses by seeking to cooperatively address common challenges instead of competing for narrower nationalistic objectives. The dissolution of the EU and the weakening of other multilateral institutions because of rich countries’ failure to rise up to the globalization challenge would be truly retrogressive. 

Event Information

Seven years ago, New Zealand recognized that if its people did not have sufficient assets as they aged, they would either face economic stress in retirement or place pressure on the government for costly additional benefits, and thus the KiwiSaver program was born. Designed to help citizens build retirement security, it guides individuals with limited financial experience while also giving them complete control of their finances. Benefits of this national automatic enrollment retirement savings plan include a $1,000 kick-start, employer contributions, and an annual tax credit. New Zealand Since its inception in July 2007, KiwiSaver has been deemed a great success, with over half of the eligible population as members, and over 70 percent of 18-24 year olds participating. Although membership continues to grow, it is at a slower rate than that seen in previous years.

Could the success of KiwiSaver mean that a similar program – at either the national or state level – might work here? On July 8^th, Diana Crossan, former Retirement Commissioner for New Zealand, will offer her insights into the KiwiSaver program and its impact on New Zealand saving, retirement security, and financial literacy. Ben Harris and David John, deputy directors of the Retirement Security Project at Brookings, will reflect on the role such a program might play in the U.S.

Email international@aarp.org to RSVP » 

Event Information

On Wednesday, June 2, the Center for Universal Education at Brookings hosted a discussion on the need to refocus the international education dialogue from school enrollment to learning achieved in developing countries. Participants, who included education experts from academia, international organizations and government, assessed the current state of systematic efforts at the global level to measure learning outcomes.

Center for Universal Education Co-Director and Senior Fellow Jacques van der Gaag opened the event by charting the landscape of learning, including education outside the primary school classroom, during early childhood development and the importance of acquiring both cognitive and non-cognitive skills for ensuring learning outcomes.

View the event summary »

Event Materials

• 20100602_education_learning_presentation_clarke
• 20100602_education_learning_presentation_gove
• 20100602_education_learning_presentation_naidoo
• 20100602_education_learning_presentation_savelyev
• 20100602_education_learning_presentation_woessman

INTRODUCTION—

Achieving universal education is a twofold challenge: to get children and youth into school and then to teach them something meaningful while they are there. While important progress has been made on the first challenge, there is a crisis unfolding in relation to learning. Around the world, there have been major gains in primary school enrollment partly due to the United Nations’ Millennium Development Goals and the abolition of school fees by many national governments. However in many countries, students are spending years in school without learning core competencies, such as reading and writing. To address this learning crisis, the global community and national governments need to place a much greater focus on the ultimate objective of education—to acquire knowledge and develop skills.

This shift in focus away from just enrollment to enrollment plus quality learning requires measuring learning outcomes. However, the global education community is not yet systematically using effective instruments for measuring primary school learning in low- and middle-income countries. This policy brief reviews the global efforts among the primary donors to support the measurement of learning outcomes. It then suggests steps needed to transition global education policy into a new paradigm of enrollment plus quality learning, which includes: scaling up the implementation of national education accounts and national assessment systems; increasing attention to monitoring early learning during child development to improve readiness for school; and expanding the systematic use of simple assessments of basic cognitive functions in the early grades to help teachers improve their practice.

Downloads

• Read the full paper

Financial Data as Driving Force Behind Improved Learning

During the past decade, school enrollments have increased dramatically, mostly thanks to UNESCO’s Education for All (EFA) movement and the UN Millennium Development Goals. From 1999 to 2008, an additional 52 million children around the world enrolled in primary schools, and the number of out-of-school children fell by 39 million. In Sub-Saharan Africa alone, enrollment rates rose by one-third during that time, even with large population increases in school-age children.

Yet enrollment is not the only indicator of success in education, and does not necessarily translate into learning. Even with these impressive gains in enrollment, many parts of the world, and particularly the poorest areas, now face a severe learning crisis. The latest data in the EFA Global Monitoring Report 2011 reveal poor literacy and numeracy skills for millions of students around the world. In Malawi and Zambia, more than one-third of sixth-grade students had not achieved the most basic literacy skills. In El Salvador, just 13 percent of third-grade students passed an international mathematics exam. Even in middle-income countries such as South Africa and Morocco, the majority of students had not acquired basic reading skills after four years of primary education. Although the focus on children out of school is fully justified, given that they certainly lack learning opportunities, the failure to focus on learning also does a disservice to the more than 600 million children in the developing world who are already in school but fail to learn very basic skills.

Downloads

• Download the Full Paper

Event Information

Despite the notable success in enrolling children in primary school over the past decade, the education agenda is unfinished as millions of children are still excluded from learning opportunities and millions more leave school without having acquired the essential knowledge and skills needed to participate in society.

On January 27, the Center for Universal Education at Brookings hosted a half-day conference that focused on the research examining “what works in education” to achieve improved learning opportunities and outcomes. In addition to hearing from researchers studying the effectiveness of various education strategies, participants discussed how to facilitate a future research agenda that could have the most meaningful impact on learning. Senior Fellow Jacques van der Gaag moderated the discussion.

View the full event summary »

Transcript

• Improving Education in the Developing World: Evidence from Randomized Evaluations Presentation (.pdf)
• Sociocultural Gradients for Language Scores by Country Slide (.pdf)
• School Resources and Educational Outcomes in Developing Countries Presentation (.pdf)
• Making Learning for All a Reality Presentation (.pdf)
• Addressing the Global Learning Crisis Presentation (.pdf)

Event Materials

• 0127_improving_education_in_developing_world
• 0127_slide_vandergaag
• 0127_edinptoh
• 0127_king
• 0127_wagner

INTRODUCTION

In an earlier policy brief, Where is the Learning? Measuring Schooling Efforts in Developing Countries, we drew attention to what was labeled “the global learning crisis.” While tremendous progress has been made over the past couple of decades to get tens of millions of additional children to enroll in school, progress in improving learning outcomes has been considerably less impressive. Although, shockingly, comprehensive learning outcome data are not available for most of the developing world, the many small scale, local or, in some cases, national studies that have been done show a dismal picture. For instance, Uwezo, an East African initiative, found that in Tanzania, only 44 percent of students in Grade 4 were able to read a basic story from Grade 2. Similarly, the Annual Status of Education Report (ASER) facilitated by Pratham found that in rural India, less than half of Grade 4 students were able to do basic subtraction. These examples demonstrate the gravity of “the global learning crisis” as students fail to master competencies appropriate for their grade level, hindering the development of life skills and success in further schooling, as well as performance in the labor market.

With about 61 million children in the developing world still not yet in school, it is too early to declare victory on the “enrollment agenda”. But we would do a disservice to the 250 million children around the world who fail to reach Grade 4 or attain minimum learning standards, if we don’t step up efforts to improve learning outcomes.

This policy brief is part of a larger effort to link resources in the education sector with outcome measures. As we have documented elsewhere, few countries systematically collect comprehensive financial data on education, although fortunately an increasing number of initiatives is trying to address this issue by producing, for instance, National Education Accounts (NEAs). When the focus of the sector changes from enrollment to enrollment plus learning, efforts to better grasp the size and use of financial resources should evolve accordingly. For instance, much learning takes place outside of the classroom, especially in the early years. For NEAs to be a useful tool for adjusting the allocation of scarce resources, the “learning” sector should be defined more broadly than the education or “schooling” sector. We will address this and related issues in a subsequent policy brief.

Once our focus becomes enrollment plus learning, we have to broaden our view and look at the entire environment in which a child develops skills, starting with the households in which children are born. It has beenknown for many decades and throughout the world, that among the best predictors of future school performance are some basic household characteristics, such as income and mother’s education level.

Data from international assessments also show a relationship between income and educational performance, exemplified by intra and intercountry results. In Colombia, average Trends in International Mathematics and Science Study (TIMSS) math scores at Grade 8 for the richest quintile of students were close to 100 points higher than those from the poorest quintile. On the other hand, the difference in average scores between the poorest quintile in the United States and the richest quintile in Colombia was about 50 points. Income is not the only predictor of success, as exemplified in Peru, where children whose mothers have completed primary school and whose maternal language is Spanish rather than an indigenous language, have a greater probability of reaching the appropriate school grade for their age. In Kenya, Uwezo found that the higher their father’s educational attainment, the more likely children were able to read a story at Grade 3 or attend extra tutoring sessions.

In addition, the larger environment (such as the village or the urban neighborhood) in which the young child grows up also has a major and lasting impact. In Tanzania, urban students in Grade 3 are three times more likely than their rural counterparts to meet standards in literacy and numeracy. Related to the impact of the larger environment, data from Nigeria suggest that girls are more disadvantaged in school attendance, as parents may be reluctant to send girls to school because of perceived fears for their safety while traveling and concerns about the physical strength required for walking the distance.

Clearly, especially in the early years, most learning takes place outside of the classroom. Consequently, children who grow up in deprived circumstances will start life with a disadvantage leading to a lack of learning in the early grades, which will have lifetime effects.

In the next section, we will summarize the evidence that the early years (ages 0 to 5) are crucial for subsequent learning achievements. From this evidence we conclude that many of the problems with learning outcomes in the developing world (and in many developed countries) need to be addressed well before school age. Before delving into what happens in schools, we explore the relationship between enrollment, learning and dropout. As the crux of this brief is to lay out the evidence on what contributes to learning, we must acknowledge the factors leading to low enrollment and dropout. Next, we turn our attention to what happens in schools and what can be done to improve these activities, as well as try to summarize the evidence about the relationship between specific school-based inputs and learning outcomes. As it turns out, this evidence is, in many cases, rather feeble. Therefore, we will first focus on school-level inputs that are necessary for a good learning environment, i.e. without which we cannot expect any learning to take place. Most of these inputs are rather obvious, but they are worth mentioning. Subsequently, we will discuss additional inputs that have proven to contribute to learning outcomes in some cases, but not in others. Clearly how these inputs are applied matters.

Next, we address factors that contribute to learning outside of a formal environment, after which we review issues in health and nutrition that are closely linked to learning outcomes. We then review the need for the collection and dissemination of learning assessments in order to impact further improvements in these areas and we try to answer the question: what are the building blocks for an education sector that promote learning? Finally we explore needs for future research in learning.

Downloads

• Download the full paper

Event Information

In 2015, 193 countries adopted the Sustainable Development Goals (SDGs), a new global agenda that is more ambitious than the preceding Millennium Development Goals and aims to make progress on some of the most pressing issues of our time. Goal 4, "To ensure inclusive and quality education for all, with relevant and effective learning outcomes," challenges the international education community to meet universal access plus learning by 2030. We know that access to primary schooling has scaled up rapidly over previous decades, but what can be learned from places where transformational changes in learning have occurred? What can governments, civil society, and the private sector do to more actively scale up quality learning?

On April 18-19, the Center for Universal Education (CUE) at Brookings launched "Millions Learning: Scaling Up Quality Education in Developing Countries," a comprehensive study that examines where learning has improved around the world and what factors have contributed to that process. This two-day event included two sessions. Monday, April 18 focused on the role of global actors in accelerating progress to meeting the SDGs. The second session on Tuesday, April 19 included a presentation of the Millions Learning report followed by panel discussions on the role of financing and technology in scaling education in developing countries.

 Join the conversation on Twitter #MillionsLearning

Video

• Getting millions to learn: What will it take to accelerate progress on meeting the Sustainable Development Goals?
• Scaling quality education: The launch of the Millions Learning report
• Do funders help or hinder scaling in education?
• What role can technology play in scaling education?

Audio

• Getting millions to learn: What will it take to accelerate progress on meeting the Sustainable Development Goals?

Transcript

• Uncorrected Transcript - Day 1 (.pdf)
• Uncorrected Transcript - Day 2 (.pdf)

Event Materials

• 20160418_millions_learning_transcript
• 20160419_millions_learning_transcript

Editor's Note: The introduction to the 2015 Brown Center Report on American Education appears below. Use the Table of Contents to navigate through the report online, or download a PDF of the full report.

TABLE OF CONTENTS

Part I: Girls, Boys, and Reading

Part II: Measuring Effects of the Common Core

Part III: Student Engagement

INTRODUCTION

The 2015 Brown Center Report (BCR) represents the 14^th edition of the series since the first issue was published in 2000.  It includes three studies.  Like all previous BCRs, the studies explore independent topics but share two characteristics: they are empirical and based on the best evidence available.  The studies in this edition are on the gender gap in reading, the impact of the Common Core State Standards -- English Language Arts on reading achievement, and student engagement.

Part one examines the gender gap in reading.  Girls outscore boys on practically every reading test given to a large population.  And they have for a long time.  A 1942 Iowa study found girls performing better than boys on tests of reading comprehension, vocabulary, and basic language skills.  Girls have outscored boys on every reading test ever given by the National Assessment of Educational Progress (NAEP)—the first long term trend test was administered in 1971—at ages nine, 13, and 17.  The gap is not confined to the U.S.  Reading tests administered as part of the Progress in International Reading Literacy Study (PIRLS) and the Program for International Student Assessment (PISA) reveal that the gender gap is a worldwide phenomenon.  In more than sixty countries participating in the two assessments, girls are better readers than boys. 

Perhaps the most surprising finding is that Finland, celebrated for its extraordinary performance on PISA for over a decade, can take pride in its high standing on the PISA reading test solely because of the performance of that nation’s young women.  With its 62 point gap, Finland has the largest gender gap of any PISA participant, with girls scoring 556 and boys scoring 494 points (the OECD average is 496, with a standard deviation of 94).   If Finland were only a nation of young men, its PISA ranking would be mediocre.

Part two is about reading achievement, too. More specifically, it’s about reading and the English Language Arts standards of the Common Core (CCSS-ELA).  It’s also about an important decision that policy analysts must make when evaluating public policies—the determination of when a policy begins. How can CCSS be properly evaluated? 

Two different indexes of CCSS-ELA implementation are presented, one based on 2011 data and the other on data collected in 2013.  In both years, state education officials were surveyed about their Common Core implementation efforts.  Because forty-six states originally signed on to the CCSS-ELA—and with at least forty still on track for full implementation by 2016—little variability exists among the states in terms of standards policy.  Of course, the four states that never adopted CCSS-ELA can serve as a small control group.  But variation is also found in how the states are implementing CCSS.  Some states are pursuing an array of activities and aiming for full implementation earlier rather than later.  Others have a narrow, targeted implementation strategy and are proceeding more slowly. 

The analysis investigates whether CCSS-ELA implementation is related to 2009-2013 gains on the fourth grade NAEP reading test.  The analysis cannot verify causal relationships between the two variables, only correlations.  States that have aggressively implemented CCSS-ELA (referred to as “strong” implementers in the study) evidence a one to one and one-half point larger gain on the NAEP scale compared to non-adopters of the standards.  This association is similar in magnitude to an advantage found in a study of eighth grade math achievement in last year’s BCR.  Although positive, these effects are quite small.  When the 2015 NAEP results are released this winter, it will be important for the fate of the Common Core project to see if strong implementers of the CCSS-ELA can maintain their momentum.

Part three is on student engagement.  PISA tests fifteen-year-olds on three subjects—reading, math, and science—every three years.  It also collects a wealth of background information from students, including their attitudes toward school and learning.  When the 2012 PISA results were released, PISA analysts published an accompanying volume, Ready to Learn: Students’ Engagement, Drive, and Self-Beliefs, exploring topics related to student engagement.

Part three provides secondary analysis of several dimensions of engagement found in the PISA report.  Intrinsic motivation, the internal rewards that encourage students to learn, is an important component of student engagement.  National scores on PISA’s index of intrinsic motivation to learn mathematics are compared to national PISA math scores.  Surprisingly, the relationship is negative.  Countries with highly motivated kids tend to score lower on the math test; conversely, higher-scoring nations tend to have less-motivated kids. 

The same is true for responses to the statements, “I do mathematics because I enjoy it,” and “I look forward to my mathematics lessons.”  Countries with students who say that they enjoy math or look forward to their math lessons tend to score lower on the PISA math test compared to countries where students respond negatively to the statements.  These counterintuitive finding may be influenced by how terms such as “enjoy” and “looking forward” are interpreted in different cultures.  Within-country analyses address that problem.  The correlation coefficients for within-country, student-level associations of achievement and other components of engagement run in the anticipated direction—they are positive.  But they are also modest in size, with correlation coefficients of 0.20 or less. 

Policymakers are interested in questions requiring analysis of aggregated data—at the national level, that means between-country data.  When countries increase their students’ intrinsic motivation to learn math, is there a concomitant increase in PISA math scores?  Data from 2003 to 2012 are examined.  Seventeen countries managed to increase student motivation, but their PISA math scores fell an average of 3.7 scale score points.  Fourteen countries showed no change on the index of intrinsic motivation—and their PISA scores also evidenced little change.  Eight countries witnessed a decline in intrinsic motivation.  Inexplicably, their PISA math scores increased by an average of 10.3 scale score points.  Motivation down, achievement up.

Correlation is not causation.  Moreover, the absence of a positive correlation—or in this case, the presence of a negative correlation—is not refutation of a possible positive relationship.  The lesson here is not that policymakers should adopt the most effective way of stamping out student motivation.  The lesson is that the level of analysis matters when analyzing achievement data.  Policy reports must be read warily—especially those freely offering policy recommendations.  Beware of analyses that exclusively rely on within- or between-country test data without making any attempt to reconcile discrepancies at other levels of analysis.  Those analysts could be cherry-picking the data.  Also, consumers of education research should grant more credence to approaches modeling change over time (as in difference in difference models) than to cross-sectional analyses that only explore statistical relationships at a single point in time. 

Downloads

• Download the report

Tom Loveless, a nonresident senior fellow in Governance Studies, explains his latest research on measuring achievement of American students.

“The bottom line here: the implementation of the common core has appeared to have very little impact on student achievement,” Loveless says. In this episode, he discusses whether the common core is failing our students, whether AP achievement is indicative of student success, and the role of principals as instructional leaders.

Also in this episode: Get to know Constanze Stelzenmüller, the Robert Bosch Senior Fellow in the Center on the United States and Europe, during our "Coffee Break” segment. Also stay tuned to hear the final episode in our centenary series with current and past Brookings scholars.

Show Notes:

The Brown Center Report on American Education

Brookings Centenary Timeline

Subscribe to the Brookings Cafeteria on iTunes, listen in all the usual places, and send feedback email to BCP@Brookings.edu.