Event Information

The biomedical innovation ecosystem continues to evolve and enhance the processes by which treatments are developed and delivered to patients. Given this changing biomedical innovation landscape, it is imperative that all stakeholders work to ensure that development programs, regulatory practices, and the policies that enable them are aligned on and achieving a common set of goals. This will require a thorough reexamination of our understanding of biomedical innovation – and the subsequent ways in which we seek to incentivize it – in order to more effectively bridge research and analysis of the process itself with the science and policy underpinning it.

Traditional research into the efficiency and effectiveness of drug development programs has tended to focus on the ‘inputs’ and process trends in product development, quantifying the innovation as discrete units. At the opposite end of the research spectrum are potential measures that could be categorized as “value” or “outcomes” metrics. Identifying the appropriate measures across this spectrum – from inputs and technological progress through outcomes and value – and how such metrics can be in conversation with each other to improve the innovation process will be the focus of this expert workshop. On October 14, the Center for Health Policy at Brookings, under a cooperative agreement with the U.S. Food and Drug Administration, convened a roundtable discussion that engaged key stakeholders from throughout the innovation ecosystem to explore the factors and characteristics that could improve our understanding of what constitutes modern “innovation” and how best to track its progress.

Event Materials

• FINAL 1014 BrookingsFDA Agenda
• FINAL 1014 BrookingsFDA Participant List

INTRODUCTION

In recent years, there has been a resurgence of interest in the issue of inequality. Part of this resurgence can be traced to new evidence of persistent and widening wealth gaps. Average incomes may be converging globally as a result of high growth in emerging markets, stronger growth in many poor countries, and slow growth in rich countries. However, the evidence also shows that within countries a parallel process of income divergence, marginalization and rising inequality is also taking place. Put differently, the rising tide of global prosperity is not lifting all boats.

Much of the international debate on inequality focuses on the distribution of income across and within countries. Other dimensions of inequality have received less attention. This is unfortunate. Amartya Sen has described development as “a process of expanding the real freedoms that people enjoy” by building human capabilities or their capacity to lead the kind of life they value. Income is a means to that end but it is a limited indicator of well-being. Moreover, a person’s income reflects not just personal choice but also their opportunities for improving health, literacy, political participation and other areas. Education is one of the most basic building blocks for the “real freedoms” that Sen describes. People denied the chance to develop their potential through education face diminished prospects and more limited opportunities in areas ranging from health and nutrition, to employment, and participation in political processes. In other words, disparities in education are powerfully connected to wider disparities, including international and intra-country income inequalities. This is why education has been identified as one of the most critical factors in breaking down the disadvantages and social inequalities that are limiting progress toward the United Nations’ Millennium Development Goals (MDGs)—development targets adopted by the international community for 2015.

Understanding patterns of educational inequality is critical at many levels. Ethical considerations are of paramount importance. Most people would accept that children’s educational achievements should not be dictated by the wealth of their parents, their gender, their race or their ethnicity. Disparities in educational opportunities are not just inequalities in a technical sense, they are also fundamental in equities—they are unjust and unfair. In an influential paper, John Roemer differentiated between inequalities that reflect factors such as luck, effort and reasonable reward, and those attributable to circumstances that limit opportunity (Roemer 1988).1 While the dividing line may often be blurred, that distinction has an intuitive appeal. Most people have a high level of aversion to the restrictions on what people—especially children—are able to achieve as a result of disparities and inherited disadvantages that limit access to education, nutrition or health care (Wagstaff, 2002). There is a wide body of opinion across political science, philosophy and economics that equal opportunity—as distinct from equality of outcomes—is a benchmark of egalitarian social justice. The theories of distributive justice associated with thinkers such as Amartya Sen, John Rawls, Ronald Dworkin and John Roemer argue, admittedly from very different perspectives, that public policy should aim at equalizing opportunity to counteract disadvantages associated with exogenous circumstances over which individuals or social groups have no control. Given the role of education as a potential leveler of opportunity, it is a national focal point for redistributive social justice.

Considerations of economic efficiency reinforce the ethical case for equalizing educational opportunities. Education is a powerful driver of productivity, economic growth, and innovation. Econometric modeling for both rich and poor countries suggests that an increase in learning achievement (as measured by test score data) of one standard deviation is associated on average with an increase in the long-run growth rate of around 2 percent per capita annually (Hanushek and Wößmann, 2010; Hanushek, 2009; Hanushek and Wößmann, 2008). Such evidence points to the critical role of education and learning in developing a skilled workforce. Countries in which large sections of the population are denied a quality education because of factors linked to potential wealth, gender, ethnicity, language and other markers for disadvantage are not just limiting a fundamental human right. They are also wasting a productive resource and undermining or weakening the human capital of the economy.

International development commitments provide another rationale for equalizing educational opportunities. This is for two reasons. First, the commitments envisage education for all and achievement of universal primary education by 2015. Second, there is mounting evidence that inequality is acting as a brake on progress toward the 2015 goals. Since around 2005, the rate of decline in the out-of-school population has slowed dramatically. Based on current trends, there may be more children out of school in 2015 than there were in 2009. Caution has to be exercised in interpreting short-run trends, especially given the weakness of data. However, the past three editions of the UNESCO Education for All Global Monitoring Report (GMR) have highlighted the role of inequality in contributing to the slowdown with governments struggling to reach populations that face deeply entrenched disadvantages (UNESCO, 2008, 2010, 2011). Therefore, picking up the pace toward the 2015 goals requires a strengthened focus on equity and strategies that target the most marginalized groups and regions of the world (Sumner and Tiwari, 2010; UN-DESA, 2009; UNESCO, 2010). It should be added that disparities in education relate not just to access, but also to learning achievement levels.

Accelerated progress in education would generate wider benefits for the MDGs. Most of the world’s poorest countries are off-track for the 2015 MDG target of halving income poverty and a long way from reaching the targets on child survival, maternal health and nutrition. Changing this picture will require policy interventions at many levels. However, there is overwhelming evidence showing that education—especially of young girls and women—can act as a potent catalyst for change. On one estimate, if all of sub-Saharan Africa’s mothers attained at least some secondary education, there would be 1.8 million fewer child deaths in the region each year. Thus while education may lack the “quick fix” appeal of vaccinations, it can powerfully reinforce health policy interventions.

Downloads

• Download the full report

Event Information

Approximately two billion adults across the world lack access to formal financial services. To address this particular economic challenge, many developing countries have made significant efforts to expand access to and use of affordable financial services for the world’s poor. Financial inclusion can be achieved via traditional banking offerings, but also through digital financial services such as mobile money, among other innovative approaches.

The Brookings Financial and Digital Inclu­sion Project (FDIP) Report and Scorecard seeks to help answer a set of fundamental questions about today’s global financial inclusion efforts, including;

1. Do country commitments make a difference in progress toward financial inclusion?
2. To what extent do mobile and other digital technologies advance finan­cial inclusion?
3. What legal, policy, and regulatory approaches promote financial inclusion? 

To answer these questions, Brookings experts John D. Villasenor, Darrell M. West, and Robin J. Lewis analyzed finan­cial inclusion in 21 geographically, economically, and politically diverse countries. This year’s report and scorecard is the first of a series of annual reports examining financial inclusion activities and assessing usage of financial services in selected countries around the world. 

On August 26, the Center for Technology Innovation at Brookings held a forum to launch the 2015 FDIP Report and discuss key research findings and recommendations. Financial inclusion experts from the public and private sectors also joined the discussion.

Join the conversation on Twitter at #FinancialInclusion and @BrookingsGov

Video

• Measuring progress on financial and digital inclusion

Audio

• Measuring progress on financial and digital inclusion

Transcript

• Uncorrected Transcript (.pdf)

Event Materials

• 20150826_financial_inclusion_transcript

Event Information

On Wednesday, June 2, the Center for Universal Education at Brookings hosted a discussion on the need to refocus the international education dialogue from school enrollment to learning achieved in developing countries. Participants, who included education experts from academia, international organizations and government, assessed the current state of systematic efforts at the global level to measure learning outcomes.

Center for Universal Education Co-Director and Senior Fellow Jacques van der Gaag opened the event by charting the landscape of learning, including education outside the primary school classroom, during early childhood development and the importance of acquiring both cognitive and non-cognitive skills for ensuring learning outcomes.

View the event summary »

Event Materials

• 20100602_education_learning_presentation_clarke
• 20100602_education_learning_presentation_gove
• 20100602_education_learning_presentation_naidoo
• 20100602_education_learning_presentation_savelyev
• 20100602_education_learning_presentation_woessman

INTRODUCTION—

Achieving universal education is a twofold challenge: to get children and youth into school and then to teach them something meaningful while they are there. While important progress has been made on the first challenge, there is a crisis unfolding in relation to learning. Around the world, there have been major gains in primary school enrollment partly due to the United Nations’ Millennium Development Goals and the abolition of school fees by many national governments. However in many countries, students are spending years in school without learning core competencies, such as reading and writing. To address this learning crisis, the global community and national governments need to place a much greater focus on the ultimate objective of education—to acquire knowledge and develop skills.

This shift in focus away from just enrollment to enrollment plus quality learning requires measuring learning outcomes. However, the global education community is not yet systematically using effective instruments for measuring primary school learning in low- and middle-income countries. This policy brief reviews the global efforts among the primary donors to support the measurement of learning outcomes. It then suggests steps needed to transition global education policy into a new paradigm of enrollment plus quality learning, which includes: scaling up the implementation of national education accounts and national assessment systems; increasing attention to monitoring early learning during child development to improve readiness for school; and expanding the systematic use of simple assessments of basic cognitive functions in the early grades to help teachers improve their practice.

Downloads

• Read the full paper

Editor’s note: The following remarks were prepared and delivered by Gary Burtless at a roundtable sponsored by the American Tax Policy Institute on January 7, 2016. Video of Burtless’ remarks are also available on the Institute’s website. Download the related slides at the right. 

To eliminate suspense from the start: Nothing I am about to say undermines the popular narrative about recent inequality trends.  For the past 35 years, U.S. inequality has increased.  Inequality has increased noticeably, no matter what income definition you care to use.  A couple of things you read in the newspaper are untrue under some income definitions. For example, under a comprehensive income definition it is false to claim that all the income gains of the past 2 or 3 decades have gone to the top 1 percent, or the top 5 percent, or the top 10 percent of income recipients.  Middle- and low-income Americans have managed to achieve income gains, too, as we shall see.

Tax policy certainly affects overall inequality, but I shall leave it for Scott, David, and Tracy to take that up. Let me turn to my main job, which is to distinguish between different reasonable income measures.

The crucial thing to know is that contradictory statements can be made about some income trends because of differences in the definition of income.  In general, the most pessimistic statements about trends rely on an income definition that is restrictive in some way.  The definition may exclude important income items, items, for example, that tend to equalize or boost family incomes.  The definition may leave out adjustments to income … adjustments that tend to boost the rate of income gain for low- or middle-income recipients, but not for top-income recipients.

The narrowest income definition commonly used to evaluate income trends is Definition #1 in my slide, “pretax private, cash income.”  Columnists and news reporters are unknowingly using this income definition when they make pronouncements about the income share of the “top 1 percent.”  The data about income under this definition are almost always based on IRS income tax returns, supplemented with a bit of information from the Commerce Department’s National Income and Product Account (NIPA) data file.

The single most common income definition used to assess income trends and inequality is the Census Bureau’s “money income” definition, Definition #2 on the slide.  It is just the same as the first definition I mentioned, except this income concept also includes government cash transfer payments – Social Security, unemployment insurance, cash public assistance, Veterans’ benefits, etc.

A slightly more expansive definition (#3) also adds food stamp (or SNAP) benefits plus other government benefits that are straightforward to evaluate. Items of this kind include the implicit rent subsidy low-income families receive in publicly-subsidized housing, school lunch subsides, and means-tested home heating subsidies.

Now we come to subtractions from income. These typically reflect families’ tax obligations.  The Census Bureau makes estimates of state and federal income tax liabilities as well as payroll taxes owed by workers (though not by their employers).  Since income and payroll taxes subtract from the income available to pay for other stuff families want to buy, it seems logical to also subtract them from countable income. This is done under income Definition #4.  Some tax obligations – notably the Earned Income Credit (EIC) – are in fact subtractions from taxes owed, which would not be a problem in the case of families that still owe positive taxes to the government.  However, the EIC is refundable to taxpayers, meaning that some families have negative tax liabilities:  The government owes them money.  In this case, if you do not take taxes into account you understate low-income families’ incomes, even as you’re overstating the net incomes available to middle- and high-income families.

Now let’s get a bit more complicated.  Forget what I said about taxes, because our next income definition (#5) also ignores them.  It is an even-more-comprehensive definition of gross or pretax income.  In addition to all those cash and near-cash items I mentioned in Definition #3, Definition #5 includes imputed income items, such as: 

• The value of your employer’s premium contribution to your employee health plan;
• The value of the government’s subsidy to your public health plan – Medicare, Medicaid, state CHIP plans, etc.
• Realized taxable gains from the sale of assets; and
• Corporate income that is earned by companies in which you own a share even though it is not income that is paid directly to you.

This is the most comprehensive income definition of which I am aware that refers to gross or pre-tax income.

Finally we have Definition #6, which subtracts your direct and indirect tax payments.  The only agency that uses this income definition is principally interested in the Federal budget, so the subtractions are limited to Federal income and payroll taxes, Federal corporate income taxes, and excise taxes.

Before we go into why you should care about any of these definitions, let me mention a somewhat less important issue, namely, how we define the income-sharing group over which we estimate inequality.  The most common assessment unit for income included under Definition #1 (“Pre-tax private cash income”) is the Federal income tax filing unit.  Sometimes this unit has one person; sometimes 2 (a married couple); and sometimes more than 2, including dependents.

The Census Bureau (and, consequently, most users of Census-published statistics) mainly uses “households” as reference units, without any adjustment for variations in the size of different households.  The Bureau’s median income estimate, for example, is estimated using the annual “money income” of households, some of which contain 1 person, some contain 2, some contain 3, and so on.

Many economists and sociologists find this unsatisfactory because they think a $20,000 annual income goes a lot farther if it is supporting just one person rather than 12.  Therefore, a number of organizations—notably, the Luxembourg Income Study (LIS), the Organisation of Economic Cooperation and Development (OECD), and the Congressional Budget Office (CBO)—assume household income is equally shared within each household, but that household “needs” increase with the square root of the number of people in the household.  That is, a household containing 9 members is assumed to require 1½ times as much income to enjoy the same standard of living as a family containing 4 members.  After an adjustment is made to account for the impact of household size, these organizations then calculate inequality among persons rather than among households.

How are these alternative income definitions estimated?  Who uses them?  What do the estimates show?  I’ll only consider a two or three basic cases.

First, pretax, private, cash income. By far the most famous users of this definition are Professors Thomas Piketty and Emmanuel Saez.  Their most celebrated product is an annual estimate of the share of total U.S. income (under this restricted definition) that is received by the top 1 percent of tax filing units.

Here is their most famous chart, showing the income share of the top 1 percent going back to 1913. (I use the Piketty-Saez estimates that exclude realized capital gains in the calculation of taxpayers’ incomes.) The notable feature of the chart is the huge rise in the top income share between 1970—when it was 8 percent of all pretax private cash income—and last year—when the comparable share was 18 percent.  

I have circled one part of the line—between 1986 and 1988—to show you how sensitive their income definition is to changes in the income tax code.  In 1986 Congress passed the Tax Reform Act of 1986 (TRA86). By 1988 the reform was fully implemented.  Wealthy taxpayers noticed that TRA86 sharply reduced the payoff to holding corporate earnings inside a separately taxed corporate entity. Rich business owners or shareholders could increase their after-tax income by arranging things so their business income was taxed only once, at the individual level.  The result was that a lot of income, once earned by and held within corporations, was now passed through to the tax returns of rich individual taxpayers. These taxpayers appeared to enjoy a sudden surge in their taxable incomes between 1986 and 1988.  No one seriously believes rich people failed to get the benefits of this income before 1987.  Before 1987 the same income simply showed up on corporate rather than on individual income tax returns.

A final point:  The chart displayed in SLIDE #6 is the source of the widely believed claim that U.S. inequality is nowadays about the same as it was at the end of the Roaring 1920s, before the Great Depression.  That is close to being true – under this income definition.

Census “money income”: This income definition is very similar to the one just discussed, except that it includes cash government transfer payments.  The producer of the series is the Census Bureau, and its most famous uses are to measure trends in real median household income and the official U.S. poverty rate. Furthermore, the Census Bureau uses the income definition to compile estimates of the Gini coefficient of household income inequality and the income shares received by each one-fifth of households, ranked from lowest to highest income, and received by the top 5 percent of households.

Here is a famous graph based on the Bureau’s “median household income” series.  I have normalized the historical series using the 1999 real median income level (1999 and 2000 were the peak income years according to Census data).  Since 1999 and 2000, median income has fallen about 10 percent.  If we accept this estimate without qualification, it certainly represents bad news for living standards of the nation’s middle class. The conclusion is contradicted by other government income statistics that use a broader, more inclusive income definition, however.

And here is the Bureau’s most widely cited distributional statistic (after its “official poverty rate” estimate).  Since 1979, the Gini coefficient has increased 17 percent under this income definition. (It is worth noting, however, that the portion of the increase that occurred between 1992 and 1993 is mainly the result of methodological changes in the way the Census Bureau ascertained incomes in its 1994 income survey.)

When you hear U.S. inequality compared with that in other rich countries, the numbers are most likely based on calculations of the LIS or OECD.  Their income definition is basically “Cash and Near-cash Public and Private income minus Income and Payroll taxes owed by households.”  Under this income definition, the U.S. looks relatively very unequal and America appears to have an exceptionally high poverty rate.  U.S. inequality has been rising under this income definition, as indeed has also been the case in most other rich countries. The increase in the United States has been above average, however, helping us to retain our leadership position, both in income inequality and in relative poverty.

We turn last to the most expansive income definition:  CBO’s measure of net after-tax income.  I will use CBO’s tabulations using this income definition to shed light on some of the inequality and living standard trends implied by the narrower income definitions discussed above.

Let’s consider some potential limitations of a couple of those definitions.  The limitations do not necessarily make them flawed or uninteresting.  They do mean the narrower income measures cannot tell us some of the things that users claim they tell us.

An obvious shortcoming of the “cash pretax private income” definition is that it excludes virtually everything the government does to equalize Americans’ incomes.  Believe it or not, the Federal tax system is mildly progressive.  It claims a bigger percentage of the (declared) incomes of the rich than it does of middle-income families’ and especially the poor.  Any pretax income measure will miss that redistribution.

More seriously, it excludes all government transfer payments.  You may think the rich get a bigger percentage of their income from government handouts compared with middle class and poorer households.  That is simply wrong.  The rich get a lot less.  And the percentage of total personal income that Americans derive from government transfer payments has gone way up over the years.  In the Roaring 1920s, Americans received almost nothing in the form of government transfers. Less than 1 percent of Americans’ incomes were received as transfer payments.  By 1970—near the low point of inequality according to the Piketty-Saez measure—8.3 percent of Americans’ personal income was derived from government transfers.  Last year, the share was 17 percent. None of the increase in government transfers is reflected in Piketty and Saez’s estimates of the trend in inequality.  Inequality is nowadays lower than it was in the late 1920s, mainly because the government does more redistribution through taxes and transfers.

Both the Piketty-Saez and the Census “money income” statistics are affected by the exclusion of government- and employer-provided health benefits from the income definition. This slide contains numbers, starting in 1960, that show the share of total U.S. personal consumption consisting of personal health care consumption.  I have divided the total into two parts. The first is the share that is paid for out of our own cash incomes (the blue part at the bottom).  This includes our out-of-pocket spending for doctors’ charges, hospital fees, pharmaceutical purchases, and other provider charges as well as our out-of-pocket spending on health insurance premiums. The second is the share of our personal health consumption that is paid out of government subsidies to Medicare, Medicaid, CHIP, etc., or out of employer subsidies to employee health plans (the red part). 

As everyone knows, the share of total consumption that consists of health consumption has gone way up.  What few people recognize is that the share that is directly paid by consumers—through payments to doctors, hospitals, and household health insurance premium payments—has remained unchanged.  All of the increase in the health consumption share since 1960 has been financed through government and employer subsidies to health insurance plans. None of those government or employer contributions is counted as “income” under the Piketty-Saez and Census “money income” definitions.  You would have to be quite a cynic to claim the subsidies have brought households no living standard improvements since 1960, yet that is how they are counted under the Piketty-Saez and Census “money income” definitions.

Final slide: How much has inequality gone up under income definitions that count all income sources and subtract the Federal income, payroll, corporation, and excise taxes we pay?  CBO gives us the numbers, though unfortunately its numbers end in 2011.

Here are CBO’s estimates of real income gains between 1979 and 2011.  These numbers show that real net incomes increased in every income category, from the very bottom to the very top.  They also show that real incomes per person have increased much faster at the top—over on the right—than in the middle or at the bottom—over on the left.  Still, contrary to a common complaint that all the income gains in recent years have been received by folks at the top, the CBO numbers suggest net income gains have been nontrivial among the poor and middle class as well as among top income recipients.

Suppose we look at trends in the more recent past, say, between 2000 and 2011.  That lower panel in this slide presents a very different picture from the one implied by the Census Bureau’s “money income” statistics.  Unlike the “money income numbers” [SLIDE #9], these show that inequality has declined since 2000.  Unlike the “money income numbers” [SLIDE #8], these show that incomes of middle-income families have improved since 2000.  There are a variety of explanations for the marked contrast between the Census Bureau and CBO numbers.  But a big one is the differing income definitions the two conclusions are based on.  The more inclusive measure of income shows faster real income gains among middle-income and poorer households, and it suggests a somewhat different trend in inequality.

Part II of the 2015 Brown Center Report on American Education

Over the next several years, policy analysts will evaluate the impact of the Common Core State Standards (CCSS) on U.S. education.  The task promises to be challenging.  The question most analysts will focus on is whether the CCSS is good or bad policy.  This section of the Brown Center Report (BCR) tackles a set of seemingly innocuous questions compared to the hot-button question of whether Common Core is wise or foolish.  The questions all have to do with when Common Core actually started, or more precisely, when the Common Core started having an effect on student learning.  And if it hasn’t yet had an effect, how will we know that CCSS has started to influence student achievement? 

The analysis below probes this issue empirically, hopefully persuading readers that deciding when a policy begins is elemental to evaluating its effects.  The question of a policy’s starting point is not always easy to answer.  Yet the answer has consequences.  You can’t figure out whether a policy worked or not unless you know when it began.^[i] 

The analysis uses surveys of state implementation to model different CCSS starting points for states and produces a second early report card on how CCSS is doing.  The first report card, focusing on math, was presented in last year’s BCR.  The current study updates state implementation ratings that were presented in that report and extends the analysis to achievement in reading.  The goal is not only to estimate CCSS’s early impact, but also to lay out a fair approach for establishing when the Common Core’s impact began—and to do it now before data are generated that either critics or supporters can use to bolster their arguments.  The experience of No Child Left Behind (NCLB) illustrates this necessity.

After the 2008 National Assessment of Educational Progress (NAEP) scores were released, former Secretary of Education Margaret Spellings claimed that the new scores showed “we are on the right track.”^[ii] She pointed out that NAEP gains in the previous decade, 1999-2009, were much larger than in prior decades.  Mark Schneider of the American Institutes of Research (and a former Commissioner of the National Center for Education Statistics [NCES]) reached a different conclusion. He compared NAEP gains from 1996-2003 to 2003-2009 and declared NCLB’s impact disappointing.  “The pre-NCLB gains were greater than the post-NCLB gains.”^[iii]  It is important to highlight that Schneider used the 2003 NAEP scores as the starting point for assessing NCLB.  A report from FairTest on the tenth anniversary of NCLB used the same demarcation for pre- and post-NCLB time frames.^[iv]  FairTest is an advocacy group critical of high stakes testing—and harshly critical of NCLB—but if the 2003 starting point for NAEP is accepted, its conclusion is indisputable, “NAEP score improvement slowed or stopped in both reading and math after NCLB was implemented.” 

Choosing 2003 as NCLB’s starting date is intuitively appealing.  The law was introduced, debated, and passed by Congress in 2001.  President Bush signed NCLB into law on January 8, 2002.  It takes time to implement any law.  The 2003 NAEP is arguably the first chance that the assessment had to register NCLB’s effects. 

Selecting 2003 is consequential, however.  Some of the largest gains in NAEP’s history were registered between 2000 and 2003.  Once 2003 is established as a starting point (or baseline), pre-2003 gains become “pre-NCLB.”  But what if the 2003 NAEP scores were influenced by NCLB? Experiments evaluating the effects of new drugs collect baseline data from subjects before treatment, not after the treatment has begun.   Similarly, evaluating the effects of public policies require that baseline data are not influenced by the policies under evaluation.   

Avoiding such problems is particularly difficult when state or local policies are adopted nationally.  The federal effort to establish a speed limit of 55 miles per hour in the 1970s is a good example.  Several states already had speed limits of 55 mph or lower prior to the federal law’s enactment.  Moreover, a few states lowered speed limits in anticipation of the federal limit while the bill was debated in Congress.  On the day President Nixon signed the bill into law—January 2, 1974—the Associated Press reported that only 29 states would be required to lower speed limits.  Evaluating the effects of the 1974 law with national data but neglecting to adjust for what states were already doing would obviously yield tainted baseline data.

There are comparable reasons for questioning 2003 as a good baseline for evaluating NCLB’s effects.  The key components of NCLB’s accountability provisions—testing students, publicizing the results, and holding schools accountable for results—were already in place in nearly half the states.  In some states they had been in place for several years.  The 1999 iteration of Quality Counts, Education Week’s annual report on state-level efforts to improve public education, entitled Rewarding Results, Punishing Failure, was devoted to state accountability systems and the assessments underpinning them. Testing and accountability are especially important because they have drawn fire from critics of NCLB, a law that wasn’t passed until years later.

The Congressional debate of NCLB legislation took all of 2001, allowing states to pass anticipatory policies.  Derek Neal and Diane Whitmore Schanzenbach reported that “with the passage of NCLB lurking on the horizon,” Illinois placed hundreds of schools on a watch list and declared that future state testing would be high stakes.^[v] In the summer and fall of 2002, with NCLB now the law of the land, state after state released lists of schools falling short of NCLB’s requirements.  Then the 2002-2003 school year began, during which the 2003 NAEP was administered.  Using 2003 as a NAEP baseline assumes that none of these activities—previous accountability systems, public lists of schools in need of improvement, anticipatory policy shifts—influenced achievement.  That is unlikely.^[vi]

Unlike NCLB, there was no “pre-CCSS” state version of Common Core.  States vary in how quickly and aggressively they have implemented CCSS.  For the BCR analyses, two indexes were constructed to model CCSS implementation.  They are based on surveys of state education agencies and named for the two years that the surveys were conducted.  The 2011 survey reported the number of programs (e.g., professional development, new materials) on which states reported spending federal funds to implement CCSS.  Strong implementers spent money on more activities.  The 2011 index was used to investigate eighth grade math achievement in the 2014 BCR.  A new implementation index was created for this year’s study of reading achievement.  The 2013 index is based on a survey asking states when they planned to complete full implementation of CCSS in classrooms.  Strong states aimed for full implementation by 2012-2013 or earlier.      

Fourth grade NAEP reading scores serve as the achievement measure.  Why fourth grade and not eighth?  Reading instruction is a key activity of elementary classrooms but by eighth grade has all but disappeared.  What remains of “reading” as an independent subject, which has typically morphed into the study of literature, is subsumed under the English-Language Arts curriculum, a catchall term that also includes writing, vocabulary, listening, and public speaking.  Most students in fourth grade are in self-contained classes; they receive instruction in all subjects from one teacher.  The impact of CCSS on reading instruction—the recommendation that non-fiction take a larger role in reading materials is a good example—will be concentrated in the activities of a single teacher in elementary schools. The burden for meeting CCSS’s press for non-fiction, on the other hand, is expected to be shared by all middle and high school teachers.^[vii] 

Table 2-1 displays NAEP gains using the 2011 implementation index.  The four year period between 2009 and 2013 is broken down into two parts: 2009-2011 and 2011-2013.  Nineteen states are categorized as “strong” implementers of CCSS on the 2011 index, and from 2009-2013, they outscored the four states that did not adopt CCSS by a little more than one scale score point (0.87 vs. -0.24 for a 1.11 difference).  The non-adopters are the logical control group for CCSS, but with only four states in that category—Alaska, Nebraska, Texas, and Virginia—it is sensitive to big changes in one or two states.  Alaska and Texas both experienced a decline in fourth grade reading scores from 2009-2013.

The 1.11 point advantage in reading gains for strong CCSS implementers is similar to the 1.27 point advantage reported last year for eighth grade math.  Both are small.  The reading difference in favor of CCSS is equal to approximately 0.03 standard deviations of the 2009 baseline reading score.  Also note that the differences were greater in 2009-2011 than in 2011-2013 and that the “medium” implementers performed as well as or better than the strong implementers over the entire four year period (gain of 0.99).

Table 2-2 displays calculations using the 2013 implementation index.  Twelve states are rated as strong CCSS implementers, seven fewer than on the 2011 index.^[viii]  Data for the non-adopters are the same as in the previous table.  In 2009-2013, the strong implementers gained 1.27 NAEP points compared to -0.24 among the non-adopters, a difference of 1.51 points.  The thirty-four states rated as medium implementers gained 0.82.  The strong implementers on this index are states that reported full implementation of CCSS-ELA by 2013.  Their larger gain in 2011-2013 (1.08 points) distinguishes them from the strong implementers in the previous table.  The overall advantage of 1.51 points over non-adopters represents about 0.04 standard deviations of the 2009 NAEP reading score, not a difference with real world significance.  Taken together, the 2011 and 2013 indexes estimate that NAEP reading gains from 2009-2013 were one to one and one-half scale score points larger in the strong CCSS implementation states compared to the states that did not adopt CCSS.

As noted above, the 2013 implementation index is based on when states scheduled full implementation of CCSS in classrooms.  Other than reading achievement, does the index seem to reflect changes in any other classroom variable believed to be related to CCSS implementation?  If the answer is “yes,” that would bolster confidence that the index is measuring changes related to CCSS implementation. 

Let’s examine the types of literature that students encounter during instruction.  Perhaps the most controversial recommendation in the CCSS-ELA standards is the call for teachers to shift the content of reading materials away from stories and other fictional forms of literature in favor of more non-fiction.  NAEP asks fourth grade teachers the extent to which they teach fiction and non-fiction over the course of the school year (see Figure 2-1). 

Historically, fiction dominates fourth grade reading instruction.  It still does.  The percentage of teachers reporting that they teach fiction to a “large extent” exceeded the percentage answering “large extent” for non-fiction by 23 points in 2009 and 25 points in 2011.  In 2013, the difference narrowed to only 15 percentage points, primarily because of non-fiction’s increased use.  Fiction still dominated in 2013, but not by as much as in 2009.

The differences reported in Table 2-3 are national indicators of fiction’s declining prominence in fourth grade reading instruction.  What about the states?  We know that they were involved to varying degrees with the implementation of Common Core from 2009-2013.  Is there evidence that fiction’s prominence was more likely to weaken in states most aggressively pursuing CCSS implementation? 

Table 2-3 displays the data tackling that question.  Fourth grade teachers in strong implementation states decisively favored the use of fiction over non-fiction in 2009 and 2011.  But the prominence of fiction in those states experienced a large decline in 2013 (-12.4 percentage points).  The decline for the entire four year period, 2009-2013, was larger in the strong implementation states (-10.8) than in the medium implementation (-7.5) or non-adoption states (-9.8).  

This section of the Brown Center Report analyzed NAEP data and two indexes of CCSS implementation, one based on data collected in 2011, the second from data collected in 2013.  NAEP scores for 2009-2013 were examined.  Fourth grade reading scores improved by 1.11 scale score points in states with strong implementation of CCSS compared to states that did not adopt CCSS.  A similar comparison in last year’s BCR found a 1.27 point difference on NAEP’s eighth grade math test, also in favor of states with strong implementation of CCSS.  These differences, although certainly encouraging to CCSS supporters, are quite small, amounting to (at most) 0.04 standard deviations (SD) on the NAEP scale.  A threshold of 0.20 SD—five times larger—is often invoked as the minimum size for a test score change to be regarded as noticeable.  The current study’s findings are also merely statistical associations and cannot be used to make causal claims.  Perhaps other factors are driving test score changes, unmeasured by NAEP or the other sources of data analyzed here. 

The analysis also found that fourth grade teachers in strong implementation states are more likely to be shifting reading instruction from fiction to non-fiction texts.  That trend should be monitored closely to see if it continues.  Other events to keep an eye on as the Common Core unfolds include the following:

1.  The 2015 NAEP scores, typically released in the late fall, will be important for the Common Core.  In most states, the first CCSS-aligned state tests will be given in the spring of 2015.  Based on the earlier experiences of Kentucky and New York, results are expected to be disappointing.  Common Core supporters can respond by explaining that assessments given for the first time often produce disappointing results.  They will also claim that the tests are more rigorous than previous state assessments.  But it will be difficult to explain stagnant or falling NAEP scores in an era when implementing CCSS commands so much attention.   

2.  Assessment will become an important implementation variable in 2015 and subsequent years.  For analysts, the strategy employed here, modeling different indicators based on information collected at different stages of implementation, should become even more useful.  Some states are planning to use Smarter Balanced Assessments, others are using the Partnership for Assessment of Readiness for College and Careers (PARCC), and still others are using their own homegrown tests.   To capture variation among the states on this important dimension of implementation, analysts will need to use indicators that are up-to-date.

3.  The politics of Common Core injects a dynamic element into implementation.  The status of implementation is constantly changing.  States may choose to suspend, to delay, or to abandon CCSS.  That will require analysts to regularly re-configure which states are considered “in” Common Core and which states are “out.”  To further complicate matters, states may be “in” some years and “out” in others.

A final word.  When the 2014 BCR was released, many CCSS supporters commented that it is too early to tell the effects of Common Core.  The point that states may need more time operating under CCSS to realize its full effects certainly has merit.  But that does not discount everything states have done so far—including professional development, purchasing new textbooks and other instructional materials, designing new assessments, buying and installing computer systems, and conducting hearings and public outreach—as part of implementing the standards.  Some states are in their fifth year of implementation.  It could be that states need more time, but innovations can also produce their biggest “pop” earlier in implementation rather than later.  Kentucky was one of the earliest states to adopt and implement CCSS.  That state’s NAEP fourth grade reading score declined in both 2009-2011 and 2011-2013.  The optimism of CCSS supporters is understandable, but a one and a half point NAEP gain might be as good as it gets for CCSS.

Downloads

• Download the report