In the past decade, a remarkable shift has occurred in the development landscape. Specifically, acknowledgment of the central role of the private sector in contributing to, even driving, economic growth and global development has grown rapidly. The data on financial flows are dramatic, indicating reversal of the relative roles of official development assistance and private financial flows. This shift is also reflected in the way development is framed and discussed, never more starkly than in the Addis Abba Action Agenda and the new set of Sustainable Development Goals (SDGs). The Millennium Development Goals (MDGs), which the SDGs follow, focused on official development assistance. In contrast, while the new set of global goals does not ignore the role of official development assistance, they reorient attention to the role of the business sector (and mobilizing host country resources).

The U.S. Agency for International Development (USAID) has been in the vanguard of donors in recognizing the important role of the private sector to development, most notably via the agency’s launch in 2001 of a program targeted on public-private partnerships (PPPs) and the estimated 1,600 USAID PPPs initiated since then. This paper provides a quantitative and qualitative presentation of USAID’s public-private partnerships and business sector participation in those PPPs. The analysis offered here is based on USAID’s PPP data set covering 2001-2014 and interviews with executives of 17 U.S. corporations that have engaged in PPPs with USAID.

The genesis of this paper is the considerable discussion by USAID and the international development community about USAID’s PPPs, but the dearth of information on what these partnerships entail. USAID’s 2014 release (updated in 2015) of a data set describing nearly 1,500 USAID PPPs since 2001 offers an opportunity to analyze the nature of those PPPs.

On a conceptual level, public-private partnerships are a win-win, even a win-win-win, as they often involve three types of organizations: a public agency, a for-profit business, and a nonprofit entity. PPPs use public resources to leverage private resources and expertise to advance a public purpose. In turn, non-public sectors—both businesses and nongovernmental organizations (NGOs)—use their funds and expertise to leverage government resources, clout, and experience to advance their own objectives, consistent with a PPP’s overall public purpose. The data from the USAID data set confirm this conceptual mutual reinforcement of public and private goals.

The goal is to utilize USAID’s recently released data set to draw conclusions on the nature of PPPs, the level of business sector engagement, and, utilizing interviews, to describe corporate perspectives on partnership with USAID.

The arguments regarding “why” PPPs are an important instrument of development are well established. This paper presents data on the “what”: what kinds of PPPs have been implemented and in what countries, sectors, and income contexts. There are other research and publications on the “how” of partnership construction and implementation. What remains missing are hard data and analysis, beyond the anecdotal, as to whether PPPs make a difference—in short, is the trouble of forming these sometimes complex alliances worth the impact that results from them?

The goal of this paper is not to provide commentary on impact since those data are not currently available on a broad scale. Similarly, this paper does not recommend replicable models or case studies (which can be found elsewhere), though these are important and can help new entrants to join and grow the field. Rather, the goal is to utilize USAID’s recently released data set to draw conclusions on the nature of PPPs, the level of business sector engagement, and, utilizing interviews, to describe corporate perspectives on partnership with USAID.

The decision to target this research on business sector partners’ engagement in PPPs—rather than on the civil society, foundation, or public partners—is based on several factors. First, USAID’s references to its PPPs tend to focus on the business sector partners, sometimes to the exclusion of other types of partners; we want to understand the role of the partners that USAID identifies as so important to PPP composition. Second, in recent years much has been written and discussed about corporate shared value, and we want to assess the extent to which shared value plays a role in USAID’s PPPs in practice.

The paper is divided into five sections. Section I is a consolidation of the principal data and findings of the research. Section II provides an in-depth “data picture” of USAID PPPs drawn from quantitative analysis of the USAID PPP data set and is primarily descriptive of PPPs to date. Section III moves beyond description and provides analysis of PPPs and business sector alignment. It contains the results of coding certain relevant fields in the data set to mine for information on the presence of business partners, commercial interests (i.e., shared value), and business sector partner expertise in PPPs. Section IV summarizes findings from a series of interviews of corporate executives on partnering with USAID. Section V presents recommendations for USAID’s partnership-making.

Downloads

• WP94PPPReport2016Web

Brookings today releases a report USAID’s Public-Private Partnerships: A Data Picture and Review of Business Engagement, which will be the subject of a public discussion on March 8 featuring a panel of Jane Nelson (Harvard University), Ann Mei Chang (U.S. Agency for International Development (USAID)), Johanna Nesseth Tuttle (Chevron Corp.), and Sarah Thorn (Wal-Mart Stores Inc.).

The report is based on USAID’s database of 1,481 public-private partnerships (PPPs) from 2001 to 2014 and a series of corporate interviews.

The value of those partnerships totals $16.5 billion, two-thirds from non-U.S. government sources – private companies, nongovernmental organizations (NGOs), foundations, and non-U.S. public institutions. Over 4000 organizations have served as resource partners in these PPPs.  Fifty-three percent are business entities, 32 percent are from the non-profit world, and 25 percent are public institutions. Eighty-five organizations have participated in five or more PPPs, led by Microsoft (62), Coca Cola (36), and Chevron (33).

The partnerships are relatively evenly distributed among three major regions—Africa, Latin American/Caribbean, and Asia—but 36 percent of the value of all PPPs is from partnerships that are global in reach.

In analyzing the data, the researchers found that 77 percent of PPPs included one or more business partner, and that 83 percent of these partnerships are connected to a business partner’s commercial interest (either shared value or more indirect strategic interest). In almost 80 percent of those PPPs, the business partner contributes some form of corporate expertise to the partnership.

The purpose of the March 8 panel discussion is to examine the report but also to go beyond by addressing outstanding questions like: how should the impact of public-private partnerships be identified, measured, and evaluated? Is shared value the Holy Grail linking corporate interest to public goods and achieving sustainable results? Where do public-private partnerships fit in USAID’s strategy for engaging the private sector in development, particularly in light of the emphasis on the role of business in advancing the new set of Sustainable Development Goals?

We hope you can join us for what should prove to be an engaging discussion.

Since 2000, China has emerged as Africa’s largest trading partner and a major source of investment finance as well. Large numbers of Chinese workers and entrepreneurs have moved to Africa in recent years, with estimates running as high as one million. China’s engagement with Africa has no doubt led to faster growth and poverty reduction on the continent. It is also relatively popular: In attitude surveys, 70 percent of African respondents have a positive view of China, higher than percentages in Asia, the Americas, or Europe.

While China’s deepening engagement with Africa has largely been associated with better economic performance, its involvement is not without controversy. This is particularly true in the West, as typical headlines portray an exploitive relationship: “Into Africa: China’s Wild Rush,” “China in Africa: Investment or Exploitation?,” and “Clinton warns against ‘new colonialism’ in Africa.” 

My forthcoming study, "China’s Engagement with Africa: From Natural Resources to Human Resources," aims to objectively assess this important new development in the world. It has six main findings:

1. First, on the scale of China’s activities in Africa: The media often portrays China’s involvement as enormous, potentially overwhelming the continent. According to data from China’s Ministry of Commerce (MOFCOM), the stock of Chinese direct investment in Africa was $32 billion at the end of 2014. This represents less than 5 percent of the total stock of foreign investment on the continent. Stocks naturally change slowly. But the "World Investment Report 2015" similarly finds that China’s share of inward direct investment flows to Africa during 2013 and 2014 was only 4.4 percent of the total. Of course, direct investment is not the only form of foreign financing. The Export-Import Bank of China and China Development Bank have also made large loans in Africa, mostly to fund infrastructure projects. In recent years, China has provided about one-sixth of the external infrastructure financing for Africa. In short, Chinese financing is substantial enough to contribute meaningfully to African investment and growth, but the notion that China has provided an overwhelming amount of finance and is buying up the whole continent is inaccurate.

2. The second main finding from the study concerns China’s direct investment and governance. China has drawn attention by making large resource-related investments in countries with poor governance indicators, such as the Democratic Republic of Congo, Angola, and Sudan. These deals are certainly part of the picture when it comes to China’s engagement with Africa. But the more general relationship between Chinese direct investment and recipients’ governance environments is different. After controlling for market size and natural resource wealth, total foreign direct investment is highly correlated with measures of property rights and rule of law, as one might expect. This is true both globally and within the African continent. China’s outward direct investment, on the other hand, is uncorrelated with measures of property rights and the rule of law after controlling for market size and natural resource wealth. In this sense, Chinese investment is indifferent to the governance environment in a particular country. While China has investments in the Democratic Republic of Congo, Angola, and Sudan, those are balanced by investments in African countries that have relatively good governance environments. South Africa, for instance, is the foremost recipient of Chinese investment. Furthermore, some of the big resource deals in poor governance environments are not working out well, so Chinese state enterprises may well rethink their approach in the future.

3. A third main finding emerges from examining MOFCOM’s registry of Chinese firms investing in Africa. In the aggregate data on Chinese investment in different countries, the big state enterprise deals naturally play an outsized role. MOFCOM’s database on Chinese firms investing in Africa, on the other hand, provides a snapshot of what small- and medium-sized Chinese firms—most of which are private—are doing in Africa. Unlike the big state-owned enterprise investments, these firms are not focused on natural resource extraction. The largest area for investment is service sectors, with significant investment in manufacturing as well. Many African economies welcome this Chinese investment in manufacturing and services.

4. The fourth finding relates to infrastructure finance. In recent years infrastructure financing has expanded and helped many African countries begin to rectify infrastructure deficiencies. Africa has been receiving about $30 billion per year in external finance for infrastructure, of which China provides one-sixth. Chinese financing is a useful complement to other sources, particularly as traditional finance from multilateral development banks and bilateral donors is concentrated on water supply and sanitation. Likewise, private participation in infrastructure is primarily aimed at telecommunications. China has filled a niche by focusing on transportation and power.

   Chinese financing of infrastructure has also enabled Chinese construction companies to gain a firm foothold on the continent. Evidence suggests that Chinese companies have become highly competitive, crowding out African construction companies. This is an area where a trade-off seems to exist between, on the one hand, getting projects completed quickly and cheaply and, on the other, facilitating the long-term development of a local construction industry.

5. This point leads to the fifth finding of the study. There are a lot of Chinese workers in Africa; the total is disproportionately high when compared to the amount of financing that China has provided and compared to migrants from other continents. This is a tentative conclusion because the data on this issue are particular weak. But estimates of Chinese migrants in Africa exceed one million. Many migrants initially move to Africa as workers on Chinese projects in infrastructure and mining and then, perceiving good economic opportunities, stay on. Similar to the dilemma confronting the continent’s construction industry, African countries face a tradeoff here: Chinese workers bring skills and entrepreneurship, but their large numbers limit African workers’ opportunities for jobs and training. The popular notion that Chinese companies only employ Chinese workers is not accurate, but the overall number of Chinese workers in Africa is large. Africa may want to take a page from China’s playbook and limit the ability of foreign investors to bring in workers, forcing them to train local labor instead.

The popular notion that Chinese companies only employ Chinese workers is not accurate, but the overall number of Chinese workers in Africa is large.

6. A final important finding of the study is that the foundation for the Africa-China economic relationship is shifting. China’s involvement in Africa stretches back decades, but the economic relationship accelerated after 2000, when China’s growth model became especially resource-intensive. It made sense for resource-poor China to import natural resources from Africa and to export manufactures in return.

These patterns of trade and investment are now likely to gradually shift in response to changing demographics. The working-age population in China has peaked and will shrink over the coming decades. This has contributed to a tightening of the labor market and an increase in wages, which benefits Chinese people. Household income and consumption are also rising. Compared to past trends, China’s changing pattern of growth is less resource-intensive, so China’s needs for energy and minerals are relatively muted. At the same time, China is likely to be a steady supplier of foreign investment to other countries, and part of that will involve moving manufacturing value chains to lower-wage locations.

Africa’s demographics are moving in the opposite direction. In fact, they resemble China’s at the beginning of its economic reform 35 years ago. About half of Africa’s population is below the age of 20, which means the working-age population will surge over the next 20 years, and will probably continue growing until the middle of the century or later. Roughly speaking, Africa needs to create about 20 million jobs per year to employ its expanding workforce. Twenty years from now, it will need to create 30 million jobs per year. Africa’s demographics present both an opportunity and a challenge. It is unrealistic to expect the China-Africa economic relationship to change overnight. Nor would it be reasonable to expect large volumes of Chinese manufacturing to move to the continent in the near future; it would be more natural for value chains to migrate from China to nearby locations such as Vietnam and Bangladesh. But if even small amounts of manufacturing shift, this could make a significant difference for African economies, which are starting out with an extremely low base of industrialization. And it is useful to have a long-term vision that an economic relationship that started out very much centered on natural resources should shift over time to a greater focus on human resources.

For more on China’s engagement in Africa, check out the Brookings event hosted by the John L. Thornton China Center and the Africa Growth Initiative this Wednesday, July 13, at 3:30pm. 

Throughout the 2000s, Chinese demand for primary goods like oil, iron, copper, and zinc helped Africa reduce poverty more than it had in decades. Even so, China’s total investment in the continent’s natural resources has been smaller than many imagine, and, with growth moving away from manufacturing and toward consumption, China’s appetite for raw materials will continue to diminish. China’s shifting economic growth model aligns with Sub-Saharan Africa’s imminent labor force boom, presenting a significant opportunity for both sides. Maximizing mutual gain will depend on China and Africa cooperating to address a host of challenges: Can African countries limit the flow of Chinese migrants and foster domestic industries? Will Chinese investors adopt global norms of social and environmental responsibility? Where does the West fit in?

This study aims to objectively assess China’s economic engagement on the African continent, the extent to which African economies are benefiting, prospects for the future, and ways to make this relationship more productive. David Dollar marshals evidence about the scale of trade, investment, infrastructure cooperation, and migration between China and Africa, all of which are relatively recent phenomena. In addition, Dollar addresses the question of whether and how China’s involvement differs from that of Africa’s other economic partners. The concluding chapter provides some tentative recommendations for African countries, China, and the West.

Downloads

• China's engagement with Africa: From natural resources to human resources

Event Information

The world’s top economies had much to discuss at the G-20 summit in Brisbane, Australia last month, including reinvigorating global growth, the reduction of trade barriers, financial regulation reforms, and global infrastructure. The G-20 meeting took place at a key time for U.S. international economic policy, as it came on the heels of President Obama’s prior stops at the APEC summit and the ASEAN summit. As the U.S. joins its G-20 colleagues in aiming to boost G-20 GDP by an additional 2 percent by 2018, there remain many questions about how G-20 countries will follow through with the goals set in Brisbane.

On December 3, the Global Economy and Development program and the Economic Studies program at Brookings welcomed U.S. Treasury Undersecretary for International Affairs Nathan Sheets in his first public address since being confirmed in September. Following the recent G-20 meeting, Sheets discussed his perspectives on priorities for international economic policy in the years ahead across key areas including trade, the international financial architecture, and the United States’ evolving economic relationships.

Join the conversation on Twitter using #GlobalEconomy

Video

• U.S. Economic Engagement on the International Stage: A Conversation with U.S. Treasury Undersecretary Nathan Sheets

Audio

• U.S. Economic Engagement on the International Stage: A Conversation with U.S. Treasury Undersecretary Nathan Sheets

Transcript

• Uncorrected Transcript (.pdf)

Event Materials

• 20141203_sheets_international_economic_engagement_transcript

Harsh geopolitical realities and historic legacies have pushed the South Caucasus states of Armenia, Azerbaijan, and Georgia back onto the foreign policy agendas of the United States, the European Union (EU), and Turkey, at a time when all three have pulled back from more activist roles in regional affairs. The South Caucasus states have now become, at best, second-tier issues for the West, but they remain closely connected to first-tier problems. To head off the prospect that festering crises in the Caucasus will lead to or feed into broader conflagrations, the United States, EU, and Turkey have to muster sufficient political will to re-engage to some degree in high-level regional diplomacy. In “Retracing the Caucasian Circle Considerations and Constraints for U.S., EU, and Turkish Engagement in the South Caucasus,” authors Fiona Hill, Kemal Kirişci, and Andrew Moffatt explore the rationale and assess the options for Western reengagement with Armenia, Azerbaijan, and Georgia given the current challenges and limitations on all sides. Based on a series of study trips to the South Caucasus and Turkey in 2014 and 2015, and numerous other interviews, the authors review some of the current factors that should be considered by Western policymakers and analysts.

Constraints and considerations for U.S., EU, and Turkish engagement in the South Caucasus:

• Divergent trends in the South Caucasus
• Russia’s influence in the South Caucasus
• Regional conflicts
• The United States’ diminishing role in the South Caucasus
• Failure to integrate the South Caucasus into the EU
• Foundering relations with Turkey
• Dashed expectations in the South Caucasus of Western engagement

Despite the challenges that have beset the West’s relations with the South Caucasus and the growing disillusionment in Armenia, Azerbaijan, and Georgia, giving up on engagement is not an option.

Policy options for the future:

• The United States, EU, and Turkey must work together, rather than separately
• “Under the radar” coordination on creative interim solutions and working with other mediators
• Focus on the development of “soft regionalism”
• Work with Georgia as the hub for furthering soft regionalism
• Devise adaptable policies as relations with Iran and China develop in the region

Downloads

• Retracing the Caucasian Circle: Considerations and constraints for U.S., EU, and Turkish engagement in the South Caucasus

Event Information

President Barack Obama has established a broad policy of engagement as a central feature of his administration’s foreign policy agenda. From the earliest days of his presidency, the president has reached out to Iran, Russia and other nations around the world, marking not only a turning of the page but possibly a whole new chapter in U.S. foreign policy. While Europeans have advocated for increased bi-lateral and multi-lateral dialogue for some time, several important questions remain. With which nations or groups should the United States and Europe engage and should there be limits to dialogue in some cases? What are the consequences if dialogue fails? Do Europeans and Americans now have the same agenda and goals for engagement?

On May 29, the Center on the United States and Europe at Brookings (CUSE) will host experts and officials from both sides of the Atlantic for the 2009 CUSE Annual Conference to address these issues. Panelists will examine the prospect of engagement with Iran and Russia, and how to deal with groups such as Hamas and the Taliban. After each panel, participants will take audience questions.

Transcript

• Uncorrected Transcript (.pdf)

Event Materials

• 20090529_cuse

Editor's note: This article appears in the April 2015 issue of Global Forum. Click here to view the full publication.

Since passage of the Affordable Care Act in 2010, the last several years have seen a groundswell in physician payment and delivery reforms designed to achieve higher value health care through incentivizing higher quality care and lower overall costs. Accountable care models, for example, are achieving marked progress by realigning provider incentives toward greater risk-sharing and increased payments and shared savings with measured improvements in quality and cost containment. Medical homes are introducing greater care coordination and team-based care management, while the use of episode-based or bundled payments is removing perverse incentives that reward volume and intensity.

These reforms are coming just as the number of highly targeted, highly priced treatments continues to expand. The U.S. Food and Drug Administration (FDA) approved a decade-high 41 novel new drugs in 2014, many of them targeted therapies approved on the basis of increasingly sophisticated progress in genomics and the understanding of disease progression. In areas like oncology, such targeted treatments have grown as a percentage of global oncology market size from 11% in 2003 to 46% in 2013. New brand specialty drug spending in the U.S. is estimated to have been $7.5 billion in 2013, or 69% of total new drug spending. The growing prevalence of these drugs and their cost to the health system are setting the stage for significant flashpoints between industry, payers, and providers, seen most clearly in the debate over hepatitis C treatment costs that roiled stakeholder interactions for most of the past year. 

More of these targeted treatments are in the development pipeline, and a growing number of public policy efforts taking shape in 2015 are focused on accelerating their availability. The House of Representatives' 21st Century Cures Initiative, for example, has released a slew of legislative proposals aimed at promoting breakthrough innovation by increasing the efficiency of drug development and regulatory review. These efforts have significant downstream implications for the pace at which targeted and specialty therapies will become available, their associated costs, and the growing importance of demonstrating value in the postmarket setting.

As payers and providers continue their push toward increased value-based care, more innovative models for connecting such reforms to drug development are needed. Earlier collaboration with industry could enable more efficient identification of unmet need, opportunities to add value through drug development, and clearer input on the value proposition and evidentiary thresholds needed for coverage. Equally important will be unique public-private collaborations that invest in developing a better postmarket data infrastructure that can more effectively identify high value uses of new treatments and support achieving value through new payment reforms.

Stronger collaboration could also improve evidence development and the coverage determination process after a targeted  treatment has gained regulatory approval. Facilitated drug access programs like those proposed by the Medicare Administrative Contractor Palmetto GBA create access points for patients to receive targeted anti-cancer agents off-label while payers and industry gather important additional outcomes data in patient registries. More systematic and efficient use of policies like Medicare's Coverage with Evidence Development (CED), which allows for provisional coverage for promising technologies or treatments while evidence continues to be collected, could enable industry and payers to work together to learn about a medical product's performance in patient populations not typically represented in clinical studies. A CED-type model could be especially useful for certain specialty drugs: data collected as a condition of payment could help payers and providers develop evidence from actual practice to improve treatment algorithms, increase adherence, and improve outcomes. 

Finally, collaborations that support stronger postmarket data collection can also support novel drug payment models that further reward value. Bundled payments that include physician-administered drugs, for example, could encourage providers to increase quality while also incentivizing manufacturers to help promote evidence-based drug use and lower costs for uses that generate low value. Outcomes-based purchasing contracts that tie price paid to a medical product's performance could be another promising approach for high-expense treatment with clearly defined and feasibly measured outcomes.

Many of these ideas are not new, but as manufacturers, payers, providers, and patients move into an increasingly value-focused era of health care, it is clear that they must work together to find new ways to both promote development of promising new treatments while also making good on the promise of value-based health care reforms.

"At the end of the day, we need to remember that Daesh is more a product of the civil wars than it is a cause of them. And the way that we’re behaving is we’re treating it as the cause.  And the problem is that in places like Syria, in Iraq, potentially in Libya, we are mounting these military campaigns to destroy Daesh and we’re not doing anything about the underlying civil wars.  And the real danger there is—we have a brilliant military and they may very well succeed in destroying Daesh—but if we haven’t dealt with the underlying civil wars, we’ll have Son of Daesh a year later." – Ken Pollack

“Part of the problem is how we want the U.S. to be more engaged and more involved and what that requires in practice. We have to be honest about a different kind of American role in the Middle East. It means committing considerable economic and political resources to this region of the world that a lot of Americans are quite frankly sick of… There is this aspect of nation-building that is in part what we have to do in the Middle East, help these countries rebuild, but we can’t do that on the cheap. We can’t do that with this relatively hands off approach.” – Shadi Hamid

In this episode of “Intersections,” Kenneth Pollack, senior fellow in the Center for Middle East Policy and Shadi Hamid, senior fellow in the Project on U.S. Relations with the Islamic World and author of "Islamic Exceptionalism: How the Struggle over Islam is Reshaping the World," discuss the current state of upheaval in the Middle East, the Arab Spring, and the political durability of Islamist movements in the region. They also explain their ideas on how and why the United States should change its approach to the Middle East and areas of potential improvement for U.S. foreign policy in the region. 

Show Notes

Fight or flight: America’s choice in the Middle East

Security and public order

Islamists on Islamism today

Temptations of Power: Islamists & Illiberal Democracy in a New Middle East

Ending the Middle East’s civil wars

A Rage for Order: The Middle East in turmoil, from Tahrir Square to ISIS

Building a better Syrian opposition army: How and why

With thanks to audio engineer and producer Zack Kulzer, Mark Hoelscher, Carisa Nietsche, Sara Abdel-Rahim, Eric Abalahin, Fred Dews and Richard Fawal.

Subscribe to the Intersections on iTunes, and send feedback email to intersections@brookings.edu.

"At the end of the day, we need to remember that Daesh is more a product of the civil wars than it is a cause of them. And the way that we’re behaving is we’re treating it as the cause.  And the problem is that in places like Syria, in Iraq, potentially in Libya, we are mounting these military campaigns to destroy Daesh and we’re not doing anything about the underlying civil wars.  And the real danger there is—we have a brilliant military and they may very well succeed in destroying Daesh—but if we haven’t dealt with the underlying civil wars, we’ll have Son of Daesh a year later." – Ken Pollack

“Part of the problem is how we want the U.S. to be more engaged and more involved and what that requires in practice. We have to be honest about a different kind of American role in the Middle East. It means committing considerable economic and political resources to this region of the world that a lot of Americans are quite frankly sick of… There is this aspect of nation-building that is in part what we have to do in the Middle East, help these countries rebuild, but we can’t do that on the cheap. We can’t do that with this relatively hands off approach.” – Shadi Hamid

In this episode of “Intersections,” Kenneth Pollack, senior fellow in the Center for Middle East Policy and Shadi Hamid, senior fellow in the Project on U.S. Relations with the Islamic World and author of "Islamic Exceptionalism: How the Struggle over Islam is Reshaping the World," discuss the current state of upheaval in the Middle East, the Arab Spring, and the political durability of Islamist movements in the region. They also explain their ideas on how and why the United States should change its approach to the Middle East and areas of potential improvement for U.S. foreign policy in the region. 

Show Notes

Fight or flight: America’s choice in the Middle East

Security and public order

Islamists on Islamism today

Temptations of Power: Islamists & Illiberal Democracy in a New Middle East

Ending the Middle East’s civil wars

A Rage for Order: The Middle East in turmoil, from Tahrir Square to ISIS

Building a better Syrian opposition army: How and why

With thanks to audio engineer and producer Zack Kulzer, Mark Hoelscher, Carisa Nietsche, Sara Abdel-Rahim, Eric Abalahin, Fred Dews and Richard Fawal.

Subscribe to the Intersections on iTunes, and send feedback email to intersections@brookings.edu.

Part III of the 2015 Brown Center Report on American Education

Student engagement refers to the intensity with which students apply themselves to learning in school.  Traits such as motivation, enjoyment, and curiosity—characteristics that have interested researchers for a long time—have been joined recently by new terms such as, “grit,” which now approaches cliché status.  International assessments collect data from students on characteristics related to engagement.  This study looks at data from the Program for International Student Assessment (PISA), an international test given to fifteen-year-olds.  In the U.S., most PISA students are in the fall of their sophomore year.  The high school years are a time when many observers worry that students lose interest in school.

Compared to their peers around the world, how do U.S. students appear on measures of engagement?  Are national indicators of engagement related to achievement?  This analysis concludes that American students are about average in terms of engagement.  Data reveal that several countries noted for their superior ranking on PISA—e.g., Korea, Japan, Finland, Poland, and the Netherlands—score below the U.S. on measures of student engagement.  Thus, the relationship of achievement to student engagement is not clear cut, with some evidence pointing toward a weak positive relationship and other evidence indicating a modest negative relationship. 

Education studies differ in units of analysis.  Some studies report data on individuals, with each student serving as an observation.  Studies of new reading or math programs, for example, usually report an average gain score or effect size representing the impact of the program on the average student.  Others studies report aggregated data, in which test scores or other measurements are averaged to yield a group score. Test scores of schools, districts, states, or countries are constructed like that.  These scores represent the performance of groups, with each group serving as a single observation, but they are really just data from individuals that have been aggregated to the group level.

Aggregated units are particularly useful for policy analysts.  Analysts are interested in how Fairfax County or the state of Virginia or the United States is doing.  Governmental bodies govern those jurisdictions and policymakers craft policy for all of the citizens within the political jurisdiction—not for an individual.  

The analytical unit is especially important when investigating topics like student engagement and their relationships with achievement.  Those relationships are inherently individual, focusing on the interaction of psychological characteristics.  They are also prone to reverse causality, meaning that the direction of cause and effect cannot readily be determined.  Consider self-esteem and academic achievement.  Determining which one is cause and which is effect has been debated for decades.  Students who are good readers enjoy books, feel pretty good about their reading abilities, and spend more time reading than other kids.  The possibility of reverse causality is one reason that beginning statistics students learn an important rule:  correlation is not causation.

Starting with the first international assessments in the 1960s, a curious pattern has emerged. Data on students’ attitudes toward studying school subjects, when examined on a national level, often exhibit the opposite relationship with achievement than one would expect.  The 2006 Brown Center Report (BCR) investigated the phenomenon in a study of “the happiness factor” in learning.^[i]  Test scores of fourth graders in 25 countries and eighth graders in 46 countries were analyzed.  Students in countries with low math scores were more likely to report that they enjoyed math than students in high-scoring countries.  Correlation coefficients for the association of enjoyment and achievement were -0.67 at fourth grade and -0.75 at eighth grade. 

Confidence in math performance was also inversely related to achievement.  Correlation coefficients for national achievement and the percentage of students responding affirmatively to the statement, “I usually do well in mathematics,” were -0.58 among fourth graders and -0.64 among eighth graders.  Nations with the most confident math students tend to perform poorly on math tests; nations with the least confident students do quite well.   

That is odd.  What’s going on?  A comparison of Singapore and the U.S. helps unravel the puzzle.  The data in figure 3-1 are for eighth graders on the 2003 Trends in Mathematics and Science Study (TIMSS).  U.S. students were very confident—84% either agreed a lot or a little (39% + 45%) with the statement that they usually do well in mathematics.  In Singapore, the figure was 64% (46% + 18%).  With a score of 605, however, Singaporean students registered about one full standard deviation (80 points) higher on the TIMSS math test compared to the U.S. score of 504. 

When within-country data are examined, the relationship exists in the expected direction.  In Singapore, highly confident students score 642, approximately 100 points above the least-confident students (551).  In the U.S., the gap between the most- and least-confident students was also about 100 points—but at a much lower level on the TIMSS scale, at 541 and 448.  Note that the least-confident Singaporean eighth grader still outscores the most-confident American, 551 to 541.

The lesson is that the unit of analysis must be considered when examining data on students’ psychological characteristics and their relationship to achievement.  If presented with country-level associations, one should wonder what the within-country associations are.  And vice versa.  Let’s keep that caution in mind as we now turn to data on fifteen-year-olds’ intrinsic motivation and how nations scored on the 2012 PISA.

PISA’s index of intrinsic motivation to learn mathematics comprises responses to four items on the student questionnaire:  1) I enjoy reading about mathematics; 2) I look forward to my mathematics lessons; 3) I do mathematics because I enjoy it; and 4) I am interested in the things I learn in mathematics.  Figure 3-2 shows the percentage of students in OECD countries—thirty of the most economically developed nations in the world—responding that they agree or strongly agree with the statements.  A little less than one-third (30.6%) of students responded favorably to reading about math, 35.5% responded favorably to looking forward to math lessons, 38.2% reported doing math because they enjoy it, and 52.9% said they were interested in the things they learn in math.  A ballpark estimate, then, is that one-third to one-half of students respond affirmatively to the individual components of PISA’s intrinsic motivation index.

Table 3-1 presents national scores on the 2012 index of intrinsic motivation to learn mathematics.  The index is scaled with an average of 0.00 and a standard deviation of 1.00.  Student index scores are averaged to produce a national score.  The scores of 39 nations are reported—29 OECD countries and 10 partner countries.^[ii]  Indonesia appears to have the most intrinsically motivated students in the world (0.80), followed by Thailand (0.77), Mexico (0.67), and Tunisia (0.59).  It is striking that developing countries top the list.  Universal education at the elementary level is only a recent reality in these countries, and they are still struggling to deliver universally accessible high schools, especially in rural areas and especially to girls.  The students who sat for PISA may be an unusually motivated group.  They also may be deeply appreciative of having an opportunity that their parents never had.

The U.S. scores about average (0.08) on the index, statistically about the same as New Zealand, Australia, Ireland, and Canada.  The bottom of the table is extremely interesting.  Among the countries with the least intrinsically motivated kids are some PISA high flyers.  Austria has the least motivated students (-0.35), but that is not statistically significantly different from the score for the Netherlands (-0.33).  What’s surprising is that Korea (-0.20), Finland (-0.22), Japan (-0.23), and Belgium (-0.24) score at the bottom of the intrinsic motivation index even though they historically do quite well on the PISA math test.

Let’s now dig a little deeper into the intrinsic motivation index.  Two components of the index are how students respond to “I do mathematics because I enjoy it” and “I look forward to my mathematics lessons.”  These sentiments are directly related to schooling.  Whether students enjoy math or look forward to math lessons is surely influenced by factors such as teachers and curriculum.  Table 3-2 rank orders PISA countries by the percentage of students who “agree” or “strongly agree” with the questionnaire prompts.  The nations’ 2012 PISA math scores are also tabled.  Indonesia scores at the top of both rankings, with 78.3% enjoying math and 72.3% looking forward to studying the subject.  However, Indonesia’s PISA math score of 375 is more than one full standard deviation below the international mean of 494 (standard deviation of 92).  The tops of the tables are primarily dominated by low-performing countries, but not exclusively so.  Denmark is an average-performing nation that has high rankings on both sentiments.  Liechtenstein, Hong Kong-China, and Switzerland do well on the PISA math test and appear to have contented, positively-oriented students.

Several nations of interest are shaded.  The bar across the middle of the tables, encompassing Australia and Germany, demarcates the median of the two lists, with 19 countries above and 19 below that position.  The United States registers above the median on looking forward to math lessons (45.4%) and a bit below the median on enjoyment (36.6%).  A similar proportion of students in Poland—a country recently celebrated in popular media and in Amanda Ripley’s book, The Smartest Kids in the World,^[iii] for making great strides on PISA tests—enjoy math (36.1%), but only 21.3% of Polish kids look forward to their math lessons, very near the bottom of the list, anchored by Netherlands at 19.8%. 

Korea also appears in Ripley’s book.  It scores poorly on both items.  Only 30.7% of Korean students enjoy math, and less than that, 21.8%, look forward to studying the subject.  Korean education is depicted unflatteringly in Ripley’s book—as an academic pressure cooker lacking joy or purpose—so its standing here is not surprising.  But Finland is another matter.  It is portrayed as laid-back and student-centered, concerned with making students feel relaxed and engaged.  Yet, only 28.8% of Finnish students say that they study mathematics because they enjoy it (among the bottom four countries) and only 24.8% report that they look forward to math lessons (among the bottom seven countries).  Korea, the pressure cooker, and Finland, the laid-back paradise, look about the same on these dimensions.

Another country that is admired for its educational system, Japan, does not fare well on these measures.  Only 30.8% of students in Japan enjoy mathematics, despite the boisterous, enthusiastic classrooms that appear in Elizabeth Green’s recent book, Building a Better Teacher.^[iv]  Japan does better on the percentage of students looking forward to their math lessons (33.7%), but still places far below the U.S.  Green’s book describes classrooms with younger students, but even so, surveys of Japanese fourth and eighth graders’ attitudes toward studying mathematics report results similar to those presented here.  American students say that they enjoy their math classes and studying math more than students in Finland, Japan, and Korea.

It is clear from Table 3-2 that at the national level, enjoying math is not positively related to math achievement.  Nor is looking forward to one’s math lessons.  The correlation coefficients reported in the last row of the table quantify the magnitude of the inverse relationships.  The -0.58 and -0.57 coefficients indicate a moderately negative association, meaning, in plain English, that countries with students who enjoy math or look forward to math lessons tend to score below average on the PISA math test.  And high-scoring nations tend to register below average on these measures of student engagement.  Country-level associations, however, should be augmented with student-level associations that are calculated within each country.

The 2012 PISA volume on student engagement does not present within-country correlation coefficients on intrinsic motivation or its components.  But it does offer within-country correlations of math achievement with three other characteristics relevant to student engagement. Table 3-3 displays statistics for students’ responses to: 1) if they feel like they belong at school; 2) their attitudes toward school, an index composed of four factors;^[v] and 3) whether they had arrived late for school in the two weeks prior to the PISA test. These measures reflect an excellent mix of behaviors and dispositions.

The within-country correlations trend in the direction expected but they are small in magnitude.  Correlation coefficients for math performance and a sense of belonging at school range from -0.02 to 0.18, meaning that the country exhibiting the strongest relationship between achievement and a sense of belonging—Thailand, with a 0.18 correlation coefficient—isn’t registering a strong relationship at all.  The OECD average is 0.08, which is trivial.  The U.S. correlation coefficient, 0.07, is also trivial.  The relationship of achievement with attitudes toward school is slightly stronger (OECD average of 0.11), but is still weak.

Of the three characteristics, arriving late for school shows the strongest correlation, an unsurprising inverse relationship of -0.14 in OECD countries and -0.20 in the U.S.  Students who tend to be tardy also tend to score lower on math tests.  But, again, the magnitude is surprisingly small.  The coefficients are statistically significant because of large sample sizes, but in a real world “would I notice this if it were in my face?” sense, no, the correlation coefficients are suggesting not much of a relationship at all.    

The PISA report presents within-country effect sizes for the intrinsic motivation index, calculating the achievement gains associated with a one unit change in the index.  One of several interesting findings is that intrinsic motivation is more strongly associated with gains at the top of the achievement distribution, among students at the 90^th percentile in math scores, than at the bottom of the distribution, among students at the 10^th percentile. 

The report summarizes the within-country effect sizes with this statement: “On average across OECD countries, a change of one unit in the index of intrinsic motivation to learn mathematics translates into a 19 score-point difference in mathematics performance.”^[vi]  This sentence can be easily misinterpreted.  It means that within each of the participating countries students who differ by one unit on PISA’s 2012 intrinsic motivation index score about 19 points apart on the 2012 math test.  It does not mean that a country that gains one unit on the intrinsic motivation index can expect a 19 point score increase.^[vii]    

Let’s now see what that association looks like at the national level.

PISA first reported national scores on the index of intrinsic motivation to learn mathematics in 2003.  Are gains that countries made on the index associated with gains on PISA’s math test?  Table 3-4 presents a score card on the question, reporting the changes that occurred in thirty-nine nations—in both the index and math scores—from 2003 to 2012.  Seventeen nations made statistically significant gains on the index; fourteen nations had gains that were, in a statistical sense, indistinguishable from zero—labeled “no change” in the table; and eight nations experienced statistically significant declines in index scores. 

The U.S. scored 0.00 in 2003 and 0.08 in 2012, notching a gain of 0.08 on the index (statistically significant).  Its PISA math score declined from 483 to 481, a decline of 2 scale score points (not statistically significant).

Table 3-4 makes it clear that national changes on PISA’s intrinsic motivation index are not associated with changes in math achievement.  The countries registering gains on the index averaged a decline of 3.7 points on PISA’s math assessment.  The countries that remained about the same on the index had math scores that also remain essentially unchanged (-0.09) And the most striking finding: countries that declined on the index (average of -0.15) actually gained an average of 10.3 points on the PISA math scale.  Intrinsic motivation went down; math scores went up.  The correlation coefficient for the relationship over all, not shown in the table, is -0.30.

The analysis above investigated student engagement.  International data from the 2012 PISA were examined on several dimensions of student engagement, focusing on a measure that PISA has employed since 2003, the index of intrinsic motivation to learn mathematics.  The U.S. scored near the middle of the distribution on the 2012 index.  PISA analysts calculated that, on average, a one unit change in the index was associated with a 19 point gain on the PISA math test.  That is the average of within-country calculations, using student-level data that measure the association of intrinsic motivation with PISA score.  It represents an effect size of about 0.20—a positive effect, but one that is generally considered small in magnitude.^[viii]

The unit of analysis matters.  Between-country associations often differ from within-country associations.  The current study used a difference in difference approach that calculated the correlation coefficient for two variables at the national level: the change in intrinsic motivation index from 2003-2012 and change in PISA score for the same time period.  That analysis produced a correlation coefficient of -0.30, a negative relationship that is also generally considered small in magnitude.

Neither approach can justify causal claims nor address the possibility of reverse causality occurring—the possibility that high math achievement boosts intrinsic motivation to learn math, rather than, or even in addition to, high levels of motivation leading to greater learning.  Poor math achievement may cause intrinsic motivation to fall.  Taken together, the analyses lead to the conclusion that PISA provides, at best, weak evidence that raising student motivation is associated with achievement gains.  Boosting motivation may even produce declines in achievement.

Here’s the bottom line for what PISA data recommends to policymakers: Programs designed to boost student engagement—perhaps a worthy pursuit even if unrelated to achievement—should be evaluated for their effects in small scale experiments before being adopted broadly.  The international evidence does not justify wide-scale concern over current levels of student engagement in the U.S. or support the hypothesis that boosting student engagement would raise student performance nationally.

Let’s conclude by considering the advantages that national-level, difference in difference analyses provide that student-level analyses may overlook.

1. They depict policy interventions more accurately.  Policies are actions of a political unit affecting all of its members.  They do not simply affect the relationship of two characteristics within an individual’s psychology. Policymakers who ask the question, “What happens when a country boosts student engagement?” are asking about a country-level phenomenon.

2.  Direction of causality can run differently at the individual and group levels.  For example, we know that enjoying a school subject and achievement on tests of that subject are positively correlated at the individual level.  But they are not always correlated—and can in fact be negatively correlated—at the group level. 

3.  By using multiple years of panel data and calculating change over time, a difference in difference analysis controls for unobserved variable bias by “baking into the cake” those unobserved variables at the baseline.  The unobserved variables are assumed to remain stable over the time period of the analysis.  For the cultural factors that many analysts suspect influence between-nation test score differences, stability may be a safe assumption.  Difference in difference, then, would be superior to cross-sectional analyses in controlling for cultural influences that are omitted from other models.

4.  Testing artifacts from a cultural source can also be dampened.  Characteristics such as enjoyment are culturally defined, and the language employed to describe them is also culturally bounded.  Consider two of the questionnaire items examined above: whether kids “enjoy” math and how much they “look forward” to math lessons.  Cultural differences in responding to these prompts will be reflected in between-country averages at the baseline, and any subsequent changes will reflect fluctuations net of those initial differences.

Downloads

• Download the report