Introduction

Reaching education goals in the coming years will require sharp increases in funding and better delivery. Despite a global focus on improving access to education, nearly 60 million children in developing countries remain out of primary school and increased investments have not translated to better education quality or improved learning outcomes (UNESCO 2015a). Even with an increase in domestic public expenditure, UNESCO estimates that the financing gap for delivering good quality universal education from pre-school through junior secondary levels by 2030 in low-income countries will be $10.6 billion, on average, between 2015 and 2030—over four times the level currently provided by official donors ($2.3 billion) (UNESCO 2015b).

Closing acute financing and delivery gaps that prevent access to quality education will be a major challenge, requiring all hands on deck. Domestic governments and foreign donors will need to step up their game substantially, but fiscal and capacity constraints are likely to prevent them remedying resource deficits on their own in the short term. Non-state actors—mainly religious and charitable organizations, private (“foundation”) schools, and a small number of for-profit schools—are already partially filling the gaps, although the precise extent of their services and their impact is unknown.

Determining the appropriate role of non-state actors in education is a contentious topic among specialists. Disagreements have revolved around serious normative issues, including such basic questions as whether non-state provision is consistent with the principle of education as a human right, and serious empirical questions relating to quality and equity implications. This discussion has been blurred by definitional issues (i.e., what is non-state and private education?); lack of clarity over distinctions between ownership, delivery, and financing; a lack of accurate data on current and potential provision rates; and an insufficient base of evidence from which to draw clear conclusions on the effectiveness of non-state engagement in education. These problems have made it difficult to generate comparisons across empirical studies, leading to significant variation in the interpretation of evidence. For some observers, evidence has fueled concern that non-state education is violating human rights principles (e.g., the report by the United Nations Rapporteur on Education),1 while for others it has provided encouragement that non-state engagement can help address financing and delivery challenges (e.g., Tooley 2009).

Our goal is to provide a neutral background to this debate and identify areas of common ground. Beginning with some big picture facts, this paper develops a detailed language around non-state actors in education. We then outline current issues and poles of debate around engagement of non-state actors in education and provide an assessment of the depth of available data and evidence. To close, we establish a typology and propose a framework for discussions around the role of non-state actors in basic education and how these actors can best contribute to the achievement of Education for All and the Sustainable Development Goals (SDGs). Our paper refers largely to basic education, including pre-primary, primary, and lower-secondary, as this is the main focus of much recent discussion around the role of non-state actors in education and an area of strong growth in developing countries.

Downloads

• Download the discussion paper

On July 5, the results from the first year of the world’s first development impact bond (DIB) for education in Rajasthan, India, were announced. The Center for Universal Education hosted a webinar in which three stakeholders in the DIB shared their perspective on the performance of the intervention, their learnings about the DIB process, and their thoughts for the future of DIBs and other results-based financing mechanisms.

Approximately 3 million girls ages 6 to 13 were out of school in India according to most recent data, 350,000 of which are in the state of Rajasthan. Child marriage is also a large issue in the state; no state-specific data exists, but nationwide 47 percent of girls ages 20 to 24 are married before age 18. According to Educate Girls, a non-governmental organization based in Rajasthan, girls’ exclusion is primarily a result of paternalistic societal mindsets and traditions. Given the evidence linking education and future life outcomes for girls, this data is greatly concerning.

The DIB finances a portion of the services provided by Educate Girls, which has been working to improve enrollment, retention, and learning outcomes for girls (and boys) in Rajasthan since 2007. The organization trains a team of community volunteers ages 18 to 30 to make door-to-door visits encouraging families to enroll their girls in school and to deliver curriculum enhancement in public school classrooms. Their volunteers are present in over 8,000 villages and 12,500 schools in Rajasthan. The DIB was launched in March of 2015 to finance services in 166 schools, which represents 5 percent of Educate Girls’ annual budget. The DIB is intended to be a “proof of concept” of the mechanism using this relatively small selection of beneficiaries.

The investor in the DIB is UBS Optimus Foundation, who has provided $238,000 in working capital to fund the service delivery. ID Insight, a non-profit evaluation firm, will evaluate the improvement in learning of girls and boys in the treatment schools in comparison to a control group and will validate the number of out of school girls enrolled. The Children’s Investment Fund Foundation serves as the outcome funder, and has agreed to pay UBS Optimus Foundation 43.16 Swiss francs ($44.37) for each unit of improved learning and 910.14 francs ($935.64) for every percentage point increase in the enrollment of girls out of school. Instiglio, a non-profit impact bond and results-based financing intermediary organization, provided technical assistance to all parties during the design of the DIB and currently provides performance management assistance to Educate Girls on behalf of UBS Optimus Foundation. 

The outcomes will be calculated in 2018, at the end of three years; however, preliminary results for the year since the launch of the DIB (representing multiple months of door-to-door visits and seven weeks of interventions in the classroom) were released last week. The payments for the DIB were structured such that the investor, UBS Optimus Foundation, would earn a 10 percent internal rate of return (IRR) on their investment at target outcome levels, which were based on Educate Girls’ past performance data. The table below presents the metrics, target outcome level, year-one result, and the progress toward the target. 

Table 1: Educate Girls DIB Results from first year of services

The DIB was challenging to implement and required DIB stakeholders to be resourceful.

First, the reliability of government data was a challenge, which necessitated flexibility in the identification of the target population and metrics. Second, given the number of stakeholders engaged and the novelty of this approach, the transaction costs were higher than they would have been for a traditional grant. This meant that strong and regular communication was crucial to the survival of the project.

The role of the outcome funder and investor were significantly different versus a grant.

The outcome funder spent more resources on defining outcomes, but spent fewer resources on managing grant activities. The investor utilized risk management and monitoring strategies informed by the activities in their commercial banking branch, which they have not used for other grants.

The DIB has changed the way the service provider operates.

In the video below, Safeena Husain from Educate Girls’ highlights the ways in which financing a portion of their program through a DIB differs from financing the program through grants. Safeena describes that in a grant, performance data is reported up to donors, but rarely makes it back down to frontline workers. The DIB has helped them to develop mobile dashboards that ensure performance data is reaching the front line and helping to identify barriers to outcomes as early as possible.

Based on the learnings from the implementation of the first DIB for education, this tool can be used to improve the value for money for the outcome funder and strengthen the performance management of a service provider. As the panelists discussed in the webinar, DIBs and other outcome-based financing mechanisms can help differentiate between organizations that are adept at fundraising and those that excel at delivering outcomes. However, service providers must be sufficiently prepared for rigorous outcome measurement if they plan to participate in a DIB; otherwise the high-stakes environment might backfire. In our research, we have closely examined the design constraints for impact bonds in the early childhood sector.

There are countless lessons to be learned from the stakeholder’s experience in the first DIB for education. We applaud the stakeholders for being transparent about the outcomes and true challenges associated with this mechanism. This transparency will be absolutely critical to ensure that DIBs are implemented and utilized appropriately moving forward.

2014 was a relatively friendless year for the Democratic Republic of Korea (DPRK or North Korea). It publicly lost its best friend and patron, China, to its erstwhile nemesis, the Republic of Korea (ROK or South Korea), when Presidents Park Geun-hye and Xi Jinping celebrated their growing friendship at the July summit in Seoul. Recently, retired PLA General Wang Hongguang wrote in the Chinese language site of Global Times, which is closely linked to the Chinese Communist Party, that China tired of cleaning up North Korea’s “mess” and would not step in to “save” North Korea if it collapses or starts a war.[1] And there is a vigorous debate in Beijing on whether the DPRK should be treated on a “normal” basis with China’s interests as the sole guide and purpose or be treated as a special case needing China’s indulgence and protection.[2] Since the Sony hack of November, North Korea has been under tighter scrutiny, both real and virtual, by Seoul, Beijing and Washington, accompanied by tighter sanctions in the new year. Bludgeoned by global condemnation of its atrocious human rights record, Pyongyang’s pariah status has intensified. Only Russia has been warming up to North Korea out of its own economic and political self-interest.

Is there any sizable country with good intentions for the region that is not giving up or beating up on North Korea? Is there any country Pyongyang likes and possibly even trusts? Mongolia stands out as the sole candidate, and it is friendly with both the East and the West.

Since the 2000s, Mongolia has played an increasingly constructive and steady role in in its bilateral ties with the DPRK and in its promotion of peace and cooperation in Northeast Asia. President Tsakhiagiin Elbegdorj, who visited Pyongyang in 2013, was the first head of state to reach out to the DPRK since Kim Jung Un assumed power and helped author the “Ulaanbaatar Dialogue on Northeast Asia Security,” which held its first meeting in June, 2014. It is a unique forum that combines official (track one) and unofficial academic/think tank/NGO (track two) participants, on a variety of important regional issues. The goals are to decrease distrust among nations and increase cooperation and peace. Both the DPRK and the ROK (Republic of Korea or South Korea) were represented at the inaugural meeting, as were the United States, China, Russia, Japan, and some European nations.

The UB Dialogue, as a consultative mechanism, has the potential to bring together policymakers, international organizations such as the United Nations Economic and Social Commission for Asia and the Pacific (ESCAP), and civil society entities and facilitate a range of initiatives related to economic cooperation; military transparency; environmental issues; non-traditional security threats; regional stability, cultural and educational exchange among the participants, including the two Koreas. These are official agenda items and goals of the UB Dialogue. With the Six-Party Talks nearly defunct and inter-Korean relations unable to address regional issues that affect the peninsula, Mongolia may be able to serve as a “Geneva or Helsinki of the East” as some observers have suggested.

Mongolia’s expanding global presence

Mongolia is uniquely positioned as the only country in Northeast Asia that enjoys good relations not only with North Korea but also South Korea, the United States, China, Russia, and Japan.

Mongolia’ relations with the United States, Canada, and Western Europe have steadily improved and deepened since the late 1980s. In recent decades, both Democratic and Republication administrations in Washington have enjoyed mutually warm and collaborative relations with Mongolia. President George W. Bush was the first sitting U.S. president to visit the country in 2005; he thanked the Mongolians for sending troops to join U.S.-led forces in Iraq and Afghanistan and for supporting anti-terrorism initiatives. Former Secretary of Defense Donald Rumsfeld also visited in the same year. In 2007, President Nambaryn Enkhbayar visited Washington to co-sign the Millennium Challenge Corporation Compact with President Bush. The next (and current) leader, President Elbegdorj, met U.S. President Barack Obama at the White House in 2011, as did the first civilian Minister of Defense, L. Bold. Vice President Joe Biden included Mongolia on a three-country Asia visit in August, 2011; China and Japan were the other two. A year later, Secretary of State Hilary Clinton took her turn in Ulaanbaatar. The most recent visit by top-level U.S. officials to Mongolia was by Secretary of Defense Chuck Hagel in April 2014.

Mongolia’s pursuit of the “third neighbor” policy allows the country to develop cooperative relations with the United States, Western Europe, ASEAN nations and others partly as “an air pocket” from its economic and security reliance on Beijing and Moscow. The softer side of this diplomatic push has been demonstrated by Ulaanbaatar’s membership in the Organization for Security and Cooperation in Europe and its previous chairmanship of the Community on Democracies.”[3]

Western experts on Mongolia applaud the way the country has developed a unique “peacekeeping niche” that facilitates participation in UN peacekeeping activities, international anti-terrorism measures, and humanitarian actions. For its small population of about three million, Mongolia takes on a heavy load of peacekeeping activities, ranking 26^th on the UN’s list of contributing nations.[4]

Since 2003, Mongolia annually hosts the “Khaan Quest” peacekeeping exercises for the purpose of tactical advancement and capacity building for its Mongolian Armed Forces (MAF) and for the improvement of regional confidence building. Although the United States and NATO play prominent roles, the Quest has attracted more diverse participants over the years so that by 2012, the number of interested parties expanded to include representatives from China and India as well as an array of developing nations such as Vietnam and Cambodia. These exercises are acknowledged as gatherings devoted to strengthening international cooperation and interoperability on peacekeeping initiatives around the world.[5]

On the economic side, Mongolia has been diversifying its external relations, with the maintenance of sovereignty and the related desire to reduce its overwhelming dependence on China as important goals. Expansion of economic relations is driven in part by a desire to participate in and benefit from global standards investment funds, and market access is a national priority. In that context, Mongolia’s relations with the West have been constructive and collaborative. For example, in 2013, the United States Trade Representative Michael Froman and Mongolia's Minister for Foreign Affairs, Luvsanvandan Bold, signed the Agreement on Transparency in Matters Related to International Trade and Investment between the United States of America and Mongolia. The Agreement commits the parties to provide opportunities for public comment on proposed laws and regulations and to publish final laws and regulations in Mongolian and English in order to facilitate access, openness, fairness, and procedural coherence in international trade and investment between Mongolia and other countries. “Additional commitments address the application of disciplines on bribery and corruption.” This type of administrative and legal modernization and the incorporation of measures to prevent and correct corruption are exemplary measures that could be helpful to the DPRK and other countries that are unfamiliar with or lagging in appropriate frameworks for doing business with diverse international actors.

Maintaining sovereignty between giants

China and Russia have vied for influence over Mongolia for many decades, from the time when Mongolia was in the Soviet sphere in influence to the present. Although 89 percent of foreign trade in 2013 was with China and Russia provides about 75 percent of Mongolia’s gasoline and diesel fuel and much of its electricity, Ulaanbaatar is assertively broadening and deepening its economic interests with the two big neighbors, especially greater transportation access and cheaper costs (vital to the landlocked nation), participation in the development of the New Silk Road corridor, and the construction of a Russian oil and gas pipeline through Mongolia that reaches China. All three countries have mutual interests and investments in developing Mongolia’s well-endowed mining industry.

But being sandwiched between two giants means Mongolia has to be prudent in preserving its sovereignty and independence, and Ulaanbaatar has done so in practical ways, balancing the two large powers’ interests with its own. The 2010 National Security Concept’s “One-Third Clause” sets a clear limit on the proportion of foreign direct investment from any one country: one-third. Legislation limits (foreign) state-owned companies from gaining control of strategic assets. And as numerous bilateral security and military cooperation agreements link Mongolia with China and Russia, UB has strategically and legally created elbow room for its autonomy. The government’s National Security and Foreign Policy Concepts outline a specific policy of not allowing foreign troops the use of its territory. Such preservationist measures to maintain sovereignty and independence in economic and security terms would be welcome examples to a North Korea which zealously prioritizes national sovereignty.

Mongolia and the Korean peninsula

Mongolia’s potential role as a non-nuclear peace broker in the region was further evidenced by its successful hosting of DPRK-Japan negotiations since 2012, which have yielded bilateral progress on longstanding abduction issues. In March 2014, Ulaanbaatar hosted the first-ever reunion between the parents of one of the abductees, Megumi Yokota (whom North Korea claims is dead), and her daughter, son-in-law, and their child who live in North Korea. Mongolia also served as a neutral venue for high-level talks on normalizing Japan-DPRK relations back in September 2007 as part of the Six-Party Talks framework. Asia Times reported that “arranging this recent meeting reflected Ulaanbaatar's ‘contribution to satisfy regional stability in Northeast Asia’ and how it could play a role in deepening understanding and normalizing DPRK-Japan relations.” President Elbegdorj's administration took particular care in staging the negotiations, including the use of the official state compound in Ikh Tenger as the meeting place. According to Alicia Campi, an American expert on Mongolia and the author of the AT article, Ikh Tenger was requested by the North Koreans.[6]

Mongolian President Elbegdorj is often described as an activist head of state, both for his focused efforts on developing Mongolia internally and advancing the country’s role and contributions internationally. One of his main foreign policy priorities is to promote regional economic integration and cooperation and peace and security. Dialogue and trust-building, two key components of his approach, coincide with ROK President Park Geun-hye’s emphasis on trustpolitik and the proposed Northeast Asia Peace and Cooperation Initiative (NAPCI). Both NAPCI and the UB Dialogue seek to chip away at distrust among Northeast Asian countries and increase collaboration and cooperation through multi-layered activities, including mutually reinforcing Track 1, 1.5 and 2 gatherings. Both emphasize multilateral cooperation on non-traditional security issues and people-to-people exchanges as ways to help build trust and resolve regional problems step by step. NAPCI held a track 1.5 forum in October 2014 in Seoul. In sharp contrast to its reaction to the first UB Dialogue of June that year, the DPRK flatly rejected the invitation to participate in the Seoul dialogue and criticized NAPCI as a cover for pressuring Pyongyang to relinquish its nuclear program and for reunification by absorption.[7]

There is no reason why the Ulaanbaatar Dialogue and NAPCI cannot be complementary and mutually reinforcing. Given that trust in inter-Korean relations is non-existent while Mongolia has gained deeper trust with both Koreas over the past two decades, NAPCI activities could benefit from Mongolia’s unique position in its relations with the DPRK. Ulaanbaatar potentially can serve as a neutral meeting ground, literally and metaphorically, for Pyongyang and Seoul. Moreover, given that the NAPCI seeks to maintain a cooperative relationship with other multilateral bodies and places emphasis on complementarity and inclusiveness, working with and supporting successful rounds of the UB Dialogues would be a principled move on the part of South Koreans. Moreover, engagement with North Korea through the UB Dialogue most likely represents an easier path to increasing inter-Korean trust than bilateral efforts and even easier than the NAPCI. South Korea’s domestic divisions and bitter left-right infighting tend to weaken the government’s position in approaches to the North. Seoul’s military standoff and competition with the North, its alliance with the United States, and participation in international sanctions regimes all cause suspicion in Pyongyang. In short, Seoul’s complex list of concerns and goals, some of which are contradictory to the spirit and practice of trust-building and cooperation with North Korea, create difficult conditions for progress through NAPCI alone.

In addition to lacking this baggage, Mongolia has unique standing with both North and South. It is a former Soviet satellite state that asserted full independence in 1990, and it is notable for successfully transitioning from a communist state to a vibrant democracy without civil war or bloodshed. President Elbegdorj’s 2013 speech in Pyongyang contained strong enunciation of the tenets of liberty. At the elite Kim Il Sung University in Pyongyang, he addressed students with these bold words: "No tyranny lasts forever. It is the desire of the people to live free that is the eternal power." And the Mongolian government has been keeping its border open to North Koreans who risk the arduous journey out of the DPRK and has permitted its airlines to transport them to South Korea.

Additionally, Mongolia has become a model of economic modernization and prosperous participation in the global economy. Although it faces some economic imbalances, its GDP rate was sky-high at 11.7 percent in 2013. There are good lessons to share with North Korea, and President Elberdorgj has made it clear that Mongolia would be very willing to work with the DPRK on economic development, IT, infrastructure, the management of mining precious earth resources and refineries. The two countries also engage in a worker exchange program, affording DPRK citizens the opportunity to breathe the air of freedom and to be exposed to South Korean television programming while they reside in Mongolia.

In recent years, Mongolia has pursued multiple types of people-to-people activities involving North Koreans, including academic exchanges, northeast Asian mayoral forums, and women’s parliamentary exchanges including female leaders from both Koreas. In June 2015, the second Track 2 conference of the UB Dialogue will convene in Ulaanbaatar with scholars from across the region and the United States with the theme of “Energy, Infrastructure, and Regional Connectivity.”

Sports and cultural initiatives in the past years have included international boxing matches in Ulaanbaatar with boxers from the DPRK, ROK, Mongolia, Russia and China. In 2013, Mongolia established an International Cooperation Fund which has supported children’s summer camps, basketball training and other exchanges with the DPRK in order to promote positive peace and people-to-people development in the region.

In the humanitarian arena, food aid to the DPRK has been channeled through international organizations, and the two countries have cooperated on physician exchanges. Prior research by Caprara and Ballen, conducted in cooperation with United Nations Special Envoy for Financing the Health Millennium Development Goals and for Malaria, has noted the additional soft power benefits of cooperative service development projects.

A recent global development forum hosted at the United Nations Asia-Pacific headquarters in Bangkok launched an Asia Pacific Peace Service Alliance which could build on these bilateral and regional exchanges in the critical area of humanitarian action and development in North Korea. An International Youth Leaders Assembly has been proposed in Ulaanbaatar for June, 2015, which would further the role of youth in fostering track two initiatives of service and dialogue.

Dr. Tsedendamba Batbayar, Mongolia’s Director of Policy Planning in the Ministry of Foreign Affairs and Trade, visited Washington in November, 2014 and noted the broad range of Mongolia-DPRK exchanges. Together with Mongolia Ambassador Bulgaa Altangerel, he emphasized his country’s desire to serve as a fair broker and mediator for the Northeast Asia region and to pursue prudent and practical measures to help build bridges of understanding between the people of North Korea and other parties.

But despite its uniquely constructive approach to dealing with the DPRK and other regional neighbors, Mongolia faces unique challenges in the mediator role it seeks to achieve. First, Ulaanbaatar has been able to gain Pyongyang’s trust because of the quiet diplomacy it has pursued, staying behind the scenes and out of the limelight. This has enabled a steady channel to the Pyongyang elite, and a focus on bilateral interests has been maintained. In short, drama has been avoided. But if Mongolia plays a more high-profile role with North Korea and multilateral actors, it will most likely be difficult to avoid some drama—posturing, rhetoric, and standoffs—emanating from various parties. Second, any increased or intensified involvement of China, Russia, and the United States in UB-led dialogue could come with the headache of big power arrogance and competition over leadership. The value of Mongolia’s role and activities for regional cooperation and peace stems from the fact that Ulaanbaatar does not assume airs or seek to dominate others. Whether China, Russia, and the United States would be able to refrain from seeking leadership and disproportionate influence in UB-led initiatives is highly questionable. Third, with respect to peninsular issues, for the UB Dialogues to gain more acceptance and credibility regionally and internationally requires that the DPRK become a consistent and collaborative presence at gatherings. Whether any nation or actor has the capacity to deliver consistent and collaborative participation by Pyongyang is an open question.

In addition, some observers believe that the impasse between North Korea and other nations is not simply the result of a trust deficit, but reflects mutually exclusive goals. While Mongolian mediation may not be able to solve the nuclear issue, it can be an effective channel – among others – for increasing communication, finding common ground, and beginning to ease tension.

Mongolia is the one Northeast Asian country that has kept its emotional cool and balanced policy interests with North Korea and other regional actors. It has not tripped over its own feet by politicizing historical grievances with its neighbors. Rather, it has exercised a calm can-do approach while its neighbors have engulfed themselves in hyper-nationalistic and ideological mire. And it has smartly used diplomacy and entrepreneurship to make friends and develop its own economy and people. These are significant assets that can be of benefit not only to UB but also to the region.

Recommendations

1. The Obama administration should actively support the Ulaanbaatar Dialogue process and encourage Seoul to find common cause in advancing greater regional dialogue and collaboration with the Mongolians through Track 2 and 1.5 processes. A precedent for this can be found in the case of Oman, which the current administration effectively tapped for back channel dialogue with Iran, kick-starting the present nuclear talks. Also, support by Washington would build on a prior exchange with Mongolia hosted by the Korea Institute for National Unification (KINU), where scholars noted potential benefits from three-way economic cooperation and the possibility of providing the North Koreans with a proven model of transformation from a closed statist system to a prosperous and more open system.

2. ROK President Park’s proposed regional cooperation mechanism should receive serious attention together with the Ulaanbaatar initiative. The two parallel efforts could benefit from being part of inter-connected strategies to defuse regional tension and forge greater trustpolitik.

3. The UN ESCAP headquarters can serve as an important multilateral bridge for humanitarian aid together with the multi-stakeholder Asia Pacific Peace Service Alliance (APPSA), which was launched at the UN headquarters in Bangkok last October. The U.S. Agency for International Development (USAID) could partner with UN ESCAP and the World Food Program to establish a verifiable humanitarian aid regime, building on prior food aid oversight protocols developed during the Bush administration. Mongolia also would be an excellent candidate for the training of an international volunteer corps for potential disaster and humanitarian relief and economic development projects concerning the DPRK and the broader Northeast Asia region. Mongolia has excellent working relations with the U.S. Peace Corps, which also helped facilitate the recent launch of the APPSA.

4. In the context of peninsula unification planning, regional economic cooperation on private and multi-stakeholder investment projects and the enabling of market-friendly policies could be further explored with Mongolia and other Northeast Asian partners in areas such as infrastructure, energy, and technology.5. Cultural and educational exchanges between Mongolia and the DPRK could be expanded on a multilateral basis over time to include the ROK, China, Russia, Japan and ASEAN nations together with UNESCO to further cultural bases and norms of peace.

Abstract

Progress in closing differences in many objective outcomes for blacks relative to whites has slowed, and even worsened, over the past three decades. However, over this period the racial gap in wellbeing has shrunk. In the early 1970s data revealed much lower levels of subjective well-being among blacks relative to whites. Investigating various measures of well-being, we find that the well-being of blacks has increased both absolutely and relative to that of whites. While a racial gap in well-being remains, two-fifths of the gap has closed and these gains have occurred despite little progress in closing other racial gaps such as those in income, employment, and education. Much of the current racial gap in well-being can be explained by differences in the objective conditions of the lives of black and white Americans. Thus making further progress will likely require progress in closing racial gaps in objective circumstances.

Downloads

• Subjective and Objective Indicators of Racial Progress

Event Information

With ongoing crises in Ukraine, Syria, and other regions of the world, U.S. global leadership is arguably as critical now as it has ever been. However, many question how the United States should exercise its leadership, what foreign policy agenda it should pursue, and how it should configure its military and security agencies going forward. In a recent speech at West Point, President Obama laid out his foreign policy agenda for the remainder of his presidency. While the Obama Administration will pursue the president’s agenda as laid out at West Point, others in Washington have different views on how best to manage U.S. foreign policy going forward.

On June 11, the Foreign Policy Program at Brookings will host Senator John McCain (R-AZ), former presidential candidate and member of the Senate Committee on Foreign Relations, for an address on the future of U.S. foreign and security policy. The address will be introduced by Brookings Senior Fellow and Director of Research for Foreign Policy Michael O’Hanlon, and the discussion following the Senator’s address will be moderated by Senior Fellow Robert Kagan.

After the program, Senator McCain will take audience questions.

Join the conversation on Twitter using #McCain

Editor's note: POLITICO Magazine released a list of the top 50 influential people in Washington, D.C., including Brookings Senior Fellow Robert Kagan and Assistant Secretary of State for European and Eurasian Affairs Victoria Nuland, described as "the ultimate American power couple."

Victoria Nuland and Robert Kagan fell in love “talking about democracy and the role of America in the world” on one of their first dates. It’s a shared passion that hasn’t faded over time.

It was just two years ago that President Obama was gushing to aides about an essay that Kagan, a historian and author, wrote about the myth of American decline—a theme Obama echoed in his State of the Union that January. This year, Kagan’s sprawling New Republic essay, “Superpowers Don’t Get to Retire,” insisted on America’s enduring responsibility to shape the world order—and issued a direct challenge to a president who has summarized his own foreign-policy doctrine with a minimalist “don’t-do-stupid-s—t” directive. Obama promptly invited Kagan in for a West Wing consult, but it was also clear that Kagan had helped rouse the president’s Republican critics, who have been increasingly adopting Kagan’s argument that just because it’s been a decade of wearying war in Iraq and Afghanistan doesn’t mean America can roll up its superpower carpet and stay home when new crises, from Iraq to Russia to Syria, beckon.

Nuland, overseeing European and Eurasian Affairs at the State Department, has been a strong advocate of the engaged approach her husband favors as a crisis with Russia has unfolded on her diplomatic turf this year. The point was made, rather sensationally, in February, when a leaked audio recording of her F-bomb-laden diatribe about the fecklessness of the European Union, which she accused of not exactly playing a constructive role trying to end the growing conflict in Ukraine, appeared on the Internet. Nuland, a career Foreign Service officer, has been an impassioned advocate for democracy-building in Eastern Europe, and while she got pushback from European counterparts over her “f—k the EU” comment, the United States has been leading the effort to impose sanctions on Russia since President Vladimir Putin seized Ukraine’s Crimean Peninsula and waged a proxy war in the country’s east—dragging a reluctant Europe along pretty much every step of the way.

Voters and policy makers bemoan our crumbling roads, airports, and public transit systems, but few jurisdictions do much about it. The odd thing is that historically low interest rates now make it cheap to fix or improve our public facilities. The mystery is why decision makers have passed on this opportunity.

The sorry state of the nation’s roads, bridges, and public infrastructure has been widely reported. Every few years the American Society of Civil Engineers draws up a report card on U.S. infrastructure, highlighting its strengths and shortcomings in a variety of areas—drinking water systems, wastewater, dams, roads, bridges, inland waterways, ports. The report card spotlights areas where spending on maintenance falls short of the amount needed to keep our infrastructure functioning efficiently. For many kinds of infrastructure, a bigger population and heavier utilization require us to invest in brand new facilities. In its latest report card, the ASCE awards our public infrastructure a grade of D+.

It’s hard to think of a time more attractive for public investment than years when total demand for goods and services is depressed. The Treasury’s borrowing cost for investment funds is near historical lows. Since 2011, the interest rate on 10-year government bonds has averaged 2.3 percent. Savers buying inflation-protected bonds have been willing to lend funds to the federal government at a real interest rate of just 0.22 percent.

So long as there is excess unemployment, especially in the building trades, the labor resources needed to fix or improve public facilities should be abundant and relatively inexpensive. Employment in the construction industry has rebounded as home building and business investment have improved. Nonetheless, construction employment has recovered only half the loss it experienced between its pre-recession peak in 2006 and its post-recession low in 2011. Skilled labor is not nearly as abundant as it was in 2011, but the trend in wage inflation does not suggest employers are bidding up worker salaries.

The federal government’s failure to use fiscal policy and, in particular, public investment policy to bring the nation closer to full employment represents a notable lapse in policymaking, perhaps the most grievous lapse since the crisis began. It unnecessarily prolonged the suffering of the nation’s long-term unemployed and it wasted a rare opportunity to rebuild the nation’s public infrastructure at relatively low cost.

Why did this failure occur? One reason is that policy makers were too optimistic when the financial crisis took place back in 2008. Most public and private forecasts at the time understated the severity of the economic fallout from the bank meltdown. Decision makers in Congress and the Administration may have believed infrastructure investment would be unhelpful in the recovery. Well-conceived infrastructure projects take many months to design and many years to complete. Policy makers may have believed the economic crisis would be over by the time federally infrastructure spending reached its peak.

When forecasters and Democratic policy makers recognized their error, voters had elected a Congress that supported only one kind of fiscal policy to deal with the crisis—big tax cuts focused on high-income tax payers. Whether or not such a policy could have been effective, it would not make additional funds available for infrastructure projects.

Harvard’s Lawrence Summers and Rachel Lipset recently pointed to another reason voters have failed to back a big program to boost infrastructure investment—government ineptitude. In the Boston Globe they documented the painfully slow progress of the Massachusetts Department of Transportation in overhauling a bridge across the Charles River. The bridge, which was built over 11 months back in 1912, has so far required four years for its reconstruction. No end date is in sight. In addition to the over-budget cost of the project, the overhaul has also caused massive and highly visible inconvenience for drivers, cyclists, and pedestrians trying to move between Boston and Cambridge.

Few readers can be under the illusion Boston’s experience is exceptional. Many of us pass near or use public facilities that are being rebuilt or repaired. We often see bafflingly little progress over a span of months or even years. As Summers and Lipset note, the conspicuous failure of public managers to complete capital projects speedily and on budget undermines voters’ confidence that infrastructure projects can be worthwhile.

Despite wide agreement the nation’s infrastructure needs to be modernized, we have made little progress toward that goal. On the contrary, government capital spending has shrunk significantly as a share of the economy. In 2014, net government investment spending on items other than defense dipped to a 60-year low when spending is measured as a percent of GDP. Using this indicator, net government investment has shrunk almost half compared with its level in the first decade of the century. For many reasons this is a good time to fix our public infrastructure. It is also an excellent time to overhaul public management of government capital projects.

Editor's note: This piece originally appeared in Inside Sources.

Event Information

The financial sector has undergone major changes in response to the Great Recession and post-crisis regulatory reform, as a result of the Dodd-Frank Act and Basel III. These changes have created serious questions about the sector’s role in supporting economic growth and how it affects financial and overall economic stability.

On June 8, the Initiative on Business and Public Policy at Brookings explored the intersection of the financial system and economic growth with the goal of informing the public policy debate. The event featured a keynote address by Richard Berner, director of the Office of Financial Research and other participants with a wide range of views from a variety of backgrounds. Among other issues, the experts considered the changing landscape of the financial sector; growth-promoting allocation and investment decisions; credit availability for low- and moderate-income households; the ideal balance between growth and stability; and the impact of the 2014 midterm elections on regulatory reform.

 Follow the conversation at @BrookingsEcon or #Finance.

Video

• Keynote remarks by Richard Berner
• The financial sector: How has it changed?
• The view from the trenches
• The future of the U.S. financial sector

Audio

• Can the financial sector promote growth and stability?

Transcript

• Uncorrected Transcript (.pdf)

Event Materials

• Aaron Kleins presentation
• 20150608 BAER slides
• 20150608 MEHTA slides
• 20150608_financial_sector_stability_transcript

Executive Summary ¹

The regional banks play an important role in the economy providing funding to consumers and small- and medium-sized businesses. Their model is simpler than that of the large Wall Street banks, with their business concentrated in the U.S.; they are less involved in trading and investment banking, and they are more reliant on deposits for their funding. We examined the balance sheets of 15 regional banks that had assets between $50 billion and $250 billion in 2003 and that remained in operation through 2014.

The regionals have undergone important changes in their financial structure as a result of the financial crisis and the subsequent regulatory changes:

• Total assets held by the regionals grew strongly since 2010. Their share of total bank assets has risen since 2010.

• Loans and leases make up by far the largest component of their assets. Since the crisis, however, they have substantially increased their holdings of securities and interest bearing balances, including government securities and reserves.

• The liabilities of the regionals were heavily concentrated in domestic deposits, a pattern that has intensified since the crisis. Deposits were 70 percent of liabilities in 2003, a number that fell through 2007 as they diversified their funding sources, but by 2014 deposits made up 82 percent of the total.

• Regulators are requiring large banks to increase their holdings of long term subordinated debt as a cushion against stress or failure. The regionals, as of 2014, had not increased their share of such liabilities.

• Like the largest banks, the regionals increased their loans and leases in line with their deposits prior to the crisis. And like the largest banks, this relation broke down after 2007, with loans growing much more slowly than deposits. Unlike the largest banks, the regionals have increased loans strongly since 2010, but there remains a significant gap between deposits and loans.

• The regional banks’ share of their net income from traditional sources (mostly loans) has been slowly declining over the period.

• The return on assets of the regionals was between 1.5 and 2.0 percent prior to the crisis. This turned sharply negative in the crisis before recovering after 2009. Between 2012 and 2014 return on assets for these banks was around 1.0 percent, well below the pre-crisis level.

As we saw with the largest banks, the structure and returns of the regional banks has changed as a result of the crisis and new regulation. Perhaps the most troubling change is that the volume of loans lags well behind the volume of deposits, a potential problem for economic growth. The asset and liability structure of the banks has also changed, but these banks have a simpler business model where deposits and loans still predominate.

Downloads

• Download the revised paper
• Media summary

Earlier this summer, we examined how the Big Four banks – Bank of America, Citigroup, JPMorgan Chase, and Wells Fargo – performed before, during, and after the 2007-09 financial crisis.  We also blogged about the lending trends within these large banks, expressing concern about the growing gap between deposits taken and loans made by the Big Four, and calling on policymakers to explore the issue further.  We have conducted a similar analysis on the regional banks - The regional banks: The evolution of the financial sector, Part II - and find that these smaller banks are actually faring somewhat better than their bigger counterparts.

Despite the mergers and acquisitions that happened during the crisis, the Big Four banks are a smaller share of banking today than they were in 2007.  The 15 regionals we evaluated, on the other hand, are thriving in the post-crisis environment and have a slightly larger share of total bank assets than they had in 2007.  The Big Four experienced rapid growth in the years leading up to the crisis but much slower growth in the years since.  The regionals, however, have been chugging along: with the exception of a small downward trend during the crisis, they have enjoyed slow but steady growth since 2003.

There is a gap between deposits and loans among the regionals, but it is smaller than the Big Four’s gap.  Tellingly, the regionals’ gap has remained basically constant in size during the recovery, unlike the Big Four’s gap, which is growing.  Bank loans are important to economic growth, and the regional banks are growing their loan portfolios faster than the biggest banks.  That may be a good sign for the future if the regional banks provide more competition for the big banks and a more competitive banking sector overall.

Event Information

The financial crisis is now eight years behind us, but its legacy lingers on. Many Americans are concerned about their financial security and are particularly worried about whether they will have enough for retirement. Guaranteed benefit pensions are gradually disappearing, leaving households to save and invest for themselves. What role could equities play for retail investors?

Another concern about the lingering impact of the crisis is that business investment and overall economic growth remains weak compared to expectations. Large companies are able to borrow at low interest rates, yet many of them have large cash holdings. However, many small and medium sized enterprises face difficulty funding their growth, paying high risk premiums on their borrowing and, in some cases, being unable to fund investments they would like to make. Equity funding can be an important source of growth financing.

On Friday, April 15, the Initiative on Business and Public Policy at Brookings examined what role equity markets can play for individual retirement security, small business investment and whether they can help jumpstart American innovation culture by fostering the transition from startups to billion dollar companies.

You can join the conversation and tweet questions for the panelists at #EquityMarkets.

Video

• Keynote address by Richard G. Ketchum
• Panel Discussion
• Keynote address by Roger Ferguson

Audio

• Not just for the professionals? Understanding equity markets for retail and small business investors

Transcript

• Uncorrected Transcript (.pdf)

Event Materials

• Equity Markets Retirement Security 2016 Apr 15 (2)
• 20160415_equity_markets_transcript

A major oil spill will result in cleanup spending that boosts GDP, but no one thinks oil spills are good. Oil spills and other forms of pollution are examples of negative externalities — harm caused to others by the economic activity of a firm or industry. These externalities represent a failure of the market, and unless there is corrective action, their presence means that there is too much production of something that causes negative spillovers.

That criticism can be applied to the financial services industry. Many say that it grew too large, triggered a financial crisis and damaged the rest of the economy. Is that still the case, and is financialization spoiling the economy? Despite the alarmist rhetoric around today’s finance sector, the answer is generally “no” because of changes made to financial regulation.

First, a check on the facts: How large is the industry and how much has it grown? The broad definition of the financial sector includes finance, insurance and real estate, known by the acronym “FIRE.” It was 17.5 percent of gross domestic product in 1990 and rose to 20.0 percent in 2014, but that figure is misleading as it includes office and apartment rents and leases — stuff that has little to do with Wall Street.

Finance and insurance separately peaked well before the financial crisis at 7.7 percent of GDP, which was up from 5.8 percent in 1990. In 2014, it was 7.0 percent of GDP. Employment in finance and insurance has been on a downtrend since 2003 and is currently 4.25 percent of total nonfarm payrolls. Most of those jobs are in offices and bank branches around the country. (The output data given here are drawn from the Bureau of Economic Analysis, GDP by Industry data. The employment data are from the Bureau of Labor Statistics, Payroll Employment data. Author’s calculations.)

Still, salaries and bonuses at the top are extremely attractive, so perhaps the externality plays out by drawing the best and brightest away from other more productive activities. The Harvard Crimson reported that in 2007, 23 percent of graduating Harvard seniors said they planned to enter finance. That is an impressive number, but things turned around sharply, with the 23 percent figure falling to 11.5 percent in 2009 after the financial crisis. At this point, the financial industry really isn’t large enough to crowd out other parts of the economy.

Meanwhile, the insurance industry serves an important social purpose providing life, property, and casualty insurance. AIG got into trouble in the crisis because it strayed into providing very risky financial services, not because of its main insurance business. Likewise, the core value of banks is financial intermediation between savers and investors, giving savers relatively secure and liquid assets while also funding investment.

There are critics of how well our banking industry serves this core purpose, a quality that is hard to determine. My judgment is that it does the job pretty well compared to most other countries. As the IMF reported in September 2015, the non-performing loan problem among European banks remains severe, whereas most U.S. banks now have strong balance sheets. Good financial intermediation means that most of the savings dollars are transferred to investors and are not lost through inefficient bank operations. A 2002 study that I participated in found bank productivity higher in the United States than in France or Germany.

The parts of the financial sector that give rise to the most concern are market-making, deal-making and the creation and trading of derivatives on Wall Street. The volume of market trading has increased exponentially because of the increased speed of computers and communications. Up to a certain point, the increased volume is helpful because it adds to the liquidity of markets, but the advent of high-frequency trading has taken us over the top. As Michael Lewis describes in his book Flash Boys, the high speed traders are finding ways to shave milliseconds off the time needed to make trades. That is thoroughly wasteful. As for deal-making, it has been going on for a long time — indeed the go-go years for deals were in the 1980s — so it is hard to blame the recent slowing of economic growth on this activity.

Still, the explosion of derivatives and other overly-complex instruments was problematic, and it is crystal clear that the mortgage market became too opaque and removed accountability from the system. The layering of complex derivatives on top of lousy mortgages (and other shaky assets) distorted the economy, resulted in the overbuilding of houses, and caused the financial crisis. There are plenty of people at fault besides the bankers, but the smart people on Wall Street were driving the process, and they should have known better. The excessive financialization obscured the reality of loans that depended upon ever-rising home prices and thus were never going to be paid back. There was an externality because the private calculations of potential profit ignored the risks being imposed on society.

Is that still the situation today? No. Things have changed. Banks and other financial institutions that create risks for the whole economy are now required to hold sufficient capital to cover losses even in periods of economic and financial stress, plus a liquidity buffer (they must pass “stress tests” administered by the Federal Reserve). The screws have been turned pretty tight, and the owners of large financial institutions will bear the costs of future failures — not taxpayers. This brings private incentives in line with the public interest, getting rid of the externality that gave us too much financialization in the first place. But to keep the future safe, we’ll have to make sure no one forgets what happened in the last crisis, and ensure that new risks are not created in other, less-regulated parts of the industry.

Editor's note: This piece originally appeared in the Washington Post.

Downloads

• Sample Chapter

• {9ABF977A-E4A6-41C8-B030-0FD655E07DBF}, 978-0-8157-5579-1, $24.95 Add to Cart
• {CD2E3D28-0096-4D03-B2DE-6567EB62AD1E}, 978-0-8157-5580-7, $54.95 Add to Cart

Event Information

Despite the attention on the mid-term races, few elections are competitive. Electoral competition, already low at the national level, is in decline in state and primary elections as well. Reformers, who point to gerrymandering and a host of other targets for change, argue that improving competition will produce voters who are more interested in elections, better-informed on issues, and more likely to turn out to the polls.

On October 27, the Brookings Institution—in conjunction with the Cato Institute and The Pew Research Center—presented a discussion and a groundbreaking survey exploring the attitudes and opinions of voters in competitive and noncompetitive congressional districts. The survey, part of Pew's regular polling on voter attitudes, was conducted through the weekend of October 21. A series of questions explored the public's perceptions, knowledge, and opinions about electoral competitiveness.

The discussion also explored a publication that addresses the startling lack of competition in our democratic system. The Marketplace of Democracy: Electoral Competition and American Politics (Brookings, 2006), considers the historical development, legal background, and political aspects of a system that is supposed to be responsive and accountable, yet for many is becoming stagnant, self-perpetuating, and tone-deaf. Michael McDonald, editor and Brookings visiting fellow, moderated a discussion among co-editor John Samples, director of the Center for Representative Government at the Cato Institute, and Andrew Kohut and Scott Keeter from The Pew Research Center, who also discussed the survey.

Transcript

• Transcript (.pdf)

Event Materials

• 20061027
• 20061027ppt

For Europe, it has been a rough year, or perhaps more accurately a rough decade. The terrorist attacks in London, Madrid, and elsewhere have taken a toll, as did the Iraq and Afghanistan wars. But things really got tough beginning with the Great Recession—and its prolonged duration for Europe, including grave economic crises in much of the southern part of the continent. That was followed by Vladimir Putin’s aggression against Ukraine, as well as the intensification of the Syrian, Libyan, and Yemeni conflicts with their tragic human consequences, including massive displacement of people and the greatest flow of refugees since World War II. The recent attacks in Paris and Brussels have added to the gloom and fear. This recent history, together with the advent of nationalistic and inward-looking policies in virtually all European Union member states, makes it easy to get despondent—and worry that the entire European project is failing.

To be sure, these are not the best of times. Europe is perceived by some, including Republican presidential candidate Donald Trump, as failing to invest enough in its own security, since NATO allies spend less than 1.4 percent of GDP on their armed forces while the United States spends twice that. However, we must not lose sight of the key structural advantages—and the important policy successes—that have brought Europe where it is today. For example, Europe’s recent progress in energy policy has been significant—good not only for economic and energy resilience, but also for NATO's collective handling of the revanchist Russia threat. 

[W]e must not lose sight of the key structural advantages—and the important policy successes—that have brought Europe where it is today.

For many years, analysts and policymakers have debated the question of Europe's dependence on natural gas from Russia. Today, this problem is largely solved. Russia provides only one-third of Europe’s gas. Importantly, Europe’s internal infrastructure for transporting natural gas in all desired directions has improved greatly. So have its available storage options, as well as its possibilities to import alternatives either by pipeline or in the form of liquefied natural gas. As a result, almost all member states are currently well-positioned to withstand even a worst-case scenario. 

Indeed, European Commission analyses show that even a multi-month long supply disruption could be addressed, albeit at real economic cost, by diversification and fuel switching. Progress in energy efficiency and renewable energy investments also help. There is more to do to enhance European energy security, but much has been done already. The Europeans have shown that, with ups and downs, they can address energy security themselves.

Already this energy success has contributed to a strategic success. Europe has been heavily criticized for not standing up more firmly to Russia in response to the annexation of Crimea and the conflict in eastern Ukraine. In fact, all EU member states have agreed to keep economic sanctions in place against Moscow. In addition, lifting the sanctions has been firmly attached to the implementation of the Minsk II agreement—and despite recent cracks in European solidarity, we hope that this stance will hold going forward. 

The notion that Europe is weak and dependent on Russian natural gas is a relic from the past.

The notion that Europe is weak and dependent on Russian natural gas is a relic from the past. Europe has a strong regulatory framework with which commercial entities, including Gazprom, have to abide. For those who doubt the impact of these regulations, just ask Google or Microsoft. With the end of so-called destination clauses, natural gas can be re-sold whenever required, as long as sufficient infrastructure is in place. Just last year, Germany re-exported over 30 billion cubic meters of gas, mostly Russian, in particular to Central and Eastern Europe (including Ukraine). That volume exceeds the annual consumption of every European state with the exceptions of Germany, Italy, France, and Britain.

In theory, Europe could even substantially wean itself off Russian gas if need be. To be sure, that would come at a major expense: over 200 billion euros of additional investments over a period of two years or more, and then an annual 35 billion euros, according to some calculations. That will almost surely not happen. But as a way of bounding the worst-case scenario, it is still informative. One might say that Europe has escalation dominance over Russia; the latter needs to export to Europe more than Europe need Russian hydrocarbons.

The internal energy market is not finished, but Europe’s energy security has significantly improved in recent years. Even though world markets are currently awash in resources, there is no time for complacence, and European leaders should finish the job, foremost by safeguarding the swift construction of the so-called Projects of Common Interest (key energy infrastructure projects that address the remaining bottlenecks in the EU market), so that the U.S. State Department can take new infrastructure projects like Nord Stream 2 off its priority list, and make energy policy another true European success story. It is already much of the way there, and Western security is the better for it.

The U.S.-Japan alliance continues to struggle with the issue of reducing and relocating the U.S. Marine Corps presence on Okinawa.  In a new op-ed in the Wall Street Journal, Mike Mochizuki (of The George Washington University) and I—recognizing the potential seriousness of this problem for the alliance, as well as the fact that the current plan to reduce and/or relocate has been straitjacketed by Japanese and Okinawan politics—propose a more significant set of changes. Our proposal would scale back the peacetime presence of the Marines on Okinawa even further than now planned, but it would preserve or even improve U.S. military responsiveness throughout the Western Pacific region in times of crisis or conflict.

Event Information

By most metrics, the 1991 Gulf War, also known as Operation Desert Storm, was a huge and rapid success for the United States and its allies. The mission of defeating Iraq's army, which invaded Kuwait the year prior, was done swiftly and decisively. However, the war's impact on soldiers who fought in it was lasting. Over 650,000 American men and women served in the conflict, and many came home with symptoms including insomnia, respiratory disorders, memory issues and others attributed to a variety of exposures – “Gulf War Illness."

On June 16, the Center for 21st Century Security and Intelligence at Brookings and Georgetown University Medical Center co-hosted a discussion on Desert Storm, its veterans, and how they are faring today. Representative Mike Coffman (R-Col.), the only member of Congress to serve in both Gulf wars, delivered an opening address before joining Michael O’Hanlon, senior fellow at Brookings, for a moderated discussion. Joel Kupersmith, former head of the Office of Research and Development of the Department of Veterans Affairs, convened a follow-on panel with Carolyn Clancy, deputy under secretary for health for organizational excellence at the Department of Veterans Affairs; Adrian Atizado, deputy national legislative director at Disabled American Veterans; and James Baraniuk, professor of medicine at Georgetown University Medical Center.

Audio

• Desert Storm after 25 years: Confronting the exposures of modern warfare

Transcript

• Uncorrected Transcript (.pdf)

Event Materials

• 20160616_desert_storm_transcript

Editor's Note: The World Bank’s Independent Evaluation Group (IEG) recently published a self-evaluation of its activities. Besides representing current thinking among evaluation experts at the World Bank, it also more broadly reflects some of the strengths and gaps in the approaches that evaluators use to assess and learn from the performance of the international institutions with which they work. The old question “Quis custodet ipsos custodes?” – loosely translated as “Who evaluates the evaluators?” – remains as relevant as ever. Johannes Linn served as an external peer reviewer of the self-evaluation and provides a bird’s-eye view on the lessons learned.

An Overview of the World Bank’s IEG Self-Evaluation Report

In 2011 the World Bank’s Independent Evaluation Group (IEG) carried out and published a self-evaluation of its activities. The self-evaluation team was led by an internal manager, but involved a respected external evaluation expert as the principal author and also an external peer reviewer.

The IEG self-evaluation follows best professional practices as codified by the Evaluation Cooperation Group (ECG). This group brings together the evaluation offices of seven major multilateral financial institutions in joint efforts designed to enhance evaluation performance and cooperation among their evaluators. One can therefore infer that the approach and focus of the IEG self-evaluation is representative of a broader set of practices that are currently used by the evaluation community of international financial organizations.

At the outset the IEG report states that “IEG is the largest evaluation department among Evaluation Capacity Group (ECG) members and is held in high regard by the international evaluation community. Independent assessments of IEG’s role as an independent evaluation function for the Bank and IFC rated it above the evaluation functions in most other ECG members, international nongovernmental organizations, and transnational corporations and found that IEG follows good practice evaluation principles.”

The self-evaluation report generally confirms this positive assessment. For four out of six areas of its mandate IEG gives itself the second highest rating (“good”) out of six possible rating categories. This includes (a) the professional quality of its evaluations, (b) its reports on how the World Bank’s management follows up on IEG recommendations, (c) cooperation with other evaluation offices, and (d) assistance to borrowing countries in improving their own evaluation capacity. In the area of appraising the World Bank’s self-evaluation and risk management practices, the report offers the third highest rating (“satisfactory”), while it gives the third lowest rating (“modest”) for IEG’s impact on the Bank’s policies, strategies and operations. In addition the self-evaluation concludes that overall the performance of IEG has been “good” and that it operates independently, effectively and efficiently.

The report makes a number of recommendations for improvement, which are likely to be helpful, but have limited impact on its activities. They cover measures to further enhance the independence of IEG and the consistency of evaluation practices as applied across the World Bank Group’s branches – the World Bank, the International Finance Corporation (IFC), and the Multilateral Investment Guarantee Agency (MIGA) –; to improve the design of evaluations and the engagement with Bank management upstream for greater impact; and monitoring the impact of recent organizational changes in IEG in terms of results achieved. The report also recommends that more be done to evaluate the Bank’s analytical work and that evaluations draw on comparative evidence.

Assessment

In terms of the parameters of self-evaluation set by the prevailing practice among the evaluators on international financial agencies, the IEG self-evaluation is accurate and helpful. From my own experience as an operational manager in the Bank whose activities were evaluated by IEG in years past, and as a user of IEG evaluations (and of evaluations of other international aid organizations) for my research on aid effectiveness, I concur that IEG is independent and effective in meeting its mandate as defined. Moreover, the self-evaluation produces useful quantitative evidence (including survey results, budget analysis, etc.) to corroborate qualitative judgments.

However, the self-evaluation suffers from a number of limitations in approach and gaps in focus, which are broadly representative of the practices prevalent among many of the evaluation offices of international aid agencies.

Approach of the IEG self-evaluation

The core of the self-evaluation report is about the evaluation process followed by IEG, with very little said about the substance of IEG’s evaluations. The following questions could have usefully been raised, but were not: do evaluations cover the right issues with the right intensity, such as growth and poverty; environmental, governance, and gender impacts; regional dimensions versus exclusive country or project focus; effectiveness in addressing the problems of fragile and conflict states; effectiveness in dealing with global public goods; sustainability and scaling up; etc. Therefore the report does not deal with the question of whether IEG effectively responds in its evaluations to the many important strategic debates and issues with which the development community is grappling.

Related to this limitation is the fact that the report assessed the quality of IEG’s mostly in terms of (a) whether its approach and processes meet certain standards established by the Evaluation Cooperation Group; and (b) how it is judged by stakeholders in response to a survey commissioned for this evaluation. Both these approaches are useful, but they do not have any basis in professional assessments of the quality of individual products. This is equivalent to IEG evaluating the World Bank’s projects on the quality of its processes (e.g., appraisal and supervision processes) and on the basis of stakeholder surveys, without evaluating individual products and their impacts.

Gaps in the Self-Evaluation and in Evaluation Practice

Careful reading of the report reveals six important gaps in the IEG self-evaluation, in the prevailing evaluation practice in the World Bank, and more generally in the way international financial organizations evaluate their own performance. The first three gaps relate to aspects of the evaluation approach used and the second three gaps relate to lack of focus in the self-evaluation on key internal organizational issues:

1. Impact Evaluations: The report notes that IEG carries out two to three impact evaluations per year, but it sidesteps the debate in the current evaluation literature and practice as to what extent the “gold standard” of randomized impact evaluation should occupy a much more central role. Given the importance of this debate and divergence of views, it would have been appropriate for the self-evaluation to assess IEG’s current practice of very limited use of randomized evaluations.

2. Evaluation of Scaling Up: The report does not address the question of to what extent current IEG practice not only assesses the performance of individual projects in terms of their outcomes and sustainability, but also in terms of whether the Bank has systematically built on its experience in specific projects to help scale up their impact through support for expansion or replication in follow-up operations or through effective hand-off to the government or other partners. In fact, currently IEG does not explicitly and systematically consider scaling up in its project and program evaluations. For example, in a recent IEG evaluation of World Bank funded municipal development projects (MDPs) , IEG found that the Bank has supported multiple MDPs in many countries over the years, but the evaluation did not address the obvious question whether the Bank systematically planned for the project sequence or built on its experience from prior projects in subsequent operations. While most other evaluation offices like IEG do not consider scaling up, some (in particular those of the International Fund for Agricultural Development and the United Nations Development Program) have started doing so in recent years.

3. Drawing on the Experience of and Benchmarking Against Other Institutions: The self-evaluation report does a good job in benchmarking IEG performance in a number of respects against that of other multilateral institutions. In the main text of the report it states that “IEG plans to develop guidelines for approach papers to ensure greater quality, in particular in drawing on comparative information from other sources and benchmarking against other institutions.” This is a welcome intention, but it is inadequately motivated in the rest of the report and not reflected in the Executive Summary. The reality is that IEG, like most multilateral evaluation offices, so far has not systematically drawn on the evaluations and relevant experience of other aid agencies in its evaluations of World Bank performance. This has severely limited the learning impact of the evaluations.

4. Bank Internal Policies, Management Processes and Incentives: IEG evaluations traditionally do not focus on how the Bank’s internal policies, management and incentives affect the quality of Bank engagement in countries. Therefore evaluations cannot offer any insights into whether and how Bank-internal operating modalities contribute to results. Two recent exceptions are notable exceptions. First, the IEG evaluation of the Bank’s approach to harmonization with other donors and alignment with country priorities assesses the incentives for staff to support harmonization and alignment. The evaluation concludes that there are insufficient incentives, a finding disputed by management. Second, is the evaluation of the Bank’s internal matrix management arrangements, which is currently under way. The self-evaluation notes that Bank management tried to quash the matrix evaluation on the grounds that it did not fall under the mandate of IEG. This is an unfortunate argument, since an assessment of the institutional reasons for the Bank’s performance is an essential component of any meaningful evaluation of Bank-supported programs. While making a good case for the specific instance of the matrix evaluation, the self-evaluation report shies away from a more general statement in support of engaging IEG on issues of Bank-internal policies, management processes and incentives. It is notable that IFAD’s Independent Office of Evaluation appears to be more aggressive in this regard: It currently is carrying out a full evaluation of IFAD’s internal efficiency and previous evaluations (e.g., an evaluation of innovation and scaling up) did not shy away from assessing internal institutional dimensions.

5. World Bank Governance: The IEG self-evaluation is even more restrictive in how it interprets its mandate regarding the evaluation of the World Bank’s governance structures and processes (including its approach to members’ voice and vote, the functioning of its board of directors, the selection of its senior management, etc.). It considers these topics beyond IEG’s mandate. This is unfortunate, since the way the Bank’s governance evolves will substantially affect its long-term legitimacy, effectiveness and viability as an international financial institution. Since IEG reports to the Bank’s board of directors, and many of the governance issues involve questions of the board’s composition, role and functioning, there is a valid question of how effectively IEG could carry out such an evaluation. However, it is notable that the IMF’s Independent Evaluation Office, which similarly reports to the IMF board of directors, published a full evaluation of the IMF’s governance in 2008, which effectively addressed many of the right questions.

6. Synergies between World Bank, IFC and MIGA: The self-evaluation report points out that the recent internal reorganization of IEG aimed to assure more effective and consistent evaluations across the three member branches of the World Bank Group. This is welcome, but the report does not assess how past evaluations addressed the question of whether the World Bank, IFC and MIGA effectively capitalized on the potential synergies among the three organizations. The recent evaluation of the World Bank Group’s response to the global economic crisis of 2008/9 provided parallel assessments of each agency’s performance, but did not address whether they work together effectively in maximizing their synergies. The reality is that the three organizations have deeply engrained institutional cultures and generally go their own ways rather than closely coordinating their activities on the ground. Future evaluations should explicitly consider whether the three effectively cooperate or not. While the World Bank is unique in the way it has organizationally separated its private sector and guarantee operations, other aid organizations also have problems of a lack of cooperation, coordination and synergy among different units within the agency. Therefore, the same comment also applies to their evaluation approaches.

Conclusions

Self-evaluations are valuable tools for performance assessment and IEG is to be congratulated for carrying out and publishing such an evaluation of its own activities. As for all self-evaluations, it should be seen as an input to an independent external evaluation, a decision that, for now, has apparently been postponed by the Bank’s board of directors.

IEG’s self-evaluation has many strengths and provides an overall positive assessment of IEG’s work. However, it does reflect some important limitations of analysis and of certain gaps in approach and coverage, which an independent external review should consider explicitly, and which IEG’s management should address. Since many of these issues also likely apply to most of the other evaluation approaches by other evaluation offices, the lessons have relevance beyond IEG and the World Bank.

Key lessons include:

• An evaluation of evaluations should focus not only on process, but also on the substantive issues that the institution is grappling with.
• An evaluation of the effectiveness of evaluations should include a professional assessment of the quality of evaluation products.
• An evaluation of evaluations should assess:
  o How effectively impact evaluations are used;
  o How scaling up of successful interventions is treated;
  o How the experience of other comparable institutions is utilized;
  o Whether and how the internal policies, management practices and incentives of the institution are effectively assessed;
  o Whether and how the governance of the institution is evaluated; and
  o Whether and how internal coordination, cooperation and synergy among units within the organizations are assessed.

Evaluations play an essential role in the accountability and learning of international aid organizations. Hence it is critical that evaluations address the right issues and use appropriate techniques. If the lessons above were reflected in the evaluation practices of the aid institutions, this would represent a significant step forward in the quality, relevance and likely impact of evaluations.