The replenishment process for concessional resources of the multilateral development banks is broken. We have come to this conclusion after a review of the experience with recent replenishments of multilateral development funds. We also base it on first-hand observation, since one of us was responsible for the World Bank’s International Development Association (IDA) replenishment consultations 20 years ago and recently served as the external chair for the last two replenishment consultations of the International Fund for Agricultural Development (IFAD), which closely follow the common multilateral development bank (MDB) practice. As many of the banks and their donors are preparing for midterm reviews as a first step toward the next round of replenishment consultations, this is a good time to take stock and consider what needs to be done to fix the replenishment process.

So what’s the problem?

Most of all, the replenishment process does not serve its key intended function of setting overall operational strategy for the development funds and holding the institutions accountable for effectively implementing the strategy. Instead, the replenishment consultations have turned into a time-consuming and costly process in which donor representatives from their capitals get bogged down in the minutiae of institutional management that are better left to the boards of directors and the managements of the MDBs. There are other problems, including lack of adequate engagement of recipient countries in donors’ deliberations, the lack of full participation of the donors’ representatives on the boards of the institutions in the process, and inflexible governance structures that serve as a disincentive for non-traditional donors (from emerging countries and from private foundations) to contribute.

But let’s focus on the consultation process. What does it look like? Typically, donor representatives from capitals assemble every three years (or four, in the case of the Asian Development Bank) for a year-long consultation round, consisting of four two-day meetings (including the meeting devoted to the midterm review of the ongoing replenishment and to setting the agenda for the next consultation process). For these meetings, MDB staff prepare, per consultation round, some 20 substantive documents that are intended to delve into operational and institutional performance in great detail. Each consultation round produces a long list of specific commitments (around 40 commitments is not uncommon), which management is required to implement and monitor, and report on in the midterm review. In effect, however, this review covers only half the replenishment cycle, which leads to the reporting, monitoring, and accountability being limited to the delivery of committed outputs (e.g., a specific sector strategy) with little attention paid to implementation, let alone outcomes.

The process is eerily reminiscent of the much maligned “Christmas tree” approach of the World Bank’s structural adjustment loans in the 1980s and 1990s, with their detailed matrixes of conditionality; lack of strategic selectivity and country ownership; focus on inputs rather than outcomes; and lack of consideration of the borrowers’ capacity and costs of implementing the Bank-imposed measures. Ironically, the donors successfully pushed the MDBs to give up on such conditionality (without ownership of the recipient countries) in their loans, but they impose the same kind of conditionality (without full ownership of the recipient countries and institutions) on the MDBs themselves—replenishment after replenishment.

Aside from lack of selectivity, strategic focus, and ownership of the commitments, the consultation process is also burdensome and costly in terms of the MDBs’ senior management and staff time as well as time spent by ministerial staff in donor capitals, with literally thousands of management and staff hours spent on producing and reviewing documentation. And the recent innovation of having donor representatives meet between consultation rounds as working groups dealing with long-term strategic issues, while welcome in principle, has imposed further costs on the MDBs and capitals in terms of preparing documentation and meetings.

It doesn’t have to be that way. Twenty years ago the process was much simpler and less costly. Even today, recent MDB capital increases, which mobilized resources for the non-concessional windows of the MDBs, were achieved with much simpler processes, and the replenishment consultations for special purpose funds, such as the Global Fund for HIV/AIDS, tuberculosis, and malaria and for the GAVI Alliance, are more streamlined than those of the MDBs.

So what’s to be done?

We recommend the following measures to fix the replenishment consultation process:

1. Focus on a few strategic issues and reduce the number of commitments with an explicit consideration of the costs and capacity requirements they imply. Shift the balance of monitoring and accountability from delivery of outputs to implementation and outcomes.
2. Prepare no more than five documents for the consultation process: (i) a midterm review on the implementation of the previous replenishment and key issues for the future; (ii) a corporate strategy or strategy update; (iii) the substantive report on how the replenishment resources will contribute to achieve the strategy; (iv) a financial outlook and strategy document; and (v) the legal document of the replenishment resolution.
3. Reduce the number of meetings for each replenishment round to no more than three and lengthen the replenishment period from three to four years or more.
4. Use the newly established working group meetings between replenishment consultation rounds to focus on one or two long-term, strategic issues, including how to fix the replenishment process.

The initiative for such changes lies with the donor representatives in the capitals, and from our interviews with donor representatives we understand that many of them broadly share our concerns. So this is a good time—indeed it is high time!—for them to act.

On June 20, I moderated a conversation on the future of the Department of Veterans Affairs with Secretary Robert McDonald. When he took office almost two years ago, Secretary McDonald inherited an organization in crisis: too many veterans faced shockingly long wait-times before they received care, VA officials had allegedly falsified records, and other allegations of mismanagement abounded.

Photo: Paul Morigi

Since he was sworn into office, Secretary McDonald has led the VA through a period of ambitious reform, anchored by the MyVA program. He and his team have embraced three core strategies that are securing meaningful change. They are important insights for all government leaders, and private sector ones as well.

1. Set bold goals

Secretary McDonald’s vision is for the VA to become the number one customer-service agency in the federal government. But he and his team know that words alone won’t make this happen. They developed twelve breakthrough priorities for 2016 that will directly improve service to veterans. These actionable short-term objectives support the VA’s longer term aim to deliver an exceptional experience for our veterans. By aiming high, and also drafting a concrete roadmap, the VA has put itself on a path to success.

2. Hybridize the best of public and private sectors

To accomplish their ambitious goal, VA leadership is applying the best practices of customer-service businesses around the nation. The Secretary and his colleagues are leveraging the goodwill, resources, and expertise of both the private and public sector. To do that, the VA has brought together diverse groups of business leaders, medical professionals, government executives, and veteran advocates under their umbrella MyVA Advisory Committee. Following the examples set by private sector leaders in service provision and innovation, the VA is developing user-friendly mobile apps for veterans, modernizing its website, and seeking to make hiring practices faster, more competitive, and more efficient. And so that no good idea is left unheard, the VA has created a "shark tank” to capture and enact suggestions and recommendations for improvement from the folks who best understand daily VA operations—VA employees themselves.

3. Data, data, data

The benefits of data-driven decision making in government are well known. As led by Secretary McDonald, the VA has continued to embrace the use of data to inform its policies and improve its performance. Already a leader in the collection and publication of data, the VA has recently taken even greater strides in sharing information between its healthcare delivery agencies. In addition to collecting administrative and health-outcomes information, the VA is gathering data from veterans about what they think . Automated kiosks allow veterans to check in for appointments, and to record their level of satisfaction with the services provided.

The results that the Secretary and his team have achieved speak for themselves:

• 5 million more appointments completed last fiscal year over the previous fiscal year
• 7 million additional hours of care for veterans in the last two years (based on an increase in the clinical workload of 11 percent over the last two years)
• 97 percent of appointments completed within 30 days of the veteran’s preferred date; 86 percent within 7 days; 22 percent the same day
• Average wait times of 5 days for primary care, 6 days for specialty care, and 2 days for mental health are
• 90 percent of veterans say they are satisfied or completely satisfied with when they got their appointment (less than 3 percent said they were dissatisfied or completely dissatisfied).
• The backlog for disability claims—once over 600,000 claims that were more than 125 days old—is down almost 90 percent.

Thanks to Secretary McDonald’s continued commitment to modernization, the VA has made significant progress. Problems, of course, remain at the VA and the Secretary has more work to do to ensure America honors the debt it owes its veterans, but the past two years of reform have moved the Department in the right direction. His strategies are instructive for managers of change everywhere.

Fred Dews and Andrew Kenealy contributed to this post.

Editor's Note: The introduction to the 2015 Brown Center Report on American Education appears below. Use the Table of Contents to navigate through the report online, or download a PDF of the full report.

TABLE OF CONTENTS

Part I: Girls, Boys, and Reading

Part II: Measuring Effects of the Common Core

Part III: Student Engagement

INTRODUCTION

The 2015 Brown Center Report (BCR) represents the 14^th edition of the series since the first issue was published in 2000.  It includes three studies.  Like all previous BCRs, the studies explore independent topics but share two characteristics: they are empirical and based on the best evidence available.  The studies in this edition are on the gender gap in reading, the impact of the Common Core State Standards -- English Language Arts on reading achievement, and student engagement.

Part one examines the gender gap in reading.  Girls outscore boys on practically every reading test given to a large population.  And they have for a long time.  A 1942 Iowa study found girls performing better than boys on tests of reading comprehension, vocabulary, and basic language skills.  Girls have outscored boys on every reading test ever given by the National Assessment of Educational Progress (NAEP)—the first long term trend test was administered in 1971—at ages nine, 13, and 17.  The gap is not confined to the U.S.  Reading tests administered as part of the Progress in International Reading Literacy Study (PIRLS) and the Program for International Student Assessment (PISA) reveal that the gender gap is a worldwide phenomenon.  In more than sixty countries participating in the two assessments, girls are better readers than boys. 

Perhaps the most surprising finding is that Finland, celebrated for its extraordinary performance on PISA for over a decade, can take pride in its high standing on the PISA reading test solely because of the performance of that nation’s young women.  With its 62 point gap, Finland has the largest gender gap of any PISA participant, with girls scoring 556 and boys scoring 494 points (the OECD average is 496, with a standard deviation of 94).   If Finland were only a nation of young men, its PISA ranking would be mediocre.

Part two is about reading achievement, too. More specifically, it’s about reading and the English Language Arts standards of the Common Core (CCSS-ELA).  It’s also about an important decision that policy analysts must make when evaluating public policies—the determination of when a policy begins. How can CCSS be properly evaluated? 

Two different indexes of CCSS-ELA implementation are presented, one based on 2011 data and the other on data collected in 2013.  In both years, state education officials were surveyed about their Common Core implementation efforts.  Because forty-six states originally signed on to the CCSS-ELA—and with at least forty still on track for full implementation by 2016—little variability exists among the states in terms of standards policy.  Of course, the four states that never adopted CCSS-ELA can serve as a small control group.  But variation is also found in how the states are implementing CCSS.  Some states are pursuing an array of activities and aiming for full implementation earlier rather than later.  Others have a narrow, targeted implementation strategy and are proceeding more slowly. 

The analysis investigates whether CCSS-ELA implementation is related to 2009-2013 gains on the fourth grade NAEP reading test.  The analysis cannot verify causal relationships between the two variables, only correlations.  States that have aggressively implemented CCSS-ELA (referred to as “strong” implementers in the study) evidence a one to one and one-half point larger gain on the NAEP scale compared to non-adopters of the standards.  This association is similar in magnitude to an advantage found in a study of eighth grade math achievement in last year’s BCR.  Although positive, these effects are quite small.  When the 2015 NAEP results are released this winter, it will be important for the fate of the Common Core project to see if strong implementers of the CCSS-ELA can maintain their momentum.

Part three is on student engagement.  PISA tests fifteen-year-olds on three subjects—reading, math, and science—every three years.  It also collects a wealth of background information from students, including their attitudes toward school and learning.  When the 2012 PISA results were released, PISA analysts published an accompanying volume, Ready to Learn: Students’ Engagement, Drive, and Self-Beliefs, exploring topics related to student engagement.

Part three provides secondary analysis of several dimensions of engagement found in the PISA report.  Intrinsic motivation, the internal rewards that encourage students to learn, is an important component of student engagement.  National scores on PISA’s index of intrinsic motivation to learn mathematics are compared to national PISA math scores.  Surprisingly, the relationship is negative.  Countries with highly motivated kids tend to score lower on the math test; conversely, higher-scoring nations tend to have less-motivated kids. 

The same is true for responses to the statements, “I do mathematics because I enjoy it,” and “I look forward to my mathematics lessons.”  Countries with students who say that they enjoy math or look forward to their math lessons tend to score lower on the PISA math test compared to countries where students respond negatively to the statements.  These counterintuitive finding may be influenced by how terms such as “enjoy” and “looking forward” are interpreted in different cultures.  Within-country analyses address that problem.  The correlation coefficients for within-country, student-level associations of achievement and other components of engagement run in the anticipated direction—they are positive.  But they are also modest in size, with correlation coefficients of 0.20 or less. 

Policymakers are interested in questions requiring analysis of aggregated data—at the national level, that means between-country data.  When countries increase their students’ intrinsic motivation to learn math, is there a concomitant increase in PISA math scores?  Data from 2003 to 2012 are examined.  Seventeen countries managed to increase student motivation, but their PISA math scores fell an average of 3.7 scale score points.  Fourteen countries showed no change on the index of intrinsic motivation—and their PISA scores also evidenced little change.  Eight countries witnessed a decline in intrinsic motivation.  Inexplicably, their PISA math scores increased by an average of 10.3 scale score points.  Motivation down, achievement up.

Correlation is not causation.  Moreover, the absence of a positive correlation—or in this case, the presence of a negative correlation—is not refutation of a possible positive relationship.  The lesson here is not that policymakers should adopt the most effective way of stamping out student motivation.  The lesson is that the level of analysis matters when analyzing achievement data.  Policy reports must be read warily—especially those freely offering policy recommendations.  Beware of analyses that exclusively rely on within- or between-country test data without making any attempt to reconcile discrepancies at other levels of analysis.  Those analysts could be cherry-picking the data.  Also, consumers of education research should grant more credence to approaches modeling change over time (as in difference in difference models) than to cross-sectional analyses that only explore statistical relationships at a single point in time. 

Downloads

• Download the report

In recent years, relations between the International Criminal Court (ICC) and the African Union have been crumbling. Relations between the two were once solid with strong support from AU member countries.  The ICC has had support from African countries because of the court’s ability to prevent Rwandan genocide-type atrocities and ease power differentials between small and large African nations in international trial situations. 

Brookings nonresident scholar, John Mukum Mbaku, discusses the rising tensions between the ICC and African countries in his contribution to Foresight Africa, a collection of short briefs on the top challenges and opportunities for Africa in 2014.

Read the related paper »

According to John Mbaku, some African countries see the ICC as quite problematic because of the perceived bias that the court brings mostly African criminal cases to trial.  Other countries feel their sovereignty is threatened by the court.  The African Union has gone so far as to ask member countries not to comply or cooperate with the ICC and has attempted (unsuccessfully) to withdraw from The Hague.  A major trigger for these the recent issues with the ICC and African countries was the election of Uhuru Kenyatta as president of Kenya. Kenyatta was indicted by the ICC for crimes against humanity and for allegedly inciting ethnic violence following the highly disputed 2007 elections in Kenya.  

John argues that repairing the ties between African countries and the ICC will be difficult, but might be possible through reforms to the ICC process and commitment to stronger judicial systems in African nations.

Read Foresight Africa 2014, which details the top priorities for Africa in the coming year, to learn more about the prospects for strengthening  international justice in Africa, and other critical issues for the region.