Editor's Note: The World Bank’s Independent Evaluation Group (IEG) recently published a self-evaluation of its activities. Besides representing current thinking among evaluation experts at the World Bank, it also more broadly reflects some of the strengths and gaps in the approaches that evaluators use to assess and learn from the performance of the international institutions with which they work. The old question “Quis custodet ipsos custodes?” – loosely translated as “Who evaluates the evaluators?” – remains as relevant as ever. Johannes Linn served as an external peer reviewer of the self-evaluation and provides a bird’s-eye view on the lessons learned.

An Overview of the World Bank’s IEG Self-Evaluation Report

In 2011 the World Bank’s Independent Evaluation Group (IEG) carried out and published a self-evaluation of its activities. The self-evaluation team was led by an internal manager, but involved a respected external evaluation expert as the principal author and also an external peer reviewer.

The IEG self-evaluation follows best professional practices as codified by the Evaluation Cooperation Group (ECG). This group brings together the evaluation offices of seven major multilateral financial institutions in joint efforts designed to enhance evaluation performance and cooperation among their evaluators. One can therefore infer that the approach and focus of the IEG self-evaluation is representative of a broader set of practices that are currently used by the evaluation community of international financial organizations.

At the outset the IEG report states that “IEG is the largest evaluation department among Evaluation Capacity Group (ECG) members and is held in high regard by the international evaluation community. Independent assessments of IEG’s role as an independent evaluation function for the Bank and IFC rated it above the evaluation functions in most other ECG members, international nongovernmental organizations, and transnational corporations and found that IEG follows good practice evaluation principles.”

The self-evaluation report generally confirms this positive assessment. For four out of six areas of its mandate IEG gives itself the second highest rating (“good”) out of six possible rating categories. This includes (a) the professional quality of its evaluations, (b) its reports on how the World Bank’s management follows up on IEG recommendations, (c) cooperation with other evaluation offices, and (d) assistance to borrowing countries in improving their own evaluation capacity. In the area of appraising the World Bank’s self-evaluation and risk management practices, the report offers the third highest rating (“satisfactory”), while it gives the third lowest rating (“modest”) for IEG’s impact on the Bank’s policies, strategies and operations. In addition the self-evaluation concludes that overall the performance of IEG has been “good” and that it operates independently, effectively and efficiently.

The report makes a number of recommendations for improvement, which are likely to be helpful, but have limited impact on its activities. They cover measures to further enhance the independence of IEG and the consistency of evaluation practices as applied across the World Bank Group’s branches – the World Bank, the International Finance Corporation (IFC), and the Multilateral Investment Guarantee Agency (MIGA) –; to improve the design of evaluations and the engagement with Bank management upstream for greater impact; and monitoring the impact of recent organizational changes in IEG in terms of results achieved. The report also recommends that more be done to evaluate the Bank’s analytical work and that evaluations draw on comparative evidence.

Assessment

In terms of the parameters of self-evaluation set by the prevailing practice among the evaluators on international financial agencies, the IEG self-evaluation is accurate and helpful. From my own experience as an operational manager in the Bank whose activities were evaluated by IEG in years past, and as a user of IEG evaluations (and of evaluations of other international aid organizations) for my research on aid effectiveness, I concur that IEG is independent and effective in meeting its mandate as defined. Moreover, the self-evaluation produces useful quantitative evidence (including survey results, budget analysis, etc.) to corroborate qualitative judgments.

However, the self-evaluation suffers from a number of limitations in approach and gaps in focus, which are broadly representative of the practices prevalent among many of the evaluation offices of international aid agencies.

Approach of the IEG self-evaluation

The core of the self-evaluation report is about the evaluation process followed by IEG, with very little said about the substance of IEG’s evaluations. The following questions could have usefully been raised, but were not: do evaluations cover the right issues with the right intensity, such as growth and poverty; environmental, governance, and gender impacts; regional dimensions versus exclusive country or project focus; effectiveness in addressing the problems of fragile and conflict states; effectiveness in dealing with global public goods; sustainability and scaling up; etc. Therefore the report does not deal with the question of whether IEG effectively responds in its evaluations to the many important strategic debates and issues with which the development community is grappling.

Related to this limitation is the fact that the report assessed the quality of IEG’s mostly in terms of (a) whether its approach and processes meet certain standards established by the Evaluation Cooperation Group; and (b) how it is judged by stakeholders in response to a survey commissioned for this evaluation. Both these approaches are useful, but they do not have any basis in professional assessments of the quality of individual products. This is equivalent to IEG evaluating the World Bank’s projects on the quality of its processes (e.g., appraisal and supervision processes) and on the basis of stakeholder surveys, without evaluating individual products and their impacts.

Gaps in the Self-Evaluation and in Evaluation Practice

Careful reading of the report reveals six important gaps in the IEG self-evaluation, in the prevailing evaluation practice in the World Bank, and more generally in the way international financial organizations evaluate their own performance. The first three gaps relate to aspects of the evaluation approach used and the second three gaps relate to lack of focus in the self-evaluation on key internal organizational issues:

1. Impact Evaluations: The report notes that IEG carries out two to three impact evaluations per year, but it sidesteps the debate in the current evaluation literature and practice as to what extent the “gold standard” of randomized impact evaluation should occupy a much more central role. Given the importance of this debate and divergence of views, it would have been appropriate for the self-evaluation to assess IEG’s current practice of very limited use of randomized evaluations.

2. Evaluation of Scaling Up: The report does not address the question of to what extent current IEG practice not only assesses the performance of individual projects in terms of their outcomes and sustainability, but also in terms of whether the Bank has systematically built on its experience in specific projects to help scale up their impact through support for expansion or replication in follow-up operations or through effective hand-off to the government or other partners. In fact, currently IEG does not explicitly and systematically consider scaling up in its project and program evaluations. For example, in a recent IEG evaluation of World Bank funded municipal development projects (MDPs) , IEG found that the Bank has supported multiple MDPs in many countries over the years, but the evaluation did not address the obvious question whether the Bank systematically planned for the project sequence or built on its experience from prior projects in subsequent operations. While most other evaluation offices like IEG do not consider scaling up, some (in particular those of the International Fund for Agricultural Development and the United Nations Development Program) have started doing so in recent years.

3. Drawing on the Experience of and Benchmarking Against Other Institutions: The self-evaluation report does a good job in benchmarking IEG performance in a number of respects against that of other multilateral institutions. In the main text of the report it states that “IEG plans to develop guidelines for approach papers to ensure greater quality, in particular in drawing on comparative information from other sources and benchmarking against other institutions.” This is a welcome intention, but it is inadequately motivated in the rest of the report and not reflected in the Executive Summary. The reality is that IEG, like most multilateral evaluation offices, so far has not systematically drawn on the evaluations and relevant experience of other aid agencies in its evaluations of World Bank performance. This has severely limited the learning impact of the evaluations.

4. Bank Internal Policies, Management Processes and Incentives: IEG evaluations traditionally do not focus on how the Bank’s internal policies, management and incentives affect the quality of Bank engagement in countries. Therefore evaluations cannot offer any insights into whether and how Bank-internal operating modalities contribute to results. Two recent exceptions are notable exceptions. First, the IEG evaluation of the Bank’s approach to harmonization with other donors and alignment with country priorities assesses the incentives for staff to support harmonization and alignment. The evaluation concludes that there are insufficient incentives, a finding disputed by management. Second, is the evaluation of the Bank’s internal matrix management arrangements, which is currently under way. The self-evaluation notes that Bank management tried to quash the matrix evaluation on the grounds that it did not fall under the mandate of IEG. This is an unfortunate argument, since an assessment of the institutional reasons for the Bank’s performance is an essential component of any meaningful evaluation of Bank-supported programs. While making a good case for the specific instance of the matrix evaluation, the self-evaluation report shies away from a more general statement in support of engaging IEG on issues of Bank-internal policies, management processes and incentives. It is notable that IFAD’s Independent Office of Evaluation appears to be more aggressive in this regard: It currently is carrying out a full evaluation of IFAD’s internal efficiency and previous evaluations (e.g., an evaluation of innovation and scaling up) did not shy away from assessing internal institutional dimensions.

5. World Bank Governance: The IEG self-evaluation is even more restrictive in how it interprets its mandate regarding the evaluation of the World Bank’s governance structures and processes (including its approach to members’ voice and vote, the functioning of its board of directors, the selection of its senior management, etc.). It considers these topics beyond IEG’s mandate. This is unfortunate, since the way the Bank’s governance evolves will substantially affect its long-term legitimacy, effectiveness and viability as an international financial institution. Since IEG reports to the Bank’s board of directors, and many of the governance issues involve questions of the board’s composition, role and functioning, there is a valid question of how effectively IEG could carry out such an evaluation. However, it is notable that the IMF’s Independent Evaluation Office, which similarly reports to the IMF board of directors, published a full evaluation of the IMF’s governance in 2008, which effectively addressed many of the right questions.

6. Synergies between World Bank, IFC and MIGA: The self-evaluation report points out that the recent internal reorganization of IEG aimed to assure more effective and consistent evaluations across the three member branches of the World Bank Group. This is welcome, but the report does not assess how past evaluations addressed the question of whether the World Bank, IFC and MIGA effectively capitalized on the potential synergies among the three organizations. The recent evaluation of the World Bank Group’s response to the global economic crisis of 2008/9 provided parallel assessments of each agency’s performance, but did not address whether they work together effectively in maximizing their synergies. The reality is that the three organizations have deeply engrained institutional cultures and generally go their own ways rather than closely coordinating their activities on the ground. Future evaluations should explicitly consider whether the three effectively cooperate or not. While the World Bank is unique in the way it has organizationally separated its private sector and guarantee operations, other aid organizations also have problems of a lack of cooperation, coordination and synergy among different units within the agency. Therefore, the same comment also applies to their evaluation approaches.

Conclusions

Self-evaluations are valuable tools for performance assessment and IEG is to be congratulated for carrying out and publishing such an evaluation of its own activities. As for all self-evaluations, it should be seen as an input to an independent external evaluation, a decision that, for now, has apparently been postponed by the Bank’s board of directors.

IEG’s self-evaluation has many strengths and provides an overall positive assessment of IEG’s work. However, it does reflect some important limitations of analysis and of certain gaps in approach and coverage, which an independent external review should consider explicitly, and which IEG’s management should address. Since many of these issues also likely apply to most of the other evaluation approaches by other evaluation offices, the lessons have relevance beyond IEG and the World Bank.

Key lessons include:

• An evaluation of evaluations should focus not only on process, but also on the substantive issues that the institution is grappling with.
• An evaluation of the effectiveness of evaluations should include a professional assessment of the quality of evaluation products.
• An evaluation of evaluations should assess:
  o How effectively impact evaluations are used;
  o How scaling up of successful interventions is treated;
  o How the experience of other comparable institutions is utilized;
  o Whether and how the internal policies, management practices and incentives of the institution are effectively assessed;
  o Whether and how the governance of the institution is evaluated; and
  o Whether and how internal coordination, cooperation and synergy among units within the organizations are assessed.

Evaluations play an essential role in the accountability and learning of international aid organizations. Hence it is critical that evaluations address the right issues and use appropriate techniques. If the lessons above were reflected in the evaluation practices of the aid institutions, this would represent a significant step forward in the quality, relevance and likely impact of evaluations.

Since its 50th anniversary in 1994, the World Bank has been led by four presidents: Lewis Preston until his untimely death in 1995; then James Wolfensohn, who gave the institution new energy, purpose and legitimacy; followed by Paul Wolfowitz, whose fractious management tossed the World Bank into deep crisis; and most recently, Robert Zoellick, who will be remembered for having stabilized the bank and provided effective leadership during its remarkably swift and strong response to the global financial crisis.

Throughout these years of ups and downs in the bank’s leadership, standing and lending, the overall trend of its global role was downhill. While it remains one of the world’s largest multilateral development finance institutions, its position relative to other multilateral financing mechanisms is now much less prominent. Other multilateral institutions have taken over key roles. For example, the European Union agencies and the regional development banks have rapidly expanded their portfolios, and new “vertical funds” such as the Global Fund for AIDS, Tuberculosis and Malaria have become major funding vehicles. At the same time, according to a 2011 OECD Development Assistance Committee report multilateral aid has declined as a share of total aid. Meanwhile, non-governmental aid flows have dramatically increased, including those from major foundations like the Bill and Melinda Gates Foundation, but also from new internet-based channels bundling small individual donations, such as Global Giving. The World Bank— which 20 years ago was still the biggest and most powerful global development agency and hence a ready target for criticism— today is just one of the many institutions that offer for development to the poor and emerging market economies.

Against this backdrop, the World Bank, its members and Dr. Kim face three options in its long-term trajectory over the next 10 to 20 years: 1) the bank can continue on its current path of gradual decline; 2) it might be radically scaled back and eventually eliminated, as other aid channels take over; or 3) it can dramatically reinvent itself as a global finance institution that bundles resources for growing global needs.

There is no doubt in this author’s mind that the World Bank should remain a key part of the global governance architecture, but that requires that the new president forge an ambitious long-term vision for the bank – something that has been lacking for the last 30 years – and then reform the institution and build the authorizing environment that will make it possible to achieve the vision.

Option 1: “Business as Usual” = Continued Gradual Decline

The first option, reflecting the business-as-usual approach that characterized most of the Zoellick years of leadership will mean that the bank will gradually continue to lose in scope, funding and relevance. Its scope will be reduced since the emerging market economies find the institution insufficiently responsive to their needs. They have seen the regional development banks take on increasing importance, as reflected in the substantially greater capital increases in recent years for some of these institutions than for the World Bank in relative terms (and in the case of the Asian Development Bank, even in absolute terms). And emerging market economies have set up their own thriving regional development banks without participation of the industrial countries, such as the Caja Andina de Fomento (CAF) in Latin America and the Eurasian Development Bank in the former Soviet Union. This trend will be reinforced with the creation of a “South Bank” or “BRIC Bank”, an initiative that is currently well underway.

At the same time, the World Bank’s soft loan window, the International Development Association (IDA), will face less support from industrial countries going through deep fiscal crises, heightened competition from other concessional funds, and a perception of reduced need, as many of the large and formerly poor developing countries graduate to middle-income status. It is significant that for the last IDA replenishment much of the increase in resources was due to its growing reliance on advance repayments made by some of its members and commitments against future repayments, thus in effect mortgaging its future financial capacity. The World Bank’s status as a knowledge leader in development will also continue to be challenged with the rise of research from developing countries and growing think tank capacity, as well as a proliferation of private and official agencies doling out advice and technical assistance.

As a result, under this option, over the next 10 to 20 years the World Bank will likely become no more than a shadow of the preeminent global institution it once was. It will linger on but will not be able to contribute substantially to address any of the major global financial, economic or social challenges in the future.
 
Option 2: “The Perfect Storm” = Breaking Up the World Bank

In 1998, the U.S. Congress established a commission to review and advise on the role of the international financial institutions. In 2000, the commission, led by Professor Allan Meltzer, released its recommendations, which included far-reaching changes for the International Monetary Fund and the World Bank, most of them designed to reduce the scope and financial capacities of these institutions in line with the conservative leanings of the majority of the commission’s members. For the World Bank, the “Meltzer Report” called for much of its loan business and financial assets to be devolved to the regional development banks, in effect ending the life of the institution as we know it. The report garnered some attention when it was first issued, but did not have much impact in the way the institution was run in the following 10 years.

In 2010, the U.S. Senate Foreign Relations Committee released a report on the international financial institutions, which called on them to aim toward “succeeding in their development and economic missions and thereby putting them out of business”. However, it did not recommend a drastic restructuring of the multilateral development banks, and instead argued strongly against any dilution of the U.S. veto right, its lock on leadership selection, and its voting share at the IMF and World Bank. While not dramatic in its short-term impact, these recommendations were likely a strong factor in the subsequent decisions made by the Obama administration to oppose a substantial increase in contributions by emerging markets during the latest round of capital increase at the World Bank to push for an American to replace Robert Zoellick as World Bank president. These actions reinforced for emerging market countries that the World Bank would not change sufficiently and quickly enough to serve their interests, and thus helped create the momentum for setting up a new “South Bank.”

While there seems to be no imminent risk of a break-up of the World Bank along the lines recommended by the Meltzer Report, the combination of fiscal austerity and conservative governments in key industrial countries, compounded by a declining interest of the emerging market countries in sustaining the institution’s future, could create the perfect storm for the bank. Specifically, as governments face constrained fiscal resources, confront the increasing fragmentation of the multilateral aid architecture, and take steps to consolidate their own aid agencies, they might conclude that it would be more efficient and fiscally prudent to rationalize the international development system. There is a obvious overlap on the ground in the day-to-day business of the World Bank and that of the regional development banks. This is a reality which is being fostered by the growing decentralization of the World Bank into regional hubs; in fact, a recent evaluation by the World Bank’s Independent Evaluation Group concluded that “[r]ather than functioning as a global institution, the bank is at risk of evolving into six regional banks”. With the growing financial strength, institutional capacity and dynamism, and the apparently greater legitimacy of regional development banks among their regional members, shareholders might eventually decide that consolidation of the World Bank’s operations with those of the regional development banks, in favor of the latter, is the preferred approach.

There are lots of reasons to think that this drastic step would be difficult to take politically, financially and administratively, and therefore the inertia common to the international governance architecture will also prevail in this case. However, the new World Bank president would be well advised to be prepared for the possibility of a “perfect storm” under which the idea of eviscerating the World Bank could gain some traction,. The more the bank is seen to fade away, as postulated under Option 1 above, the greater is the likelihood that Option 2 would be given serious consideration.

Option 3: “A Different World Bank” = Creating a Stronger Global Institution for the Coming Decades

Despite all the criticism and the decline in its relative role as a development finance institution in recent decades, the World Bank is still one of the strongest and most effective development institutions in a world. According to a recent independent ranking of the principal multilateral and bilateral aid institutions by the Brookings Institution and the Center for Global Development “IDA consistently ranks among the best aid agencies in each dimension of quality”.

A third, radically different option from the first two, would build on this strength and ensure that the world has an institution 10 to 20 years from now which helps the global community and individual countries to respond effectively to the many global challenges which the world will undoubtedly face: continued poverty, hunger, conflict and fragility, major infrastructure and energy needs, education and health challenges, and global warming and environmental challenges. On top of this, global financial crises will likely recur and require institutions like the World Bank to help countries provide safety nets and the structural foundations of long-term growth, as the bank has amply demonstrated since 2008. With this as a broad mandate, how could the World Bank respond under new dynamic?

First, it would change its organizational and operating modalities to take a leaf out of the book of the vertical funds, which have been so successful in tackling major development challenges in a focused and scaled-up manner. This means substantially rebalancing the internal matrix between the regional and country departments on the one hand and the technical departments on the other hand. According to the same evaluation cited above, the World Bank has tipped too far toward short-term country priorities and has failed to adequately reflect the need for long-term, dedicated sectoral engagement. The World Bank needs to fortify its reputation as an institution that can muster the strongest technical expertise, fielding team with broad global experience and with first rate regional and country perspective. This does not imply that the World Bank would abandon its engagement at the country level, but it means that it would systematically support the pursuit of long-term sectoral and sub-sectoral strategies at the country level, linked to regional and global initiatives, and involving private-public partnership to assure that development challenges are addressed at scale and in a sustained manner.

Second, recognizing that all countries have unmet needs for which they need long-term finance and best practice in areas such as infrastructure, energy, climate change and environment, the World Bank could become a truly global development institution by opening up its funding windows to all countries, not just an arbitrarily defined subset of developing countries. This would require substantially revising the current graduation rules and possibly the financial instruments. This would mean that the World Bank becomes the global equivalent of the European Investment Bank (EIB) and of the German Kreditanstalt fuer Wiederaufbau (KfW)—development banks that have successfully supported the infrastructure development of the more advanced countries.

Third, the World Bank would focus its own knowledge management activities and support for research and development in developing countries much more on a search for effective and scalable solutions, linked closely to its operational engagement which would be specifically designed to support the scaling up of tested innovations, along the lines pioneered by the Bill and Melinda Gates Foundation.

Fourth, for those countries with strong project management capacities, the World Bank would dramatically simplify its lending processes, following the example of the EIB. This would make it a much more efficient operational institution, making it a more attractive partner to its borrowing member countries, especially the emerging market economies.

Fifth, the membership of the World Bank would fix some fundamental problems with its financial structure and governance. It would invite the emerging market economies to make significantly larger contributions to its capital base in line with their much-enhanced economic and financial capacities. It would revamp the bank’s voting and voice rules to reflect the changed global economic weights and financial contributions of emerging markets. The bank would also explore, based on the experience of the vertical funds, tapping the resources of non-official partners, such as foundations and the private sector as part of its capital and contribution base. Of course, this would bring with it further significant changes in the governance of the World Bank. And the bank would move swiftly to a transparent selection of its leadership on the basis of merit without reference to nationality.

Conclusion: The New World Bank President Needs to Work with the G-20 Leaders to Chart a Course Forward
 
The new president will have to make a choice between these three options. Undoubtedly, the easiest choice is “business-as-usual”, perhaps embellished with some marginal changes that reflect the perspective and new insights that an outsider will bring. There is no doubt that the forces of institutional and political inertia tend to prevent dramatic change. However, it is also possible that Dr. Kim, with his background in a relatively narrow sectoral area may recognize the need for a more vertical approach in the bank’s organizational and operational model. Therefore, he may be more inclined than others to explore Option 3.

If he pursues Option 3, Dr. Kim will need a lot of help. The best place to look for help might be the G-20 leadership. One could hope that at least some of the leaders of the G-20 understand that Options 1 and 2 are not in the interest of their countries and the international community. Hopefully, they would be willing to push their peers to contemplate some radical changes in the multilateral development architecture. This might involve the setting up of a high-level commission as recently recommended by this author, which would review the future of the World Bank as part of a broader approach to rationalize the multilateral system in the interest of greater efficiency and effectiveness. But in setting up such a commission, the G-20 should state a clear objective, namely that the World Bank, perhaps the strongest existing global development institution, should not be gutted or gradually starved out of existence. Instead, it needs to be remade into a focused, effective and truly global institution. If Dr. Kim embraces this vision and develops actionable ideas for the commission and the G-20 leaders to consider and support, then he may bring the right medicine for an ailing giant.

Leaders will head to the G-20 Summit in Los Cabos, Mexico, among renewed serious concern about the world economy. The turmoil that started with the U.S. subprime mortgage crisis has resulted in now almost five years of ongoing instability. The emerging market economies fared much better than the advanced economies and pulled out of the crisis already in 2009, but the slowdown we are now facing in 2012 is again global, demonstrating the interdependence in the world economy. The emerging market economies have stronger underlying trend growth rates, but they remain vulnerable to a downturn in the advanced economies. The center of concern is now squarely on Europe, with a recession threatening most European countries, even those that had reasonably good performances so far. After an encouraging start in 2012, the U.S. economy, while not close to a recession, is also showing signs of a slowdown rather than the hoped for steady acceleration of growth. And the slowdown is spreading across the globe.

At a time like this it would be desirable and necessary that the G-20 show real initiative and cohesion. The essays in this collection look at the challenge from various angles. There is concern that the G-20 is losing its sense of purpose, that cohesion is decreasing rather than increasing, and that policy initiatives are reactive to events rather than proactive. Let us hope that at this moment of great difficulty, the G-20 will succeed in giving the world economy a new sense of direction and confidence. It is much needed.

Download » (PDF)

Editor’s Note: The following piece is a chapter from the 2012 edition of Eurasian Development Bank’s Eurasian Integration Yearbook.

INTRODUCTION

For centuries Central Asia was in the backwater of global political and economic attention, tales of “Great Games” and “Silk Roads” notwithstanding. However, interest in Central Asia from outside the region has been on the rise in recent years: Central Asia’s energy resources are of great importance to its neighbours in Europe and Asia. In addition, China wants a peaceful backyard, while Russia considers Central Asia part of its historical economic and regional interests and draws heavily on Central Asia migrants. Turkey is attracted by the common Turkic heritage of the region. Iran shares language and cultural ties with the Tajik people. The Central Asia’s Islamic tradition connects it with the Middle East and other Islamic countries. And now NATO countries rely on Central Asia for transit of their nonlethal military supplies in their engagement in Afghanistan.

There is wide agreement that economic prosperity and political stability in Central Asia is critical not only for the 60-plus million inhabitants of the region, but also for Central Asia’s neighbours, since Central Asia serves as a strategically important land bridge between Europe and Asia. Since the five Central Asian countries are landlocked small economies, a critical prerequisite for long-term economic growth and political stability is successful economic integration underpinned by effective regional cooperation.

This paper therefore addresses the central question of what are the prospects for regional economic integration and regional cooperation in Central Asia. It starts by briefly reviewing the role of Central Asia in the context of the overall process of Eurasian continental economic integration. It then considers what are the benefits and obstacles of regional integration and cooperation in Central Asia against the backdrop of lessons of international experience with regional integration and cooperation, and looks at four of the most important recent regional cooperation initiatives. In closing, the paper provides an answer to the question whether regional integration and cooperation in Central Asia are for real or only a mirage.

Downloads

• Download the full paper

The challenge of rural poverty and food insecurity in the developing world remains daunting. Recent estimates show that “there are still about 1.2 billion extremely poor people in the world. In addition, about 870 million people are undernourished, and about 2 billion people suffer from micronutrient deficiency. About 70 percent of the world’s poor live in rural areas, and many have some dependency on agriculture,” (Cleaver 2012). Addressing this challenge by assisting rural small-holder farmers in developing countries is the mandate of the International Fund for Agricultural Development (IFAD), an international financial institution based in Rome.

The International Fund for Agricultural Development is a relatively small donor in the global aid architecture, accounting for approximately one-half of 1 percent of all aid paid directly to developing countries in 2010. Although more significant in its core area of agricultural and rural development, IFAD still accounts for less than 5 percent of total official development assistance in that sector.1 Confronted with the gap between its small size and the large scale of the problem it has been mandated to address, IFAD seeks ways to increase its impact for every dollar it invests in agriculture and rural development on behalf of its member states. One indicator of this intention to scale up is that it has set a goal to reach 90 million rural poor between 2012 and 2015 and lift 80 million out of poverty during that time. These numbers are roughly three times the number of poor IFAD has reached previously during a similar time span. More generally, IFAD has declared that scaling up is “mission critical,” and this scaling-up objective is now firmly embedded in its corporate strategy and planning statements. Also, increasingly, IFAD’s operational practices are geared towards helping its clients achieve scaling up on the ground with the support of its loans and grants.

This was not always the case. For many years, IFAD stressed innovation as the key to success, giving little attention to systematically replicating and building on successful innovations. In this regard, IFAD was not alone. In fact, few aid agencies have systematically pursued the scaling up of successful projects. However, in 2009, IFAD management decided to explore how it could increase its focus on scaling up. It gave a grant to the Brookings Institution to review IFAD’s experience with scaling up and to assess its operational strategies, policies and processes with a view to strengthening its approach to scaling up. Based on an extensive review of IFAD documentation, two country case studies and intensive interactions with IFAD staff and managers, the Brookings team prepared a report that it submitted to IFAD management in June 2010 and published as a Brookings Global Working Paper in early 2011 (Linn et al. 2011).

Download the paper (PDF) »

Downloads

• Download the paper

Downloads

• Sample Chapter
• Table of Contents

• {9ABF977A-E4A6-41C8-B030-0FD655E07DBF}, 978-0-8157-2419-3, $29.95 Add to Cart

Event Information

The year 2015 will be a milestone year, with the adoption of the Sustainable Development Goals (SDGs) and the post-2015 development agenda by world leaders in September; the Addis Ababa Accord on financing for development in July; and the conclusion of climate negotiations at COP21 in Paris in December. The draft Addis Ababa Accord, which focuses on the actions needed to attain the SDGs, highlights the key role envisaged for the multilateral development banks (MDBs) in the post-2015 agenda. Paragraph 65 of the draft accord notes: “We call on the international finance institutions to establish a process to examine the role, scale, and functioning of the multilateral and regional development finance institutions to make them more responsive to the sustainable development agenda.”          

Against this backdrop, on April 18, 2015, the Global Economy and Development program at Brookings held a private roundtable with the leaders of the MDBs and other key stakeholders to discuss the role of the MDBs in supporting the post-2015 development agenda.

The meeting focused on four questions:

1. What does the post-2015 development agenda and the ambitions of the Addis and Paris conferences imply for the MDBs?


Given the ability of the MDBs to leverage shareholder resources, they can be efficient and effective mechanisms for scaling up development cooperation, particularly with respect to the agenda on investing in people and to the financing of sustainable infrastructure. New roles, instruments and partnerships might be needed.

2. How can MDBs best take advantage of the political attention that is being paid to the various conferences in 2015?   


The World Bank and selected regional development banks have launched a series of initiatives to optimize their balance sheets, address other constraints and enhance their catalytic role in crowding in private finance. And new institutions and mechanisms are coming to the fore. But the responses are not coordinated to best take advantage of each MDB’s comparative advantage.

3. What are the key impediments to scaling up the role and engagement of the MDBs?


Views on constraints are likely to differ but discussions should cover policy dialogue, capacity building, capital, leverage, shareholder backing on volume, instruments on leverage and risk mitigation, safeguards, and governance. 

4. How should the MDBs respond to the proposal to establish a process to examine the role, scale and functioning of the multilateral and regional development finance institutions to make them more responsive to the sustainable development agenda?   


A proactive response and engagement on the part of the MDBs would facilitate a better understanding of the contribution that the MDBs can make and greater support among shareholders for a coherent and stepped-up role.

Event Materials

• Participant List
• MDB PostEvent Summary

The replenishment process for concessional resources of the multilateral development banks is broken. We have come to this conclusion after a review of the experience with recent replenishments of multilateral development funds. We also base it on first-hand observation, since one of us was responsible for the World Bank’s International Development Association (IDA) replenishment consultations 20 years ago and recently served as the external chair for the last two replenishment consultations of the International Fund for Agricultural Development (IFAD), which closely follow the common multilateral development bank (MDB) practice. As many of the banks and their donors are preparing for midterm reviews as a first step toward the next round of replenishment consultations, this is a good time to take stock and consider what needs to be done to fix the replenishment process.

So what’s the problem?

Most of all, the replenishment process does not serve its key intended function of setting overall operational strategy for the development funds and holding the institutions accountable for effectively implementing the strategy. Instead, the replenishment consultations have turned into a time-consuming and costly process in which donor representatives from their capitals get bogged down in the minutiae of institutional management that are better left to the boards of directors and the managements of the MDBs. There are other problems, including lack of adequate engagement of recipient countries in donors’ deliberations, the lack of full participation of the donors’ representatives on the boards of the institutions in the process, and inflexible governance structures that serve as a disincentive for non-traditional donors (from emerging countries and from private foundations) to contribute.

But let’s focus on the consultation process. What does it look like? Typically, donor representatives from capitals assemble every three years (or four, in the case of the Asian Development Bank) for a year-long consultation round, consisting of four two-day meetings (including the meeting devoted to the midterm review of the ongoing replenishment and to setting the agenda for the next consultation process). For these meetings, MDB staff prepare, per consultation round, some 20 substantive documents that are intended to delve into operational and institutional performance in great detail. Each consultation round produces a long list of specific commitments (around 40 commitments is not uncommon), which management is required to implement and monitor, and report on in the midterm review. In effect, however, this review covers only half the replenishment cycle, which leads to the reporting, monitoring, and accountability being limited to the delivery of committed outputs (e.g., a specific sector strategy) with little attention paid to implementation, let alone outcomes.

The process is eerily reminiscent of the much maligned “Christmas tree” approach of the World Bank’s structural adjustment loans in the 1980s and 1990s, with their detailed matrixes of conditionality; lack of strategic selectivity and country ownership; focus on inputs rather than outcomes; and lack of consideration of the borrowers’ capacity and costs of implementing the Bank-imposed measures. Ironically, the donors successfully pushed the MDBs to give up on such conditionality (without ownership of the recipient countries) in their loans, but they impose the same kind of conditionality (without full ownership of the recipient countries and institutions) on the MDBs themselves—replenishment after replenishment.

Aside from lack of selectivity, strategic focus, and ownership of the commitments, the consultation process is also burdensome and costly in terms of the MDBs’ senior management and staff time as well as time spent by ministerial staff in donor capitals, with literally thousands of management and staff hours spent on producing and reviewing documentation. And the recent innovation of having donor representatives meet between consultation rounds as working groups dealing with long-term strategic issues, while welcome in principle, has imposed further costs on the MDBs and capitals in terms of preparing documentation and meetings.

It doesn’t have to be that way. Twenty years ago the process was much simpler and less costly. Even today, recent MDB capital increases, which mobilized resources for the non-concessional windows of the MDBs, were achieved with much simpler processes, and the replenishment consultations for special purpose funds, such as the Global Fund for HIV/AIDS, tuberculosis, and malaria and for the GAVI Alliance, are more streamlined than those of the MDBs.

So what’s to be done?

We recommend the following measures to fix the replenishment consultation process:

1. Focus on a few strategic issues and reduce the number of commitments with an explicit consideration of the costs and capacity requirements they imply. Shift the balance of monitoring and accountability from delivery of outputs to implementation and outcomes.
2. Prepare no more than five documents for the consultation process: (i) a midterm review on the implementation of the previous replenishment and key issues for the future; (ii) a corporate strategy or strategy update; (iii) the substantive report on how the replenishment resources will contribute to achieve the strategy; (iv) a financial outlook and strategy document; and (v) the legal document of the replenishment resolution.
3. Reduce the number of meetings for each replenishment round to no more than three and lengthen the replenishment period from three to four years or more.
4. Use the newly established working group meetings between replenishment consultation rounds to focus on one or two long-term, strategic issues, including how to fix the replenishment process.

The initiative for such changes lies with the donor representatives in the capitals, and from our interviews with donor representatives we understand that many of them broadly share our concerns. So this is a good time—indeed it is high time!—for them to act.

So we’ve almost reached the end of the year 2015, which could go down in the history of global sustainable development efforts as one of the more significant years, with the trifecta of the approval of the Sustainable Development Goals (SDGs), the agreement on the Addis Agenda on Financing for Development (FfD) and the (shortly to be completed) Paris COP21 Climate Summit. Yet, all will depend on how the agreements with their ambitious targets are implemented on the ground.

Effective implementation will require a theory of change—a way to think about how we are to get from “here” in 2015 to “there” in 2030. The key problem is what has very appropriately been called by some “the missing middle,” i.e., the gap between the top-down global targets on the one hand and the bottom-up development initiatives, projects, and programs that are supported by governments, aid agencies, foundations, and social entrepreneurs.

One way to begin to close this gap is to aim for scaled-up global efforts in specific areas, as is pledged in the Addis Agenda, including efforts to fight global hunger and malnutrition, international tax cooperation and international cooperation to strengthen capacities of municipalities and other local authorities, investments and international coopera­tion to allow all children to complete free, equitable, inclusive and quality early childhood, primary and secondary education, and concessional and non-concessional financing.

Another way is to develop country-specific national targets and plans consistent with the SDG, Addis, and COP21 targets, as is currently being done with the assistance of the United Nations Development Program’s MAPS program. This can provide broad guidance on policy priorities and resource mobilization strategies to be pursued at the national level and can help national and international actors to prioritize their interventions in areas where a country’s needs are greatest.

However, calling for expanded global efforts in particular priority areas and defining national targets and plans is not enough. Individual development actors have to link their specific projects and programs with the national SDG, Addis, and COP21 targets. They systematically have to pursue a scaling-up strategy in their areas of engagement, i.e., to develop and pursue pathways from individual time-bound interventions to impact at a scale in a way that will help achieve the global and national targets. A recent paper I co-authored with Larry Cooley summarizes two complementary approaches of how one might design and implement such scaling-up pathways. The main point, however, is that only the pursuit of such scaling-up pathways constitutes a meaningful theory of change that offers hope for effective implementation of the new global sustainable development targets.

Fortunately, over the last decade, development analysts and agencies have increasingly focused on the question of how to scale up impact of successful development interventions. Leading the charge, the World Bank in 2004, under its president Jim Wolfensohn, organized a high-level international conference in Shanghai in cooperation with the Chinese authorities on the topic of scaling up development impact and published the associated analytical work. However, with changes in the leadership at the World Bank, the initiative passed to others in the mid-2000s, including the Brookings Institution, ExpandNet (a group of academics working with the World Health Organization), Management Systems International (MSI), and Stanford University. They developed analytical frameworks for systematically assessing scalability of development initiatives and innovations, analyzed the experience with more or less successful scaling-up initiatives, including in fragile and conflict-affected states, and established networks that bring together development experts and practitioners to share knowledge.

By now, many international development agencies (including GIZ, JICA, USAID, African Development Bank, IFAD and UNDP), foundations (including the Bill & Melinda Gates Foundation and Rockefeller Foundation) and leading development NGOs (including Heifer International, Save the Children and the World Resources Institute), among others, have focused on how best to scale up development impact, while the OECD recently introduced a prize for the most successful scaling-up development initiatives. The International Fund for Agricultural Development (IFAD) is perhaps the most advanced among the agencies, having developed a systematic operational approach to the innovation-learning-scaling-up cycle. In a collaborative effort with the Brookings Institution, IFAD reviewed its operational practices and experience and then prepared operational design and evaluation guidelines, which can serve as a good example for other development agencies. The World Bank, while yet to develop a systematic institution-wide approach to the scaling-up agenda, is exploring in specific areas how best to pursue scaled-up impact, such as in the areas of mother and child health, social enterprise innovation, and the “science of delivery.”

Now that the international community has agreed on the SDGs and the Addis Agenda, and is closing in on an agreement in Paris on how to respond to climate change, it is the right time to bridge the “missing middle” by linking the sustainable development and climate targets with effective scaling-up methodologies and practices among the development actors. In practical terms, this requires the following steps:

• Developing shared definitions, analytical frameworks, and operational approaches to scaling up among development experts;
• Developing sectoral and sub-sectoral strategies at country level that link short- and medium-term programs and interventions through scaling-up pathways with the longer-term SDG and climate targets;
• Introducing effective operational policies and practices in the development agencies in country strategies, project design, and monitoring and evaluation;
• Developing multi-stakeholder partnerships around key development interventions with the shared goal of pursuing well-identified scaling-up pathways focused on the achievement of the SDGs and climate targets;
• Developing incentive schemes based on the growing experience with “challenge funds” that focus not only on innovation, but also on scaling up, such as the recently established Global Innovation Fund; and
• Further building up expert and institutional networks to share experience and approaches, such as the Community of Practice on Scaling Up, recently set up by MSI and the Results for Development Institute.

To achieve the desired outcomes of the Sustainable Development Goals as well as the global targets from the Paris COP21 Climate Summit by 2030, governments will have to find ways to meet the top-down objectives with bottom-up approaches. A systematic focus on scaling up successful development interventions could serve to bridge this gap, or what’s been called the “missing middle.” However, the question remains how to actually address the challenge of scaling up.

When Arna Hartmann, adjunct professor of international development, and I first looked at the scaling up agenda in development work in the mid-2000s, we concluded that development agencies were insufficiently focused on supporting the scaling up of successful development interventions. The pervasive focus on one-off projects all too often resulted in what I’ve come to refer to as “pilots to nowhere.” As a first step to fix this, we recommended that each aid organization carry out a review to be sure to focus effectively on scaling up. 

The institutional dimension is critical, given their role in developing and implementing scaling up pathways. Of course, individuals serve as champions, designers, and implementers, but experience illustrates that if individuals lack a strong link to a supportive institution, scaling up is most likely to be short-lived and unsustainable. “Institutions” include many different types of organizations, such as government ministries and departments, private firms and social enterprises, civil society organizations, and both public and private external donors and financiers.

The Brookings book “Getting to Scale: How to Bring Development Solutions to Millions of Poor People” explores the opportunities and challenges that such organizations face, on their own or, better yet, partnering with each other, in scaling up the development impact of their successful interventions.

Over the past decade I have worked with 10 foreign aid institutions—multilateral and bilateral agencies, as well as big global non-governmental organizations—helping them to focus systematically on scaling up operational work and developing approaches to do so. There are common lessons that apply across the board to these agencies, with one salutary example being the International Fund for Agricultural Development (IFAD) which has tackled the scaling up agenda systematically and persistently.

Following are eight takeaway lessons I gleaned from my work with IFAD:

1. Look into the “black box” of institutions. It is not enough to decide that an institution should focus on and support scaling up of successful development interventions. You actually need to look at how institutions function in terms of their mission statement and corporate strategy, their policies and processes, their operational instruments, their budgets, management and staff incentives, and their monitoring and evaluation practices. Check out the Brookings working paper that summarizes the results of a scaling up review of the IFAD.
2. Scaling needs to be pursued institution-wide. Tasking one unit in an organization with innovation and scaling up, or creating special outside entities (like the Global Innovation Fund set up jointly by a number of donor agencies) is a good first step. But ultimately, a comprehensive approach must be mainstreamed so that all operational activities are geared toward scaling up.
3. Scaling up must be championed from the top. The governing boards and leadership of the institutions need to commit to scaling up and persistently stay on message, since, like any fundamental institutional change, effectively scaling up takes time, perhaps a decade or more as with IFAD.
4. The scaling up process must be grown within the institution. External analysis and advice from consultants can play an important role in institutional reviews. But for lasting institutional change, the leadership must come from within and involve broad participation from managers and staff in developing operational policies and processes that are tailored to an institution’s specific culture, tasks, and organizational structure.
5. A well-articulated operational approach for scaling up needs to be put in place. For more on this, take a look at a recent paper by Larry Cooley and I that reviews two helpful operational approaches, which are also covered in Cooley’s blog. For the education sector, the Center for Universal Education at Brookings just published its report “Millions Learning,” which provides a useful scaling up approach specifically tailored to the education sector.
6. Operational staffs need to receive practical guidance and training. It is not enough to tell staff that they have to focus on scaling up and then give them a general framework. They also need practical guidance and training, ideally tailored to the specific business lines they are engaged in. IFAD, for example, developed overall operational guidelines for scaling up, as well as guidance notes for specific area of engagement, including livestock development, agricultural value chains, land tenure security, etc.  This guidance and training ideally should also be extended to consultants working with the agency on project preparation, implementation, and evaluation, as well as to the agency’s local counterpart organizations.
7. New approaches to monitoring and evaluation (M&E) have to be crafted. Typically the M&E for development projects is backward looking and focused on accountability, narrow issues of implementation, and short-term results. Scaling up requires continuous learning, structured experimentation, and innovation based on evidence, including whether the enabling conditions for scaling up are being established. And it is important to monitor and evaluate the institutional mainstreaming process of scaling up to ensure that it is effectively pursued. I’d recommend looking at how the German Agency for International Development (GIZ) carried out a corporate-wide evaluation of its scaling up experience.
8. Scaling up helps aid organizations mobilize financial resources. Scaling up leverages limited institutional resources in two ways: First, an organization can multiply the impact of its own financial capacity by linking up with public and private agencies and building multi-stakeholder coalitions in support of scaling up. Second, when an organization demonstrates that it is pursuing not only one-off results but also scaled up impact, funders or shareholders of the organization tend to be more motivated to support the organization. This certainly was one of the drivers of IFAD’s successful financial replenishment consultation rounds over the last decade.

By adopting these lessons, development organizations can actually begin to scale up to the level necessary to bridge the missing middle. The key will be to assure that a focus on scaling up is not the exception but instead becomes ingrained in the institutional DNA. Simply put, in designing and implementing development programs and projects, the question needs to be answered, “What’s next, if this intervention works?”