In July 2013, the World Bank decided to phase-out lending for new coal-fired power plants in middle-income countries, except in rare circumstances where no financially feasible alternatives to coal exist. This decision was made for a combination of reasons including concerns about local air pollution and global climate change, as well as evidence that these projects have little trouble attracting private capital without World Bank involvement. Now, policymakers are considering whether the World Bank’s policy should also cover projects designed to retrofit existing coal-fired power plants in middle-income countries by adding scrubbers and other technologies that increase efficiency and reduce air pollution. 

There are several fundamental questions underlying this debate: Is financing coal power plant retrofits a good use of World Bank resources? If so, should the World Bank insist on the use of best available technologies when it finances these retrofits? These questions are vitally important, as retrofit technologies are designed to minimize toxic air pollutants, including soot and smog, which are both dangerous for human health and the world’s climate. Older coal plants without retrofit technologies are less efficient, and emit more pollutants per unit of coal burned than those with retrofits applied. Evidence shows that soot and smog can cause respiratory illness and asthma, especially in children and elderly people, and can diminish local agricultural production by reducing sunlight. Furthermore, in many countries coal plants are the single largest source of carbon dioxide emissions driving climate change. 

To help inform the policy debate, this analysis surveys the technologies in use in more than 2,000 coal-fired power plants currently in operation, under construction, or planned in middle-income countries. The findings reveal that roughly 70 percent of these power plants rely on old, inefficient technologies. Retrofitting these plants would reduce pollution, increase efficiency and save lives. In middle-income countries that do not mandate coal retrofits, the World Bank could play a helpful role in financing those improvements, particularly as part of broader policy reforms designed to reduce climate pollution and increase efficiency across the power sector.

Importantly, however, the data also show that important qualifications should be made. First, because coal is a major source of greenhouse gas emissions and retrofits are likely to keep coal plants operating longer, the World Bank should insist that retrofit projects occur within a context of national and local policy reforms designed to abate greenhouse gas pollution. Toward this end, the World Bank should continue to help countries build capacity to adopt and enforce climate pollution controls and other offsetting actions and policies. Second, the World Bank should insist that projects it finances use best available pollution control technologies. Already, the substantial majority of coal retrofits completed to date in middle-income countries have used best available technologies. These retrofits were almost universally financed exclusively by private capital. The World Bank should not use its capital to support inferior retrofit technologies that are below the standards already adopted by the private sector in middle-income countries.

Downloads

• Download the full report (PDF)

Editor's Note: The World Bank’s Independent Evaluation Group (IEG) recently published a self-evaluation of its activities. Besides representing current thinking among evaluation experts at the World Bank, it also more broadly reflects some of the strengths and gaps in the approaches that evaluators use to assess and learn from the performance of the international institutions with which they work. The old question “Quis custodet ipsos custodes?” – loosely translated as “Who evaluates the evaluators?” – remains as relevant as ever. Johannes Linn served as an external peer reviewer of the self-evaluation and provides a bird’s-eye view on the lessons learned.

An Overview of the World Bank’s IEG Self-Evaluation Report

In 2011 the World Bank’s Independent Evaluation Group (IEG) carried out and published a self-evaluation of its activities. The self-evaluation team was led by an internal manager, but involved a respected external evaluation expert as the principal author and also an external peer reviewer.

The IEG self-evaluation follows best professional practices as codified by the Evaluation Cooperation Group (ECG). This group brings together the evaluation offices of seven major multilateral financial institutions in joint efforts designed to enhance evaluation performance and cooperation among their evaluators. One can therefore infer that the approach and focus of the IEG self-evaluation is representative of a broader set of practices that are currently used by the evaluation community of international financial organizations.

At the outset the IEG report states that “IEG is the largest evaluation department among Evaluation Capacity Group (ECG) members and is held in high regard by the international evaluation community. Independent assessments of IEG’s role as an independent evaluation function for the Bank and IFC rated it above the evaluation functions in most other ECG members, international nongovernmental organizations, and transnational corporations and found that IEG follows good practice evaluation principles.”

The self-evaluation report generally confirms this positive assessment. For four out of six areas of its mandate IEG gives itself the second highest rating (“good”) out of six possible rating categories. This includes (a) the professional quality of its evaluations, (b) its reports on how the World Bank’s management follows up on IEG recommendations, (c) cooperation with other evaluation offices, and (d) assistance to borrowing countries in improving their own evaluation capacity. In the area of appraising the World Bank’s self-evaluation and risk management practices, the report offers the third highest rating (“satisfactory”), while it gives the third lowest rating (“modest”) for IEG’s impact on the Bank’s policies, strategies and operations. In addition the self-evaluation concludes that overall the performance of IEG has been “good” and that it operates independently, effectively and efficiently.

The report makes a number of recommendations for improvement, which are likely to be helpful, but have limited impact on its activities. They cover measures to further enhance the independence of IEG and the consistency of evaluation practices as applied across the World Bank Group’s branches – the World Bank, the International Finance Corporation (IFC), and the Multilateral Investment Guarantee Agency (MIGA) –; to improve the design of evaluations and the engagement with Bank management upstream for greater impact; and monitoring the impact of recent organizational changes in IEG in terms of results achieved. The report also recommends that more be done to evaluate the Bank’s analytical work and that evaluations draw on comparative evidence.

Assessment

In terms of the parameters of self-evaluation set by the prevailing practice among the evaluators on international financial agencies, the IEG self-evaluation is accurate and helpful. From my own experience as an operational manager in the Bank whose activities were evaluated by IEG in years past, and as a user of IEG evaluations (and of evaluations of other international aid organizations) for my research on aid effectiveness, I concur that IEG is independent and effective in meeting its mandate as defined. Moreover, the self-evaluation produces useful quantitative evidence (including survey results, budget analysis, etc.) to corroborate qualitative judgments.

However, the self-evaluation suffers from a number of limitations in approach and gaps in focus, which are broadly representative of the practices prevalent among many of the evaluation offices of international aid agencies.

Approach of the IEG self-evaluation

The core of the self-evaluation report is about the evaluation process followed by IEG, with very little said about the substance of IEG’s evaluations. The following questions could have usefully been raised, but were not: do evaluations cover the right issues with the right intensity, such as growth and poverty; environmental, governance, and gender impacts; regional dimensions versus exclusive country or project focus; effectiveness in addressing the problems of fragile and conflict states; effectiveness in dealing with global public goods; sustainability and scaling up; etc. Therefore the report does not deal with the question of whether IEG effectively responds in its evaluations to the many important strategic debates and issues with which the development community is grappling.

Related to this limitation is the fact that the report assessed the quality of IEG’s mostly in terms of (a) whether its approach and processes meet certain standards established by the Evaluation Cooperation Group; and (b) how it is judged by stakeholders in response to a survey commissioned for this evaluation. Both these approaches are useful, but they do not have any basis in professional assessments of the quality of individual products. This is equivalent to IEG evaluating the World Bank’s projects on the quality of its processes (e.g., appraisal and supervision processes) and on the basis of stakeholder surveys, without evaluating individual products and their impacts.

Gaps in the Self-Evaluation and in Evaluation Practice

Careful reading of the report reveals six important gaps in the IEG self-evaluation, in the prevailing evaluation practice in the World Bank, and more generally in the way international financial organizations evaluate their own performance. The first three gaps relate to aspects of the evaluation approach used and the second three gaps relate to lack of focus in the self-evaluation on key internal organizational issues:

1. Impact Evaluations: The report notes that IEG carries out two to three impact evaluations per year, but it sidesteps the debate in the current evaluation literature and practice as to what extent the “gold standard” of randomized impact evaluation should occupy a much more central role. Given the importance of this debate and divergence of views, it would have been appropriate for the self-evaluation to assess IEG’s current practice of very limited use of randomized evaluations.

2. Evaluation of Scaling Up: The report does not address the question of to what extent current IEG practice not only assesses the performance of individual projects in terms of their outcomes and sustainability, but also in terms of whether the Bank has systematically built on its experience in specific projects to help scale up their impact through support for expansion or replication in follow-up operations or through effective hand-off to the government or other partners. In fact, currently IEG does not explicitly and systematically consider scaling up in its project and program evaluations. For example, in a recent IEG evaluation of World Bank funded municipal development projects (MDPs) , IEG found that the Bank has supported multiple MDPs in many countries over the years, but the evaluation did not address the obvious question whether the Bank systematically planned for the project sequence or built on its experience from prior projects in subsequent operations. While most other evaluation offices like IEG do not consider scaling up, some (in particular those of the International Fund for Agricultural Development and the United Nations Development Program) have started doing so in recent years.

3. Drawing on the Experience of and Benchmarking Against Other Institutions: The self-evaluation report does a good job in benchmarking IEG performance in a number of respects against that of other multilateral institutions. In the main text of the report it states that “IEG plans to develop guidelines for approach papers to ensure greater quality, in particular in drawing on comparative information from other sources and benchmarking against other institutions.” This is a welcome intention, but it is inadequately motivated in the rest of the report and not reflected in the Executive Summary. The reality is that IEG, like most multilateral evaluation offices, so far has not systematically drawn on the evaluations and relevant experience of other aid agencies in its evaluations of World Bank performance. This has severely limited the learning impact of the evaluations.

4. Bank Internal Policies, Management Processes and Incentives: IEG evaluations traditionally do not focus on how the Bank’s internal policies, management and incentives affect the quality of Bank engagement in countries. Therefore evaluations cannot offer any insights into whether and how Bank-internal operating modalities contribute to results. Two recent exceptions are notable exceptions. First, the IEG evaluation of the Bank’s approach to harmonization with other donors and alignment with country priorities assesses the incentives for staff to support harmonization and alignment. The evaluation concludes that there are insufficient incentives, a finding disputed by management. Second, is the evaluation of the Bank’s internal matrix management arrangements, which is currently under way. The self-evaluation notes that Bank management tried to quash the matrix evaluation on the grounds that it did not fall under the mandate of IEG. This is an unfortunate argument, since an assessment of the institutional reasons for the Bank’s performance is an essential component of any meaningful evaluation of Bank-supported programs. While making a good case for the specific instance of the matrix evaluation, the self-evaluation report shies away from a more general statement in support of engaging IEG on issues of Bank-internal policies, management processes and incentives. It is notable that IFAD’s Independent Office of Evaluation appears to be more aggressive in this regard: It currently is carrying out a full evaluation of IFAD’s internal efficiency and previous evaluations (e.g., an evaluation of innovation and scaling up) did not shy away from assessing internal institutional dimensions.

5. World Bank Governance: The IEG self-evaluation is even more restrictive in how it interprets its mandate regarding the evaluation of the World Bank’s governance structures and processes (including its approach to members’ voice and vote, the functioning of its board of directors, the selection of its senior management, etc.). It considers these topics beyond IEG’s mandate. This is unfortunate, since the way the Bank’s governance evolves will substantially affect its long-term legitimacy, effectiveness and viability as an international financial institution. Since IEG reports to the Bank’s board of directors, and many of the governance issues involve questions of the board’s composition, role and functioning, there is a valid question of how effectively IEG could carry out such an evaluation. However, it is notable that the IMF’s Independent Evaluation Office, which similarly reports to the IMF board of directors, published a full evaluation of the IMF’s governance in 2008, which effectively addressed many of the right questions.

6. Synergies between World Bank, IFC and MIGA: The self-evaluation report points out that the recent internal reorganization of IEG aimed to assure more effective and consistent evaluations across the three member branches of the World Bank Group. This is welcome, but the report does not assess how past evaluations addressed the question of whether the World Bank, IFC and MIGA effectively capitalized on the potential synergies among the three organizations. The recent evaluation of the World Bank Group’s response to the global economic crisis of 2008/9 provided parallel assessments of each agency’s performance, but did not address whether they work together effectively in maximizing their synergies. The reality is that the three organizations have deeply engrained institutional cultures and generally go their own ways rather than closely coordinating their activities on the ground. Future evaluations should explicitly consider whether the three effectively cooperate or not. While the World Bank is unique in the way it has organizationally separated its private sector and guarantee operations, other aid organizations also have problems of a lack of cooperation, coordination and synergy among different units within the agency. Therefore, the same comment also applies to their evaluation approaches.

Conclusions

Self-evaluations are valuable tools for performance assessment and IEG is to be congratulated for carrying out and publishing such an evaluation of its own activities. As for all self-evaluations, it should be seen as an input to an independent external evaluation, a decision that, for now, has apparently been postponed by the Bank’s board of directors.

IEG’s self-evaluation has many strengths and provides an overall positive assessment of IEG’s work. However, it does reflect some important limitations of analysis and of certain gaps in approach and coverage, which an independent external review should consider explicitly, and which IEG’s management should address. Since many of these issues also likely apply to most of the other evaluation approaches by other evaluation offices, the lessons have relevance beyond IEG and the World Bank.

Key lessons include:

• An evaluation of evaluations should focus not only on process, but also on the substantive issues that the institution is grappling with.
• An evaluation of the effectiveness of evaluations should include a professional assessment of the quality of evaluation products.
• An evaluation of evaluations should assess:
  o How effectively impact evaluations are used;
  o How scaling up of successful interventions is treated;
  o How the experience of other comparable institutions is utilized;
  o Whether and how the internal policies, management practices and incentives of the institution are effectively assessed;
  o Whether and how the governance of the institution is evaluated; and
  o Whether and how internal coordination, cooperation and synergy among units within the organizations are assessed.

Evaluations play an essential role in the accountability and learning of international aid organizations. Hence it is critical that evaluations address the right issues and use appropriate techniques. If the lessons above were reflected in the evaluation practices of the aid institutions, this would represent a significant step forward in the quality, relevance and likely impact of evaluations.

The U.S. nominee for the World Bank presidency, South Korean-born physician Jim Yong Kim, is one of three candidates for the post, along with Nigerian Finance Minister Ngozi Okonjo-Iweala and former Colombian finance minister Jose Antonio Ocampo. According to Colin Bradford, the presence of several viable candidates—from different parts of the world—for the World Bank presidency means that the entire international community could have a say in selecting the next World Bank president, rather than the U.S. nominee being automatically confirmed. This change in the nominating process, says Bradford, is good for the Bank because it reflects growing demands for representation from emerging economies.
 

Video

• Change World Bank Nominating Process

Event Information

On March 26, the Center for Universal Education at Brookings hosted an expert consultation on the development of the World Bank Group's new Education Strategy. The consultative meeting brought together a small group of experts from diverse fields. The purpose of the discussion was to gather input and suggestions aimed at strengthening the World Bank Group's work in the education sector.

Elizabeth King, Director of Education in the Human Development Network at the World Bank, opened the event by providing an overview of the Bank’s current approach to education, and how it has evolved over the last several decades. She described the Bank’s priorities as reconnecting education to the broader development agenda, supporting more equitable access, ensuring better learning, and strengthening education systems. The Bank’s main operating principals are taking a whole-sector approach, building the evidence base in education, and measuring the results and impact. Beginning with this extensive consultation process, the Bank is demonstrating its willingness to work with others in the development community to build a larger and more robust evidence base from which to draw lessons to improve the quality of limited staff to maximize the impact of Bank activities, to underscore its commitment to partnerships with other organizations and civil society groups, and to move toward improving the measurement of results so as to be able to further improve the Bank’s education programs around the world.

View the event summary »

Event Materials

• 20100326_world_bank_participants

Mercutio: I am hurt. A plague a’ both your houses! I am sped. Is he gone and hath nothing? — Romeo and Juliet, Act 3, scene 1, 90–92

The eurozone crisis, which includes the Greek crisis but is not restricted to it, has undermined the credibility of the EU institutions and left millions of Europeans disillusioned with the European Project. The euro was either introduced too early, or it included countries that should never have been included, or both were true. High rates of inflation left countries in the periphery uncompetitive and the constraint of a single currency removed a key adjustment mechanism. Capital flows allowed this problem to be papered over until the global financial crisis hit.

The leaders of the international institutions, the European Commission, the European Central Bank, and the International Monetary Fund, together with the governments of the stronger economies, were asked to figure out a solution and they emphasized fiscal consolidation, which they made a condition for assistance with heavy debt burdens. The eurozone as a whole has paid the price, with real GDP in the first quarter of 2015 being about 1.5 percent below its peak in the first quarter of 2008, seven years earlier, and with a current unemployment rate of 11 percent. By contrast, the sluggish U.S. recovery looks rocket-powered, with GDP 8.6 percent above its previous peak and an unemployment rate of 5.5 percent.

The burden of the euro crisis has been very unevenly distributed, with Greece facing unemployment of 25 percent and rising, Spain 23 percent, Italy 12 percent, and Ireland 9.7 percent, while German unemployment is 4.7 percent. It is not surprising that so many Europeans are unhappy with their policy leaders who moved too quickly into a currency union and then dealt with the crisis in a way that pushed countries into economic depression. The common currency has been a boon to Germany, with its $287 billion current account surplus, but the bane of the southern periphery. Greece bears considerable culpability for its own problems, having failed to collect taxes or open up an economy full of competitive restrictions, but that does not excuse the policy failures among Europe’s leaders. A plague on both sides in the Greek crisis!

During the Great Moderation, it seemed that the Bretton Woods institutions were losing their usefulness because private markets could provide needed funding. The financial crisis and the global recession that followed it shattered this belief. The IMF did not foresee the crisis, nor was it a central player in dealing with the period of greatest peril from 2007 to 2009. National treasuries, the Federal Reserve, and the European Central Bank were the only institutions that had the resources and the power to deal with the bank failures, the shortage of liquidity, and the freezing up of markets. Still, the IMF became relevant again and played an important role in the euro crisis, although at the cost of sharing the unpopularity of the policy response to that crisis.

China’s new Asian Infrastructure Investment Bank is the result of China’s growing power and influence and the failure of the West, particularly the United States, to come to terms with this seismic shift. The Trans-Pacific Partnership trade negotiations have deliberately excluded China, the largest economy in Asia and largest trading partner in the world. Reform of the governance structure of the World Bank and the IMF has stalled with disproportionate power still held by the United States and Europe. Unsurprisingly, China has decided to exercise its influence in other ways, establishing the new Asian bank and increasing the role of the yuan in international transactions. U.S. policymakers underestimated China’s strength and the willingness of other countries to cooperate with it, and the result has been to reduce the role and influence of the Bretton Woods institutions.

Can the old institutions be reinvented and made more effective? In Europe, the biggest problem is that bad decisions were made by national governments and by the international institutions (although the ECB policies have been generally good). The World Bank and IMF do need to reform their governance, but it may be too late to bring China back into the fold.

Editor's Note: The World Bank’s Independent Evaluation Group (IEG) recently published a self-evaluation of its activities. Besides representing current thinking among evaluation experts at the World Bank, it also more broadly reflects some of the strengths and gaps in the approaches that evaluators use to assess and learn from the performance of the international institutions with which they work. The old question “Quis custodet ipsos custodes?” – loosely translated as “Who evaluates the evaluators?” – remains as relevant as ever. Johannes Linn served as an external peer reviewer of the self-evaluation and provides a bird’s-eye view on the lessons learned.

An Overview of the World Bank’s IEG Self-Evaluation Report

In 2011 the World Bank’s Independent Evaluation Group (IEG) carried out and published a self-evaluation of its activities. The self-evaluation team was led by an internal manager, but involved a respected external evaluation expert as the principal author and also an external peer reviewer.

The IEG self-evaluation follows best professional practices as codified by the Evaluation Cooperation Group (ECG). This group brings together the evaluation offices of seven major multilateral financial institutions in joint efforts designed to enhance evaluation performance and cooperation among their evaluators. One can therefore infer that the approach and focus of the IEG self-evaluation is representative of a broader set of practices that are currently used by the evaluation community of international financial organizations.

At the outset the IEG report states that “IEG is the largest evaluation department among Evaluation Capacity Group (ECG) members and is held in high regard by the international evaluation community. Independent assessments of IEG’s role as an independent evaluation function for the Bank and IFC rated it above the evaluation functions in most other ECG members, international nongovernmental organizations, and transnational corporations and found that IEG follows good practice evaluation principles.”

The self-evaluation report generally confirms this positive assessment. For four out of six areas of its mandate IEG gives itself the second highest rating (“good”) out of six possible rating categories. This includes (a) the professional quality of its evaluations, (b) its reports on how the World Bank’s management follows up on IEG recommendations, (c) cooperation with other evaluation offices, and (d) assistance to borrowing countries in improving their own evaluation capacity. In the area of appraising the World Bank’s self-evaluation and risk management practices, the report offers the third highest rating (“satisfactory”), while it gives the third lowest rating (“modest”) for IEG’s impact on the Bank’s policies, strategies and operations. In addition the self-evaluation concludes that overall the performance of IEG has been “good” and that it operates independently, effectively and efficiently.

The report makes a number of recommendations for improvement, which are likely to be helpful, but have limited impact on its activities. They cover measures to further enhance the independence of IEG and the consistency of evaluation practices as applied across the World Bank Group’s branches – the World Bank, the International Finance Corporation (IFC), and the Multilateral Investment Guarantee Agency (MIGA) –; to improve the design of evaluations and the engagement with Bank management upstream for greater impact; and monitoring the impact of recent organizational changes in IEG in terms of results achieved. The report also recommends that more be done to evaluate the Bank’s analytical work and that evaluations draw on comparative evidence.

Assessment

In terms of the parameters of self-evaluation set by the prevailing practice among the evaluators on international financial agencies, the IEG self-evaluation is accurate and helpful. From my own experience as an operational manager in the Bank whose activities were evaluated by IEG in years past, and as a user of IEG evaluations (and of evaluations of other international aid organizations) for my research on aid effectiveness, I concur that IEG is independent and effective in meeting its mandate as defined. Moreover, the self-evaluation produces useful quantitative evidence (including survey results, budget analysis, etc.) to corroborate qualitative judgments.

However, the self-evaluation suffers from a number of limitations in approach and gaps in focus, which are broadly representative of the practices prevalent among many of the evaluation offices of international aid agencies.

Approach of the IEG self-evaluation

The core of the self-evaluation report is about the evaluation process followed by IEG, with very little said about the substance of IEG’s evaluations. The following questions could have usefully been raised, but were not: do evaluations cover the right issues with the right intensity, such as growth and poverty; environmental, governance, and gender impacts; regional dimensions versus exclusive country or project focus; effectiveness in addressing the problems of fragile and conflict states; effectiveness in dealing with global public goods; sustainability and scaling up; etc. Therefore the report does not deal with the question of whether IEG effectively responds in its evaluations to the many important strategic debates and issues with which the development community is grappling.

Related to this limitation is the fact that the report assessed the quality of IEG’s mostly in terms of (a) whether its approach and processes meet certain standards established by the Evaluation Cooperation Group; and (b) how it is judged by stakeholders in response to a survey commissioned for this evaluation. Both these approaches are useful, but they do not have any basis in professional assessments of the quality of individual products. This is equivalent to IEG evaluating the World Bank’s projects on the quality of its processes (e.g., appraisal and supervision processes) and on the basis of stakeholder surveys, without evaluating individual products and their impacts.

Gaps in the Self-Evaluation and in Evaluation Practice

Careful reading of the report reveals six important gaps in the IEG self-evaluation, in the prevailing evaluation practice in the World Bank, and more generally in the way international financial organizations evaluate their own performance. The first three gaps relate to aspects of the evaluation approach used and the second three gaps relate to lack of focus in the self-evaluation on key internal organizational issues:

1. Impact Evaluations: The report notes that IEG carries out two to three impact evaluations per year, but it sidesteps the debate in the current evaluation literature and practice as to what extent the “gold standard” of randomized impact evaluation should occupy a much more central role. Given the importance of this debate and divergence of views, it would have been appropriate for the self-evaluation to assess IEG’s current practice of very limited use of randomized evaluations.

2. Evaluation of Scaling Up: The report does not address the question of to what extent current IEG practice not only assesses the performance of individual projects in terms of their outcomes and sustainability, but also in terms of whether the Bank has systematically built on its experience in specific projects to help scale up their impact through support for expansion or replication in follow-up operations or through effective hand-off to the government or other partners. In fact, currently IEG does not explicitly and systematically consider scaling up in its project and program evaluations. For example, in a recent IEG evaluation of World Bank funded municipal development projects (MDPs) , IEG found that the Bank has supported multiple MDPs in many countries over the years, but the evaluation did not address the obvious question whether the Bank systematically planned for the project sequence or built on its experience from prior projects in subsequent operations. While most other evaluation offices like IEG do not consider scaling up, some (in particular those of the International Fund for Agricultural Development and the United Nations Development Program) have started doing so in recent years.

3. Drawing on the Experience of and Benchmarking Against Other Institutions: The self-evaluation report does a good job in benchmarking IEG performance in a number of respects against that of other multilateral institutions. In the main text of the report it states that “IEG plans to develop guidelines for approach papers to ensure greater quality, in particular in drawing on comparative information from other sources and benchmarking against other institutions.” This is a welcome intention, but it is inadequately motivated in the rest of the report and not reflected in the Executive Summary. The reality is that IEG, like most multilateral evaluation offices, so far has not systematically drawn on the evaluations and relevant experience of other aid agencies in its evaluations of World Bank performance. This has severely limited the learning impact of the evaluations.

4. Bank Internal Policies, Management Processes and Incentives: IEG evaluations traditionally do not focus on how the Bank’s internal policies, management and incentives affect the quality of Bank engagement in countries. Therefore evaluations cannot offer any insights into whether and how Bank-internal operating modalities contribute to results. Two recent exceptions are notable exceptions. First, the IEG evaluation of the Bank’s approach to harmonization with other donors and alignment with country priorities assesses the incentives for staff to support harmonization and alignment. The evaluation concludes that there are insufficient incentives, a finding disputed by management. Second, is the evaluation of the Bank’s internal matrix management arrangements, which is currently under way. The self-evaluation notes that Bank management tried to quash the matrix evaluation on the grounds that it did not fall under the mandate of IEG. This is an unfortunate argument, since an assessment of the institutional reasons for the Bank’s performance is an essential component of any meaningful evaluation of Bank-supported programs. While making a good case for the specific instance of the matrix evaluation, the self-evaluation report shies away from a more general statement in support of engaging IEG on issues of Bank-internal policies, management processes and incentives. It is notable that IFAD’s Independent Office of Evaluation appears to be more aggressive in this regard: It currently is carrying out a full evaluation of IFAD’s internal efficiency and previous evaluations (e.g., an evaluation of innovation and scaling up) did not shy away from assessing internal institutional dimensions.

5. World Bank Governance: The IEG self-evaluation is even more restrictive in how it interprets its mandate regarding the evaluation of the World Bank’s governance structures and processes (including its approach to members’ voice and vote, the functioning of its board of directors, the selection of its senior management, etc.). It considers these topics beyond IEG’s mandate. This is unfortunate, since the way the Bank’s governance evolves will substantially affect its long-term legitimacy, effectiveness and viability as an international financial institution. Since IEG reports to the Bank’s board of directors, and many of the governance issues involve questions of the board’s composition, role and functioning, there is a valid question of how effectively IEG could carry out such an evaluation. However, it is notable that the IMF’s Independent Evaluation Office, which similarly reports to the IMF board of directors, published a full evaluation of the IMF’s governance in 2008, which effectively addressed many of the right questions.

6. Synergies between World Bank, IFC and MIGA: The self-evaluation report points out that the recent internal reorganization of IEG aimed to assure more effective and consistent evaluations across the three member branches of the World Bank Group. This is welcome, but the report does not assess how past evaluations addressed the question of whether the World Bank, IFC and MIGA effectively capitalized on the potential synergies among the three organizations. The recent evaluation of the World Bank Group’s response to the global economic crisis of 2008/9 provided parallel assessments of each agency’s performance, but did not address whether they work together effectively in maximizing their synergies. The reality is that the three organizations have deeply engrained institutional cultures and generally go their own ways rather than closely coordinating their activities on the ground. Future evaluations should explicitly consider whether the three effectively cooperate or not. While the World Bank is unique in the way it has organizationally separated its private sector and guarantee operations, other aid organizations also have problems of a lack of cooperation, coordination and synergy among different units within the agency. Therefore, the same comment also applies to their evaluation approaches.

Conclusions

Self-evaluations are valuable tools for performance assessment and IEG is to be congratulated for carrying out and publishing such an evaluation of its own activities. As for all self-evaluations, it should be seen as an input to an independent external evaluation, a decision that, for now, has apparently been postponed by the Bank’s board of directors.

IEG’s self-evaluation has many strengths and provides an overall positive assessment of IEG’s work. However, it does reflect some important limitations of analysis and of certain gaps in approach and coverage, which an independent external review should consider explicitly, and which IEG’s management should address. Since many of these issues also likely apply to most of the other evaluation approaches by other evaluation offices, the lessons have relevance beyond IEG and the World Bank.

Key lessons include:

• An evaluation of evaluations should focus not only on process, but also on the substantive issues that the institution is grappling with.
• An evaluation of the effectiveness of evaluations should include a professional assessment of the quality of evaluation products.
• An evaluation of evaluations should assess:
  o How effectively impact evaluations are used;
  o How scaling up of successful interventions is treated;
  o How the experience of other comparable institutions is utilized;
  o Whether and how the internal policies, management practices and incentives of the institution are effectively assessed;
  o Whether and how the governance of the institution is evaluated; and
  o Whether and how internal coordination, cooperation and synergy among units within the organizations are assessed.

Evaluations play an essential role in the accountability and learning of international aid organizations. Hence it is critical that evaluations address the right issues and use appropriate techniques. If the lessons above were reflected in the evaluation practices of the aid institutions, this would represent a significant step forward in the quality, relevance and likely impact of evaluations.

Since its 50th anniversary in 1994, the World Bank has been led by four presidents: Lewis Preston until his untimely death in 1995; then James Wolfensohn, who gave the institution new energy, purpose and legitimacy; followed by Paul Wolfowitz, whose fractious management tossed the World Bank into deep crisis; and most recently, Robert Zoellick, who will be remembered for having stabilized the bank and provided effective leadership during its remarkably swift and strong response to the global financial crisis.

Throughout these years of ups and downs in the bank’s leadership, standing and lending, the overall trend of its global role was downhill. While it remains one of the world’s largest multilateral development finance institutions, its position relative to other multilateral financing mechanisms is now much less prominent. Other multilateral institutions have taken over key roles. For example, the European Union agencies and the regional development banks have rapidly expanded their portfolios, and new “vertical funds” such as the Global Fund for AIDS, Tuberculosis and Malaria have become major funding vehicles. At the same time, according to a 2011 OECD Development Assistance Committee report multilateral aid has declined as a share of total aid. Meanwhile, non-governmental aid flows have dramatically increased, including those from major foundations like the Bill and Melinda Gates Foundation, but also from new internet-based channels bundling small individual donations, such as Global Giving. The World Bank— which 20 years ago was still the biggest and most powerful global development agency and hence a ready target for criticism— today is just one of the many institutions that offer for development to the poor and emerging market economies.

Against this backdrop, the World Bank, its members and Dr. Kim face three options in its long-term trajectory over the next 10 to 20 years: 1) the bank can continue on its current path of gradual decline; 2) it might be radically scaled back and eventually eliminated, as other aid channels take over; or 3) it can dramatically reinvent itself as a global finance institution that bundles resources for growing global needs.

There is no doubt in this author’s mind that the World Bank should remain a key part of the global governance architecture, but that requires that the new president forge an ambitious long-term vision for the bank – something that has been lacking for the last 30 years – and then reform the institution and build the authorizing environment that will make it possible to achieve the vision.

Option 1: “Business as Usual” = Continued Gradual Decline

The first option, reflecting the business-as-usual approach that characterized most of the Zoellick years of leadership will mean that the bank will gradually continue to lose in scope, funding and relevance. Its scope will be reduced since the emerging market economies find the institution insufficiently responsive to their needs. They have seen the regional development banks take on increasing importance, as reflected in the substantially greater capital increases in recent years for some of these institutions than for the World Bank in relative terms (and in the case of the Asian Development Bank, even in absolute terms). And emerging market economies have set up their own thriving regional development banks without participation of the industrial countries, such as the Caja Andina de Fomento (CAF) in Latin America and the Eurasian Development Bank in the former Soviet Union. This trend will be reinforced with the creation of a “South Bank” or “BRIC Bank”, an initiative that is currently well underway.

At the same time, the World Bank’s soft loan window, the International Development Association (IDA), will face less support from industrial countries going through deep fiscal crises, heightened competition from other concessional funds, and a perception of reduced need, as many of the large and formerly poor developing countries graduate to middle-income status. It is significant that for the last IDA replenishment much of the increase in resources was due to its growing reliance on advance repayments made by some of its members and commitments against future repayments, thus in effect mortgaging its future financial capacity. The World Bank’s status as a knowledge leader in development will also continue to be challenged with the rise of research from developing countries and growing think tank capacity, as well as a proliferation of private and official agencies doling out advice and technical assistance.

As a result, under this option, over the next 10 to 20 years the World Bank will likely become no more than a shadow of the preeminent global institution it once was. It will linger on but will not be able to contribute substantially to address any of the major global financial, economic or social challenges in the future.
 
Option 2: “The Perfect Storm” = Breaking Up the World Bank

In 1998, the U.S. Congress established a commission to review and advise on the role of the international financial institutions. In 2000, the commission, led by Professor Allan Meltzer, released its recommendations, which included far-reaching changes for the International Monetary Fund and the World Bank, most of them designed to reduce the scope and financial capacities of these institutions in line with the conservative leanings of the majority of the commission’s members. For the World Bank, the “Meltzer Report” called for much of its loan business and financial assets to be devolved to the regional development banks, in effect ending the life of the institution as we know it. The report garnered some attention when it was first issued, but did not have much impact in the way the institution was run in the following 10 years.

In 2010, the U.S. Senate Foreign Relations Committee released a report on the international financial institutions, which called on them to aim toward “succeeding in their development and economic missions and thereby putting them out of business”. However, it did not recommend a drastic restructuring of the multilateral development banks, and instead argued strongly against any dilution of the U.S. veto right, its lock on leadership selection, and its voting share at the IMF and World Bank. While not dramatic in its short-term impact, these recommendations were likely a strong factor in the subsequent decisions made by the Obama administration to oppose a substantial increase in contributions by emerging markets during the latest round of capital increase at the World Bank to push for an American to replace Robert Zoellick as World Bank president. These actions reinforced for emerging market countries that the World Bank would not change sufficiently and quickly enough to serve their interests, and thus helped create the momentum for setting up a new “South Bank.”

While there seems to be no imminent risk of a break-up of the World Bank along the lines recommended by the Meltzer Report, the combination of fiscal austerity and conservative governments in key industrial countries, compounded by a declining interest of the emerging market countries in sustaining the institution’s future, could create the perfect storm for the bank. Specifically, as governments face constrained fiscal resources, confront the increasing fragmentation of the multilateral aid architecture, and take steps to consolidate their own aid agencies, they might conclude that it would be more efficient and fiscally prudent to rationalize the international development system. There is a obvious overlap on the ground in the day-to-day business of the World Bank and that of the regional development banks. This is a reality which is being fostered by the growing decentralization of the World Bank into regional hubs; in fact, a recent evaluation by the World Bank’s Independent Evaluation Group concluded that “[r]ather than functioning as a global institution, the bank is at risk of evolving into six regional banks”. With the growing financial strength, institutional capacity and dynamism, and the apparently greater legitimacy of regional development banks among their regional members, shareholders might eventually decide that consolidation of the World Bank’s operations with those of the regional development banks, in favor of the latter, is the preferred approach.

There are lots of reasons to think that this drastic step would be difficult to take politically, financially and administratively, and therefore the inertia common to the international governance architecture will also prevail in this case. However, the new World Bank president would be well advised to be prepared for the possibility of a “perfect storm” under which the idea of eviscerating the World Bank could gain some traction,. The more the bank is seen to fade away, as postulated under Option 1 above, the greater is the likelihood that Option 2 would be given serious consideration.

Option 3: “A Different World Bank” = Creating a Stronger Global Institution for the Coming Decades

Despite all the criticism and the decline in its relative role as a development finance institution in recent decades, the World Bank is still one of the strongest and most effective development institutions in a world. According to a recent independent ranking of the principal multilateral and bilateral aid institutions by the Brookings Institution and the Center for Global Development “IDA consistently ranks among the best aid agencies in each dimension of quality”.

A third, radically different option from the first two, would build on this strength and ensure that the world has an institution 10 to 20 years from now which helps the global community and individual countries to respond effectively to the many global challenges which the world will undoubtedly face: continued poverty, hunger, conflict and fragility, major infrastructure and energy needs, education and health challenges, and global warming and environmental challenges. On top of this, global financial crises will likely recur and require institutions like the World Bank to help countries provide safety nets and the structural foundations of long-term growth, as the bank has amply demonstrated since 2008. With this as a broad mandate, how could the World Bank respond under new dynamic?

First, it would change its organizational and operating modalities to take a leaf out of the book of the vertical funds, which have been so successful in tackling major development challenges in a focused and scaled-up manner. This means substantially rebalancing the internal matrix between the regional and country departments on the one hand and the technical departments on the other hand. According to the same evaluation cited above, the World Bank has tipped too far toward short-term country priorities and has failed to adequately reflect the need for long-term, dedicated sectoral engagement. The World Bank needs to fortify its reputation as an institution that can muster the strongest technical expertise, fielding team with broad global experience and with first rate regional and country perspective. This does not imply that the World Bank would abandon its engagement at the country level, but it means that it would systematically support the pursuit of long-term sectoral and sub-sectoral strategies at the country level, linked to regional and global initiatives, and involving private-public partnership to assure that development challenges are addressed at scale and in a sustained manner.

Second, recognizing that all countries have unmet needs for which they need long-term finance and best practice in areas such as infrastructure, energy, climate change and environment, the World Bank could become a truly global development institution by opening up its funding windows to all countries, not just an arbitrarily defined subset of developing countries. This would require substantially revising the current graduation rules and possibly the financial instruments. This would mean that the World Bank becomes the global equivalent of the European Investment Bank (EIB) and of the German Kreditanstalt fuer Wiederaufbau (KfW)—development banks that have successfully supported the infrastructure development of the more advanced countries.

Third, the World Bank would focus its own knowledge management activities and support for research and development in developing countries much more on a search for effective and scalable solutions, linked closely to its operational engagement which would be specifically designed to support the scaling up of tested innovations, along the lines pioneered by the Bill and Melinda Gates Foundation.

Fourth, for those countries with strong project management capacities, the World Bank would dramatically simplify its lending processes, following the example of the EIB. This would make it a much more efficient operational institution, making it a more attractive partner to its borrowing member countries, especially the emerging market economies.

Fifth, the membership of the World Bank would fix some fundamental problems with its financial structure and governance. It would invite the emerging market economies to make significantly larger contributions to its capital base in line with their much-enhanced economic and financial capacities. It would revamp the bank’s voting and voice rules to reflect the changed global economic weights and financial contributions of emerging markets. The bank would also explore, based on the experience of the vertical funds, tapping the resources of non-official partners, such as foundations and the private sector as part of its capital and contribution base. Of course, this would bring with it further significant changes in the governance of the World Bank. And the bank would move swiftly to a transparent selection of its leadership on the basis of merit without reference to nationality.

Conclusion: The New World Bank President Needs to Work with the G-20 Leaders to Chart a Course Forward
 
The new president will have to make a choice between these three options. Undoubtedly, the easiest choice is “business-as-usual”, perhaps embellished with some marginal changes that reflect the perspective and new insights that an outsider will bring. There is no doubt that the forces of institutional and political inertia tend to prevent dramatic change. However, it is also possible that Dr. Kim, with his background in a relatively narrow sectoral area may recognize the need for a more vertical approach in the bank’s organizational and operational model. Therefore, he may be more inclined than others to explore Option 3.

If he pursues Option 3, Dr. Kim will need a lot of help. The best place to look for help might be the G-20 leadership. One could hope that at least some of the leaders of the G-20 understand that Options 1 and 2 are not in the interest of their countries and the international community. Hopefully, they would be willing to push their peers to contemplate some radical changes in the multilateral development architecture. This might involve the setting up of a high-level commission as recently recommended by this author, which would review the future of the World Bank as part of a broader approach to rationalize the multilateral system in the interest of greater efficiency and effectiveness. But in setting up such a commission, the G-20 should state a clear objective, namely that the World Bank, perhaps the strongest existing global development institution, should not be gutted or gradually starved out of existence. Instead, it needs to be remade into a focused, effective and truly global institution. If Dr. Kim embraces this vision and develops actionable ideas for the commission and the G-20 leaders to consider and support, then he may bring the right medicine for an ailing giant.