eva Assessing the impact of foreign assistance: The role of evaluation By webfeeds.brookings.edu Published On :: Wed, 30 Mar 2016 15:00:00 -0400 Event Information March 30, 20163:00 PM - 4:30 PM EDTSaul/Zilkha RoomsBrookings Institution1775 Massachusetts Avenue NWWashington, DC 20036 A conversation with USAID Administrator Gayle SmithOn March 30, Global Economy and Development at Brookings and the Modernizing Foreign Assistance Network (MFAN) hosted Gayle Smith, administrator of U.S. Agency for International Development (USAID) for an address on the fifth anniversary of the USAID policy on evaluation. A principal recommendation of the Presidential Policy Determination on Global Development, signed by President Obama in 2010, was greater accountability for U.S. foreign assistance funds, including evaluation of development programs. In 2011, USAID adopted a formal policy on evaluation and since has average some 200 evaluations a year. Among the issues that will be addressed during the event are the success and challenges in implementing the evaluation policy, the use of alternative evaluation methods, and building a system and process for turning evaluations into learning. Administrator Smith was introduced by Brookings Senior Fellow George Ingram. Following her address, he moderated a panel discussion of Ruth Levine, Wade Warren, and Jodi Nelson. Join the conversation on Twitter using #AIDeval Video Assessing the impact of foreign assistance: The role of evaluation Transcript Uncorrected Transcript (.pdf) Event Materials 20160330_usaid_evaluation_transcript Full Article
eva CVE’s relevance and challenges: Central Asia as surprising snapshot By webfeeds.brookings.edu Published On :: Tue, 07 Jan 2020 20:01:50 +0000 Full Article
eva Prevalence and characteristics of surprise out-of-network bills from professionals in ambulatory surgery centers By webfeeds.brookings.edu Published On :: Thu, 16 Apr 2020 14:33:48 +0000 Full Article
eva Prevalence and characteristics of surprise out-of-network bills from professionals in ambulatory surgery centers By webfeeds.brookings.edu Published On :: Thu, 16 Apr 2020 14:33:48 +0000 Full Article
eva Risk evaluation and mitigation strategies (REMS): Building a framework for effective patient counseling on medication risks and benefits By webfeeds.brookings.edu Published On :: Fri, 24 Jul 2015 08:45:00 -0400 Event Information July 24, 20158:45 AM - 4:15 PM EDTThe Brookings Institution1775 Massachusetts Ave., NWWashington, DC Under the Food and Drug Administration Amendments Act (FDAAA) of 2007, the FDA has the authority to require pharmaceutical manufacturers to develop Risk Evaluation and Mitigation Strategies (REMS) for drugs or biologics that carry serious potential or known risks. Since that time, the REMS program has become an important tool in ensuring that riskier drugs are used safely, and it has allowed FDA to facilitate access to a host of drugs that may not otherwise have been approved. However, concerns have arisen regarding the effects of REMS programs on patient access to products, as well as the undue burden that the requirements place on the health care system. In response to these concerns, FDA has initiated reform efforts aimed at improving the standardization, assessment, and integration of REMS within the health care system. As part of this broader initiative, the agency is pursuing four priority projects, one of which focuses on improving provider-patient benefit-risk counseling for drugs that have a REMS attached. Under a cooperative agreement with FDA, the Center for Health Policy at Brookings held an expert workshop on July 24 titled, “Risk Evaluation and Mitigation Strategies (REMS): Building a Framework for Effective Patient Counseling on Medication Risks and Benefits”. This workshop was the first in a series of convening activities that will seek input from stakeholders across academia, industry, health systems, and patient advocacy groups, among others. Through these activities, Brookings and FDA will further develop and refine an evidence-based framework of best practices and principles that can be used to inform the development and effective use of REMS tools and processes. Event Materials REMS_PBRC_Meeting_AgendaREMS BR Speaker BiosREMS BenefitRisk Meeting SummaryREMS BenefitRisk communication white paper Full Article
eva Optimal solar subsidy policy design and incentive pass-through evaluation: using US California as an example By webfeeds.brookings.edu Published On :: Mon, 04 Jul 2016 14:30:00 -0400 Renewable energy is an important source to tackle against climate change, as the latest IPCC report has pointed out. However, due to the existence of multiple market failures such as negative externalities of fossil fuels and knowledge spillovers of new technology, government subsidies are still needed to develop renewable energy, such as solar photovoltaic (PV) cells. In the United States, there have been various forms of subsidies for PV, varying from the federal level to the state level, and from the city level to the utility level. California, as the pioneer of solar PV development, has put forward the biggest state-level subsidy program for PV, the California Solar Initiative (CSI). The CSI has planned to spend around $2.2 Billion in 2007–2016 to install roughly 2 GW PV capacity, with the average subsidy level as high as $1.1/W. How to evaluate the cost-effectiveness and incentive pass-through of this program are the two major research questions we are pursing. Our cost-effectiveness analysis is based on a constrained optimization model that we developed, where the objective is to install as much PV capacity as possible under a fixed budget constraint. Both the analytical and computational results suggest that due to a strong peer effect and the learning-by-doing effect, one can shift subsides from later periods to early periods so that the final PV installed capacity can be increased by 8.1% (or 32 MW). However, if the decision-maker has other policy objectives or constraints in mind, such as maintaining the policy certainty, then, the optimally calculated subsidy policy would look like the CSI. As to the incentive pass-through question, we took a structural approach and in addition used the method of regression discontinuity (RD). While in general, the incentive pass-through rate depends on the curvature of the demand and supply curve and the level of market competition, our two estimations indicate that the incentive pass-through for the CSI program is almost complete. In other words, almost all of the incentive has been enjoyed by the customer, and the PV installers did not retain much. Based on the RD design, we observe that PV installers tend to consider the CSI incentive as exogenous to their pricing decision. The relative good performance of the CSI in terms of both the cost-effectiveness and the incentive pass-through aspect are tightly related to its policy design and program management. International speaking, the biggest challenge for the design of any PV subsidy program is the quick running out of the budget, and in the end, it looks like customers are rushing for the subsidy. Such rushing behavior is a clear indication of higher-than-needed incentive levels. Due to the policy rigidity and rapid PV technological change, the PV subsidy policy may lag behind the PV cost decline; and as a result, rational customers could rush for any unnecessarily high subsidy. Due to the high uncertainty and unpredictability of future PV costs, the CSI put forward a new design that links the incentive level change and the installed capacity goal fulfillment. Specifically, the CSI has designed nine steps to achieve its policy goal; at each step, there is a PV capacity goal that corresponds to an incentive level. Once the capacity goal is finished, the incentive level will decrease to the next lower level. Furthermore, to maintain the policy certainty, the CSI regulated that every step-wise change in the incentive level should not be higher than $0.45/W, nor smaller than $0.05/W, together with other three constraints. A good subsidy policy not only requires flexible policy design to respond to fast-changing environment, but also demands an efficient program management system, digitalized if possible. For the CSI, the authority has contracted out a third-party to maintain a good database system for the program. Specifically, the database has documented in detail every PV system that customers requested. Key data fields include 22 important dates during the PV installation process, customers’ zip code, city, utility and county information, and various characteristics of the PV system such as price, system size, incentive, PV module and installer. All information is publicly available, which to some extent fills in the information gap held by customers and fosters the market competition among PV installers. For customers to receive the incentive, their PV systems have to pass the inspection of the local government, and also to be interconnected to the grid. On the supply side, the CSI has also certified and created a list of PV installers that every customer can choose from. Although the CSI has ended in 2014 due to fast PV cost reduction starting from 2009, its experience has been transferred to other areas in the United States and in Europe. It is highly possible that other similar new technologies and products (e.g. the electric car and the battery) can adopt the CSI policy design, too. In summary, a good and successful policy may need to be simply, clear, credible, foreseeable, flexible, end-able, and incentive-compatible. The PV subsidy policy in China still has a long way to go when compared to the CSI. Authors Changgui Dong Full Article
eva A tale of two trade fairs: Milwaukee’s globally relevant water proposition By webfeeds.brookings.edu Published On :: Wed, 27 Jul 2016 13:47:00 +0000 As we have previously discussed, the decision to prioritize a single primary cluster in a regional economic development plan is challenging. For Milwaukee, this was especially difficult in development of its global trade and investment plan because it has three legitimate clusters: energy, power and controls; food and beverage; and water technologies. The team developing the plan was reluctant to pick a favorite. Full Article Uncategorized
eva Evaluating the Evaluators: Some Lessons from a Recent World Bank Self-Evaluation By webfeeds.brookings.edu Published On :: Tue, 21 Feb 2012 14:15:00 -0500 Editor's Note: The World Bank’s Independent Evaluation Group (IEG) recently published a self-evaluation of its activities. Besides representing current thinking among evaluation experts at the World Bank, it also more broadly reflects some of the strengths and gaps in the approaches that evaluators use to assess and learn from the performance of the international institutions with which they work. The old question “Quis custodet ipsos custodes?” – loosely translated as “Who evaluates the evaluators?” – remains as relevant as ever. Johannes Linn served as an external peer reviewer of the self-evaluation and provides a bird’s-eye view on the lessons learned. An Overview of the World Bank’s IEG Self-Evaluation Report In 2011 the World Bank’s Independent Evaluation Group (IEG) carried out and published a self-evaluation of its activities. The self-evaluation team was led by an internal manager, but involved a respected external evaluation expert as the principal author and also an external peer reviewer. The IEG self-evaluation follows best professional practices as codified by the Evaluation Cooperation Group (ECG). This group brings together the evaluation offices of seven major multilateral financial institutions in joint efforts designed to enhance evaluation performance and cooperation among their evaluators. One can therefore infer that the approach and focus of the IEG self-evaluation is representative of a broader set of practices that are currently used by the evaluation community of international financial organizations. At the outset the IEG report states that “IEG is the largest evaluation department among Evaluation Capacity Group (ECG) members and is held in high regard by the international evaluation community. Independent assessments of IEG’s role as an independent evaluation function for the Bank and IFC rated it above the evaluation functions in most other ECG members, international nongovernmental organizations, and transnational corporations and found that IEG follows good practice evaluation principles.” The self-evaluation report generally confirms this positive assessment. For four out of six areas of its mandate IEG gives itself the second highest rating (“good”) out of six possible rating categories. This includes (a) the professional quality of its evaluations, (b) its reports on how the World Bank’s management follows up on IEG recommendations, (c) cooperation with other evaluation offices, and (d) assistance to borrowing countries in improving their own evaluation capacity. In the area of appraising the World Bank’s self-evaluation and risk management practices, the report offers the third highest rating (“satisfactory”), while it gives the third lowest rating (“modest”) for IEG’s impact on the Bank’s policies, strategies and operations. In addition the self-evaluation concludes that overall the performance of IEG has been “good” and that it operates independently, effectively and efficiently. The report makes a number of recommendations for improvement, which are likely to be helpful, but have limited impact on its activities. They cover measures to further enhance the independence of IEG and the consistency of evaluation practices as applied across the World Bank Group’s branches – the World Bank, the International Finance Corporation (IFC), and the Multilateral Investment Guarantee Agency (MIGA) –; to improve the design of evaluations and the engagement with Bank management upstream for greater impact; and monitoring the impact of recent organizational changes in IEG in terms of results achieved. The report also recommends that more be done to evaluate the Bank’s analytical work and that evaluations draw on comparative evidence. Assessment In terms of the parameters of self-evaluation set by the prevailing practice among the evaluators on international financial agencies, the IEG self-evaluation is accurate and helpful. From my own experience as an operational manager in the Bank whose activities were evaluated by IEG in years past, and as a user of IEG evaluations (and of evaluations of other international aid organizations) for my research on aid effectiveness, I concur that IEG is independent and effective in meeting its mandate as defined. Moreover, the self-evaluation produces useful quantitative evidence (including survey results, budget analysis, etc.) to corroborate qualitative judgments. However, the self-evaluation suffers from a number of limitations in approach and gaps in focus, which are broadly representative of the practices prevalent among many of the evaluation offices of international aid agencies. Approach of the IEG self-evaluation The core of the self-evaluation report is about the evaluation process followed by IEG, with very little said about the substance of IEG’s evaluations. The following questions could have usefully been raised, but were not: do evaluations cover the right issues with the right intensity, such as growth and poverty; environmental, governance, and gender impacts; regional dimensions versus exclusive country or project focus; effectiveness in addressing the problems of fragile and conflict states; effectiveness in dealing with global public goods; sustainability and scaling up; etc. Therefore the report does not deal with the question of whether IEG effectively responds in its evaluations to the many important strategic debates and issues with which the development community is grappling. Related to this limitation is the fact that the report assessed the quality of IEG’s mostly in terms of (a) whether its approach and processes meet certain standards established by the Evaluation Cooperation Group; and (b) how it is judged by stakeholders in response to a survey commissioned for this evaluation. Both these approaches are useful, but they do not have any basis in professional assessments of the quality of individual products. This is equivalent to IEG evaluating the World Bank’s projects on the quality of its processes (e.g., appraisal and supervision processes) and on the basis of stakeholder surveys, without evaluating individual products and their impacts. Gaps in the Self-Evaluation and in Evaluation Practice Careful reading of the report reveals six important gaps in the IEG self-evaluation, in the prevailing evaluation practice in the World Bank, and more generally in the way international financial organizations evaluate their own performance. The first three gaps relate to aspects of the evaluation approach used and the second three gaps relate to lack of focus in the self-evaluation on key internal organizational issues: 1. Impact Evaluations: The report notes that IEG carries out two to three impact evaluations per year, but it sidesteps the debate in the current evaluation literature and practice as to what extent the “gold standard” of randomized impact evaluation should occupy a much more central role. Given the importance of this debate and divergence of views, it would have been appropriate for the self-evaluation to assess IEG’s current practice of very limited use of randomized evaluations. 2. Evaluation of Scaling Up: The report does not address the question of to what extent current IEG practice not only assesses the performance of individual projects in terms of their outcomes and sustainability, but also in terms of whether the Bank has systematically built on its experience in specific projects to help scale up their impact through support for expansion or replication in follow-up operations or through effective hand-off to the government or other partners. In fact, currently IEG does not explicitly and systematically consider scaling up in its project and program evaluations. For example, in a recent IEG evaluation of World Bank funded municipal development projects (MDPs) , IEG found that the Bank has supported multiple MDPs in many countries over the years, but the evaluation did not address the obvious question whether the Bank systematically planned for the project sequence or built on its experience from prior projects in subsequent operations. While most other evaluation offices like IEG do not consider scaling up, some (in particular those of the International Fund for Agricultural Development and the United Nations Development Program) have started doing so in recent years. 3. Drawing on the Experience of and Benchmarking Against Other Institutions: The self-evaluation report does a good job in benchmarking IEG performance in a number of respects against that of other multilateral institutions. In the main text of the report it states that “IEG plans to develop guidelines for approach papers to ensure greater quality, in particular in drawing on comparative information from other sources and benchmarking against other institutions.” This is a welcome intention, but it is inadequately motivated in the rest of the report and not reflected in the Executive Summary. The reality is that IEG, like most multilateral evaluation offices, so far has not systematically drawn on the evaluations and relevant experience of other aid agencies in its evaluations of World Bank performance. This has severely limited the learning impact of the evaluations. 4. Bank Internal Policies, Management Processes and Incentives: IEG evaluations traditionally do not focus on how the Bank’s internal policies, management and incentives affect the quality of Bank engagement in countries. Therefore evaluations cannot offer any insights into whether and how Bank-internal operating modalities contribute to results. Two recent exceptions are notable exceptions. First, the IEG evaluation of the Bank’s approach to harmonization with other donors and alignment with country priorities assesses the incentives for staff to support harmonization and alignment. The evaluation concludes that there are insufficient incentives, a finding disputed by management. Second, is the evaluation of the Bank’s internal matrix management arrangements, which is currently under way. The self-evaluation notes that Bank management tried to quash the matrix evaluation on the grounds that it did not fall under the mandate of IEG. This is an unfortunate argument, since an assessment of the institutional reasons for the Bank’s performance is an essential component of any meaningful evaluation of Bank-supported programs. While making a good case for the specific instance of the matrix evaluation, the self-evaluation report shies away from a more general statement in support of engaging IEG on issues of Bank-internal policies, management processes and incentives. It is notable that IFAD’s Independent Office of Evaluation appears to be more aggressive in this regard: It currently is carrying out a full evaluation of IFAD’s internal efficiency and previous evaluations (e.g., an evaluation of innovation and scaling up) did not shy away from assessing internal institutional dimensions. 5. World Bank Governance: The IEG self-evaluation is even more restrictive in how it interprets its mandate regarding the evaluation of the World Bank’s governance structures and processes (including its approach to members’ voice and vote, the functioning of its board of directors, the selection of its senior management, etc.). It considers these topics beyond IEG’s mandate. This is unfortunate, since the way the Bank’s governance evolves will substantially affect its long-term legitimacy, effectiveness and viability as an international financial institution. Since IEG reports to the Bank’s board of directors, and many of the governance issues involve questions of the board’s composition, role and functioning, there is a valid question of how effectively IEG could carry out such an evaluation. However, it is notable that the IMF’s Independent Evaluation Office, which similarly reports to the IMF board of directors, published a full evaluation of the IMF’s governance in 2008, which effectively addressed many of the right questions. 6. Synergies between World Bank, IFC and MIGA: The self-evaluation report points out that the recent internal reorganization of IEG aimed to assure more effective and consistent evaluations across the three member branches of the World Bank Group. This is welcome, but the report does not assess how past evaluations addressed the question of whether the World Bank, IFC and MIGA effectively capitalized on the potential synergies among the three organizations. The recent evaluation of the World Bank Group’s response to the global economic crisis of 2008/9 provided parallel assessments of each agency’s performance, but did not address whether they work together effectively in maximizing their synergies. The reality is that the three organizations have deeply engrained institutional cultures and generally go their own ways rather than closely coordinating their activities on the ground. Future evaluations should explicitly consider whether the three effectively cooperate or not. While the World Bank is unique in the way it has organizationally separated its private sector and guarantee operations, other aid organizations also have problems of a lack of cooperation, coordination and synergy among different units within the agency. Therefore, the same comment also applies to their evaluation approaches. Conclusions Self-evaluations are valuable tools for performance assessment and IEG is to be congratulated for carrying out and publishing such an evaluation of its own activities. As for all self-evaluations, it should be seen as an input to an independent external evaluation, a decision that, for now, has apparently been postponed by the Bank’s board of directors. IEG’s self-evaluation has many strengths and provides an overall positive assessment of IEG’s work. However, it does reflect some important limitations of analysis and of certain gaps in approach and coverage, which an independent external review should consider explicitly, and which IEG’s management should address. Since many of these issues also likely apply to most of the other evaluation approaches by other evaluation offices, the lessons have relevance beyond IEG and the World Bank. Key lessons include: An evaluation of evaluations should focus not only on process, but also on the substantive issues that the institution is grappling with. An evaluation of the effectiveness of evaluations should include a professional assessment of the quality of evaluation products. An evaluation of evaluations should assess: o How effectively impact evaluations are used; o How scaling up of successful interventions is treated; o How the experience of other comparable institutions is utilized; o Whether and how the internal policies, management practices and incentives of the institution are effectively assessed; o Whether and how the governance of the institution is evaluated; and o Whether and how internal coordination, cooperation and synergy among units within the organizations are assessed. Evaluations play an essential role in the accountability and learning of international aid organizations. Hence it is critical that evaluations address the right issues and use appropriate techniques. If the lessons above were reflected in the evaluation practices of the aid institutions, this would represent a significant step forward in the quality, relevance and likely impact of evaluations. Authors Johannes F. Linn Image Source: © Christian Hartmann / Reuters Full Article
eva 40 years later- The relevance of Okun’s "Equality and Efficiency: The Big Tradeoff" By webfeeds.brookings.edu Published On :: Mon, 04 May 2015 10:30:00 -0400 Event Information May 4, 201510:30 AM - 12:00 PM EDTFalk AuditoriumBrookings Falk Auditorium1775 Massachusetts Ave., NWWashington, DC 20036 Register for the EventForty years after its initial publication, Equality and Efficiency: The Big Tradeoff remains an influential work from one of the most important macroeconomists over the last century, Arthur M. Okun (1928-1980). Okun’s theory on market economies reminds readers of an engaging dual theme: the market needs a place, and the market needs to be kept in its place. Articulated in a way that remains relevant even during today’s discussions on broadening gaps in income inequality, Okun emphasized that institutions in a capitalist democracy prod us to get ahead of our neighbors economically after telling us to stay in line socially. On May 4, The Brookings Institution Press re-released Okun’s classic work with a new foreword from Former Treasury Secretary Lawrence H. Summers, in addition to “Further Thoughts on Equality and Efficiency,” a paper published by Okun in 1977. The event included opening remarks from Brookings Senior Fellow George Perry, with a keynote address from Larry Summers. Following these remarks, David Wessel moderated a panel discussion with former Chair of the Council of Economic Advisers Greg Mankiw, Economic Studies’ Melissa Kearney and Justin Wolfers, and Washington Center for Equitable Growth's Heather Boushey regarding the history and impact of Okun’s work. Download a copy of Lawrence Summers' opening remarks. Ted Gayer, Vice President and Director of Economic Studies and Joseph Pechman Senior Fellow, reads Lawrence Summers's opening remarks. David Wessel (right), Director of the Hutchins Center on Fiscal and Monetary Policy, moderates a panel discussion with N. Gregory Mankiw, Melissa Kearney, and Heather Boushey. Janet Yellen, Chair of the Board of Governors of the Federal Reserve System, listens to the discussion from the audience. To Yellen's right is former Congressional Budget Office director, Doug Elmendorf. Video 40 years later- The relevance of Okun’s "Equality and Efficiency: The Big Tradeoff" Audio 40 years later- The relevance of Okun’s "Equality and Efficiency: The Big Tradeoff" Transcript Uncorrected Transcript (.pdf) Event Materials 20150504_okun_equality_efficiency_transcript050415 Summers Okun Speech Full Article
eva Electing a president: The significance of Nevada By webfeeds.brookings.edu Published On :: Tue, 18 Feb 2020 15:47:00 +0000 In establishing the first states to vote in the Democratic presidential nomination campaign, the party selected four states representing each U.S. region. These events are almost like a preseason before the big contests in March such as Super Tuesday when California and Texas cast ballots. The four early states that select delegates in February start… Full Article
eva Limits on Nevada’s legislature keep it from serving the state By webfeeds.brookings.edu Published On :: Fri, 14 Jun 2019 11:00:57 +0000 In the last 30 years, Nevada has evolved from a sparsely and homogenously populated rural outpost to one of the most urban and diverse states in the country. Nevada’s population is now majority-minority. The Las Vegas-Henderson-Paradise Metropolitan statistical area with over 2.2 million residents is the 28th largest in the country and is home to… Full Article
eva Electing a president: The significance of Nevada By webfeeds.brookings.edu Published On :: Tue, 18 Feb 2020 15:47:00 +0000 In establishing the first states to vote in the Democratic presidential nomination campaign, the party selected four states representing each U.S. region. These events are almost like a preseason before the big contests in March such as Super Tuesday when California and Texas cast ballots. The four early states that select delegates in February start… Full Article
eva A systematic review of systems dynamics and agent-based obesity models: Evaluating obesity as part of the global syndemic By webfeeds.brookings.edu Published On :: Fri, 19 Jul 2019 13:02:35 +0000 Full Article
eva A systematic review of systems dynamics and agent-based obesity models: Evaluating obesity as part of the global syndemic By webfeeds.brookings.edu Published On :: Fri, 19 Jul 2019 13:02:35 +0000 Full Article
eva Evaluating the Evaluators: Some Lessons from a Recent World Bank Self-Evaluation By webfeeds.brookings.edu Published On :: Tue, 21 Feb 2012 14:15:00 -0500 Editor's Note: The World Bank’s Independent Evaluation Group (IEG) recently published a self-evaluation of its activities. Besides representing current thinking among evaluation experts at the World Bank, it also more broadly reflects some of the strengths and gaps in the approaches that evaluators use to assess and learn from the performance of the international institutions with which they work. The old question “Quis custodet ipsos custodes?” – loosely translated as “Who evaluates the evaluators?” – remains as relevant as ever. Johannes Linn served as an external peer reviewer of the self-evaluation and provides a bird’s-eye view on the lessons learned. An Overview of the World Bank’s IEG Self-Evaluation Report In 2011 the World Bank’s Independent Evaluation Group (IEG) carried out and published a self-evaluation of its activities. The self-evaluation team was led by an internal manager, but involved a respected external evaluation expert as the principal author and also an external peer reviewer. The IEG self-evaluation follows best professional practices as codified by the Evaluation Cooperation Group (ECG). This group brings together the evaluation offices of seven major multilateral financial institutions in joint efforts designed to enhance evaluation performance and cooperation among their evaluators. One can therefore infer that the approach and focus of the IEG self-evaluation is representative of a broader set of practices that are currently used by the evaluation community of international financial organizations. At the outset the IEG report states that “IEG is the largest evaluation department among Evaluation Capacity Group (ECG) members and is held in high regard by the international evaluation community. Independent assessments of IEG’s role as an independent evaluation function for the Bank and IFC rated it above the evaluation functions in most other ECG members, international nongovernmental organizations, and transnational corporations and found that IEG follows good practice evaluation principles.” The self-evaluation report generally confirms this positive assessment. For four out of six areas of its mandate IEG gives itself the second highest rating (“good”) out of six possible rating categories. This includes (a) the professional quality of its evaluations, (b) its reports on how the World Bank’s management follows up on IEG recommendations, (c) cooperation with other evaluation offices, and (d) assistance to borrowing countries in improving their own evaluation capacity. In the area of appraising the World Bank’s self-evaluation and risk management practices, the report offers the third highest rating (“satisfactory”), while it gives the third lowest rating (“modest”) for IEG’s impact on the Bank’s policies, strategies and operations. In addition the self-evaluation concludes that overall the performance of IEG has been “good” and that it operates independently, effectively and efficiently. The report makes a number of recommendations for improvement, which are likely to be helpful, but have limited impact on its activities. They cover measures to further enhance the independence of IEG and the consistency of evaluation practices as applied across the World Bank Group’s branches – the World Bank, the International Finance Corporation (IFC), and the Multilateral Investment Guarantee Agency (MIGA) –; to improve the design of evaluations and the engagement with Bank management upstream for greater impact; and monitoring the impact of recent organizational changes in IEG in terms of results achieved. The report also recommends that more be done to evaluate the Bank’s analytical work and that evaluations draw on comparative evidence. Assessment In terms of the parameters of self-evaluation set by the prevailing practice among the evaluators on international financial agencies, the IEG self-evaluation is accurate and helpful. From my own experience as an operational manager in the Bank whose activities were evaluated by IEG in years past, and as a user of IEG evaluations (and of evaluations of other international aid organizations) for my research on aid effectiveness, I concur that IEG is independent and effective in meeting its mandate as defined. Moreover, the self-evaluation produces useful quantitative evidence (including survey results, budget analysis, etc.) to corroborate qualitative judgments. However, the self-evaluation suffers from a number of limitations in approach and gaps in focus, which are broadly representative of the practices prevalent among many of the evaluation offices of international aid agencies. Approach of the IEG self-evaluation The core of the self-evaluation report is about the evaluation process followed by IEG, with very little said about the substance of IEG’s evaluations. The following questions could have usefully been raised, but were not: do evaluations cover the right issues with the right intensity, such as growth and poverty; environmental, governance, and gender impacts; regional dimensions versus exclusive country or project focus; effectiveness in addressing the problems of fragile and conflict states; effectiveness in dealing with global public goods; sustainability and scaling up; etc. Therefore the report does not deal with the question of whether IEG effectively responds in its evaluations to the many important strategic debates and issues with which the development community is grappling. Related to this limitation is the fact that the report assessed the quality of IEG’s mostly in terms of (a) whether its approach and processes meet certain standards established by the Evaluation Cooperation Group; and (b) how it is judged by stakeholders in response to a survey commissioned for this evaluation. Both these approaches are useful, but they do not have any basis in professional assessments of the quality of individual products. This is equivalent to IEG evaluating the World Bank’s projects on the quality of its processes (e.g., appraisal and supervision processes) and on the basis of stakeholder surveys, without evaluating individual products and their impacts. Gaps in the Self-Evaluation and in Evaluation Practice Careful reading of the report reveals six important gaps in the IEG self-evaluation, in the prevailing evaluation practice in the World Bank, and more generally in the way international financial organizations evaluate their own performance. The first three gaps relate to aspects of the evaluation approach used and the second three gaps relate to lack of focus in the self-evaluation on key internal organizational issues: 1. Impact Evaluations: The report notes that IEG carries out two to three impact evaluations per year, but it sidesteps the debate in the current evaluation literature and practice as to what extent the “gold standard” of randomized impact evaluation should occupy a much more central role. Given the importance of this debate and divergence of views, it would have been appropriate for the self-evaluation to assess IEG’s current practice of very limited use of randomized evaluations. 2. Evaluation of Scaling Up: The report does not address the question of to what extent current IEG practice not only assesses the performance of individual projects in terms of their outcomes and sustainability, but also in terms of whether the Bank has systematically built on its experience in specific projects to help scale up their impact through support for expansion or replication in follow-up operations or through effective hand-off to the government or other partners. In fact, currently IEG does not explicitly and systematically consider scaling up in its project and program evaluations. For example, in a recent IEG evaluation of World Bank funded municipal development projects (MDPs) , IEG found that the Bank has supported multiple MDPs in many countries over the years, but the evaluation did not address the obvious question whether the Bank systematically planned for the project sequence or built on its experience from prior projects in subsequent operations. While most other evaluation offices like IEG do not consider scaling up, some (in particular those of the International Fund for Agricultural Development and the United Nations Development Program) have started doing so in recent years. 3. Drawing on the Experience of and Benchmarking Against Other Institutions: The self-evaluation report does a good job in benchmarking IEG performance in a number of respects against that of other multilateral institutions. In the main text of the report it states that “IEG plans to develop guidelines for approach papers to ensure greater quality, in particular in drawing on comparative information from other sources and benchmarking against other institutions.” This is a welcome intention, but it is inadequately motivated in the rest of the report and not reflected in the Executive Summary. The reality is that IEG, like most multilateral evaluation offices, so far has not systematically drawn on the evaluations and relevant experience of other aid agencies in its evaluations of World Bank performance. This has severely limited the learning impact of the evaluations. 4. Bank Internal Policies, Management Processes and Incentives: IEG evaluations traditionally do not focus on how the Bank’s internal policies, management and incentives affect the quality of Bank engagement in countries. Therefore evaluations cannot offer any insights into whether and how Bank-internal operating modalities contribute to results. Two recent exceptions are notable exceptions. First, the IEG evaluation of the Bank’s approach to harmonization with other donors and alignment with country priorities assesses the incentives for staff to support harmonization and alignment. The evaluation concludes that there are insufficient incentives, a finding disputed by management. Second, is the evaluation of the Bank’s internal matrix management arrangements, which is currently under way. The self-evaluation notes that Bank management tried to quash the matrix evaluation on the grounds that it did not fall under the mandate of IEG. This is an unfortunate argument, since an assessment of the institutional reasons for the Bank’s performance is an essential component of any meaningful evaluation of Bank-supported programs. While making a good case for the specific instance of the matrix evaluation, the self-evaluation report shies away from a more general statement in support of engaging IEG on issues of Bank-internal policies, management processes and incentives. It is notable that IFAD’s Independent Office of Evaluation appears to be more aggressive in this regard: It currently is carrying out a full evaluation of IFAD’s internal efficiency and previous evaluations (e.g., an evaluation of innovation and scaling up) did not shy away from assessing internal institutional dimensions. 5. World Bank Governance: The IEG self-evaluation is even more restrictive in how it interprets its mandate regarding the evaluation of the World Bank’s governance structures and processes (including its approach to members’ voice and vote, the functioning of its board of directors, the selection of its senior management, etc.). It considers these topics beyond IEG’s mandate. This is unfortunate, since the way the Bank’s governance evolves will substantially affect its long-term legitimacy, effectiveness and viability as an international financial institution. Since IEG reports to the Bank’s board of directors, and many of the governance issues involve questions of the board’s composition, role and functioning, there is a valid question of how effectively IEG could carry out such an evaluation. However, it is notable that the IMF’s Independent Evaluation Office, which similarly reports to the IMF board of directors, published a full evaluation of the IMF’s governance in 2008, which effectively addressed many of the right questions. 6. Synergies between World Bank, IFC and MIGA: The self-evaluation report points out that the recent internal reorganization of IEG aimed to assure more effective and consistent evaluations across the three member branches of the World Bank Group. This is welcome, but the report does not assess how past evaluations addressed the question of whether the World Bank, IFC and MIGA effectively capitalized on the potential synergies among the three organizations. The recent evaluation of the World Bank Group’s response to the global economic crisis of 2008/9 provided parallel assessments of each agency’s performance, but did not address whether they work together effectively in maximizing their synergies. The reality is that the three organizations have deeply engrained institutional cultures and generally go their own ways rather than closely coordinating their activities on the ground. Future evaluations should explicitly consider whether the three effectively cooperate or not. While the World Bank is unique in the way it has organizationally separated its private sector and guarantee operations, other aid organizations also have problems of a lack of cooperation, coordination and synergy among different units within the agency. Therefore, the same comment also applies to their evaluation approaches. Conclusions Self-evaluations are valuable tools for performance assessment and IEG is to be congratulated for carrying out and publishing such an evaluation of its own activities. As for all self-evaluations, it should be seen as an input to an independent external evaluation, a decision that, for now, has apparently been postponed by the Bank’s board of directors. IEG’s self-evaluation has many strengths and provides an overall positive assessment of IEG’s work. However, it does reflect some important limitations of analysis and of certain gaps in approach and coverage, which an independent external review should consider explicitly, and which IEG’s management should address. Since many of these issues also likely apply to most of the other evaluation approaches by other evaluation offices, the lessons have relevance beyond IEG and the World Bank. Key lessons include: An evaluation of evaluations should focus not only on process, but also on the substantive issues that the institution is grappling with. An evaluation of the effectiveness of evaluations should include a professional assessment of the quality of evaluation products. An evaluation of evaluations should assess: o How effectively impact evaluations are used; o How scaling up of successful interventions is treated; o How the experience of other comparable institutions is utilized; o Whether and how the internal policies, management practices and incentives of the institution are effectively assessed; o Whether and how the governance of the institution is evaluated; and o Whether and how internal coordination, cooperation and synergy among units within the organizations are assessed. Evaluations play an essential role in the accountability and learning of international aid organizations. Hence it is critical that evaluations address the right issues and use appropriate techniques. If the lessons above were reflected in the evaluation practices of the aid institutions, this would represent a significant step forward in the quality, relevance and likely impact of evaluations. Authors Johannes F. Linn Image Source: © Christian Hartmann / Reuters Full Article
eva Wretched Excess or the future of housing design? Another look at the car elevator By www.treehugger.com Published On :: Thu, 06 Jun 2013 10:00:00 -0400 There is a perverse logic to this idea of bringing your car to your apartment. Full Article Design
eva Wretched Excess or Clever Design? Apartment tower with car elevators is definitely the former. By www.treehugger.com Published On :: Mon, 15 Dec 2014 15:45:38 -0500 Two years ago we couldn't decide, but when you see it in action the answer is obvious. Full Article Design
eva Elevated bike lanes will abound in Utopia By www.treehugger.com Published On :: Mon, 01 Jun 2015 08:44:57 -0400 An annual Toronto event is pretty utopian, as two major highways are closed to cars and given over to cyclists. Full Article Design
eva Wallclimbers: an earlier attempt at people-powered elevators By www.treehugger.com Published On :: Mon, 10 Jul 2017 12:56:23 -0400 Cartoonist and inventor Steven M. Johnson has us climbing the walls. Full Article Design
eva Second Forest House is an elevated treehouse with a lighter impact By www.treehugger.com Published On :: Mon, 10 Sep 2018 15:01:25 -0400 Built with a lighter impact in mind, this small mountain retreat is nestled among the trees in Vietnam. Full Article Design
eva Illegal Logging by Pakistan's Timber Mafia Increased Flooding Devastation By www.treehugger.com Published On :: Wed, 18 Aug 2010 13:11:00 -0400 Forget for the moment about to what degree climate change has influenced the flooding in Pakistan. A new article in China Dialogue brings to light a new angle on one very aggravating factor on the overwhelming devastation: Illegal logging by the 'timber Full Article Business
eva World's First Solar-Geothermal Hybrid Plant Opens in the Nevada Desert By www.treehugger.com Published On :: Tue, 17 Jul 2012 05:00:00 -0400 A recently opened power plant in the Nevada desert uses two types of renewable energy. Full Article Technology
eva Neither Pessimism Nor Optimism Is Relevant for Greens, But Determination Is By www.treehugger.com Published On :: Thu, 30 Aug 2012 06:11:51 -0400 Futurist Guy Dauncey acknowledges that the stakes are high and the challenges are enormous. Yet defeatism is not an option. Full Article Business
eva EvanEco goes to the Carnival, Monbiot on Biochar, Accidental Eco Maniac Highlights Women + Water, and More By www.treehugger.com Published On :: Thu, 26 Mar 2009 10:13:14 -0400 EvanEco: Carnival of the Green 172 by Don Bosch "Got milk? Grab a glass of the white stuff and a chocolate chip cookie, and join me for some green bloggy goodness..."Guardian Ethical Living: Ethical Superstore and Natural Collection merge into giant Full Article Living
eva 200 evangelical scientists ask Congress to pass climate legislation By www.treehugger.com Published On :: Mon, 22 Jul 2013 15:56:29 -0400 Two hundred evangelical scientists recently wrote a letter to Congress making the religious case for passing climate change legislation. Full Article Living
eva Evangelical Christians call on Obama to protect public lands from oil and gas leasing By www.treehugger.com Published On :: Tue, 22 Oct 2013 10:26:41 -0400 Rev. Richard Cizik and the nonprofit New Evangelical Partnership for the Common Good have released a video message to President Obama calling on him to do more to protect public lands from destruction by private industry. Full Article Energy
eva ELEVATE puts a solar powered, green walled, rainwater collecting tiny house on a pedestal By www.treehugger.com Published On :: Wed, 05 Aug 2015 13:17:18 -0400 Hawaiian engineers think that it can address many of the world's problems. Are they too ambitious? Full Article Design
eva BMW proposes elevated e-bike cycleways to ease congestion & speed commutes By www.treehugger.com Published On :: Wed, 22 Nov 2017 15:45:24 -0500 Instead of banning cars from city centers, this automaker recommends elevated roads for electric bikes & scooters to ease congestion and emissions. Full Article Transportation
eva Google Earth's revamp immerses users in scenic places around the world By www.treehugger.com Published On :: Wed, 19 Apr 2017 11:07:45 -0400 The overhaul now includes the Voyager feature for deep exploration, not just satellite views. Full Article Technology
eva Wildlife is thriving in Fukushima evacuation zones (video) By www.treehugger.com Published On :: Mon, 06 Jan 2020 15:54:13 -0500 A lesson in what happens when the humans disappear. Full Article Science
eva Yeti robot scans polar ice to warn researchers of dangerous crevasses By www.treehugger.com Published On :: Mon, 02 Sep 2013 07:00:00 -0400 The autonomous robot allows scientists to more safely study polar enviroments and also collects valuable information on ice conditions for climate research. Full Article Technology
eva Man's $15K tiny house has DIY elevator bed & heated ceiling (Video) By www.treehugger.com Published On :: Wed, 09 Jan 2019 13:36:44 -0500 This young man's mortgage-free home includes an interesting self-built version of the rare species known as the heated ceiling. Full Article Design
eva Brilliant tiny house features $500 DIY elevator bed built with free plans (Video) By www.treehugger.com Published On :: Tue, 14 Feb 2017 07:00:00 -0500 This rustic, modern tiny home features lots of smart space-saving ideas, plus a self-built bed that can lift up and down automatically. Full Article Design
eva New Lessons From Old Buildings: Bin Laden's Medieval Hideout By www.treehugger.com Published On :: Wed, 04 May 2011 08:41:26 -0400 I write often about the lessons that one can learn from old buildings, usually discussing ventilation and lighting. It turns out that there are lessons in security and defence as well; Eli Lehrer of Frum Forum notices some Full Article Design
eva Devastating Earthquake in Eastern Turkey Fails to Shake Ankara's Nuclear Plans By www.treehugger.com Published On :: Sat, 29 Oct 2011 08:00:00 -0400 Reports that Turkey would reconsider its nuclear plans following a deadly earthquake in the eastern part of the country sounded too good to be true. They were. Full Article Energy
eva Developed-Developing Nation Split on Climate Obligations Needs Reevaluation By www.treehugger.com Published On :: Thu, 10 May 2012 11:10:00 -0400 We need a new system to determine which nations ought to be forced to cut emissions, not the 20-year old outdated one we've got now. Full Article Business
eva Natural air conditioner cools with the power of terracotta and evaporation By www.treehugger.com Published On :: Fri, 04 Apr 2014 07:00:00 -0400 Using an age-old method to cool the air, this terracotta "cold pot" could be an energy-efficient and low-maintenance alternative to conventional air conditioners. Full Article Design
eva These dogs may save the citrus industry from a devastating pandemic By www.treehugger.com Published On :: Tue, 04 Feb 2020 18:57:02 -0500 Researchers find that dogs can be trained to sniff out the bacteria that causes citrus greening, with 99+ percent accuracy Full Article Science
eva Can TWIN elevators make buildings more energy efficient? By www.treehugger.com Published On :: Mon, 25 Apr 2016 14:11:23 -0400 Thyssenkrupp's TWIN elevators have a very unique feature that could save buildings energy and space. Full Article Business
eva Urban tiny house's versatile lounge comes with an elevator bed (Video) By www.treehugger.com Published On :: Tue, 17 Apr 2018 13:18:43 -0400 This well-crafted tiny dwelling has its own space-saving elevator bed that can go up and down with the push of a button. Full Article Design
eva Devastating Spruce Beetle outbreak in Colorado linked to drought, climate change By www.treehugger.com Published On :: Thu, 10 Oct 2013 17:38:14 -0400 The Mountain Pine Beetle is getting more press because of the havoc it is causing in the Southern Rocky Mountains, but the Spruce Beetle has the potential to be equally or even more devastating in Colorado. Full Article Science
eva Elevator bed & folding porch grace this tiny house builder's home (Video) By www.treehugger.com Published On :: Wed, 26 Dec 2018 16:50:31 -0500 This tiny house builder and architect just built himself and his wife a bigger (but still tiny) home, equipped with a space-saving elevator bed. Full Article Design
eva "Comfy Chic" checklist can elevate your mood and wardrobe during isolation By www.treehugger.com Published On :: Mon, 20 Apr 2020 07:00:00 -0400 Better yet, Citizenne Style's tips don't require you to make any new online purchases. Full Article Living
eva The future of transportation is the bus, the bike and the elevator By www.treehugger.com Published On :: Thu, 31 Oct 2019 09:36:01 -0400 Henry Grabar writes a really wonderful article that shows how "a better world is possible." Full Article Design
eva UN evaluates principles to ensure safe and healthy workplaces By www.treehugger.com Published On :: Tue, 25 Sep 2018 04:15:00 -0400 It is estimated that one worker dies every 15 seconds from toxic exposures at work Full Article Business
eva Sophisticated Cyber-Attacks Prompt Corporate Boards To Elevate Information Security To The C-Suite - Video OneTitle By feedproxy.google.com Published On :: 10 Jun 2014 12:00:00 EDT SOPHISTICATED CYBER-ATTACKS PROMPT CORPORATE BOARDS TO ELEVATE INFORMATION SECURITY TO THE C-SUITE Full Article Banking Financial Services Computer Electronics High Tech Security Workforce Management Human Resources New Products Services Broadcast Feed Announcements MultiVu Video
eva Michelle Obama, Dr. Jill Biden, Mark Kelly And LeVar Burton Host Story Time Event For Elementary School Students - Joining Forces Event at Fort Leonard Wood, MO By feedproxy.google.com Published On :: 06 May 2016 14:35:00 EDT Joining Forces Event at Fort Leonard Wood, MO Full Article Books Education Television Children-related News Broadcast Feed Announcements Veterans MultiVu Video
eva TELEMUNDO REVELA INNOVADORA CAMPAÑA PARA LA NUEVA TEMPORADA DE “EL SEÑOR DE LOS CIELOS” QUE SE ESTRENARÁ EL LUNES 28 DE MARZO A LAS 10PM/9C - EPK By feedproxy.google.com Published On :: 21 Mar 2016 16:10:00 EDT EPK Full Article Entertainment Television New Products Services Broadcast Feed Announcements
eva Michelle Obama, Dr. Jill Biden, Mark Kelly And LeVar Burton Host Story Time Event For Elementary School Students - Joining Forces Event at Fort Leonard Wood, MO By feedproxy.google.com Published On :: 06 May 2016 14:35:00 EDT Joining Forces Event at Fort Leonard Wood, MO Full Article Books Education Television Children-related News Broadcast Feed Announcements Veterans MultiVu Video
eva La OEM, la FEMA, y el Ad Council dan inicio a la undécima versión anual del Mes Nacional de la Preparación en la ciudad de Nueva York - Waiting- NYC Spanish :30 By feedproxy.google.com Published On :: 05 Sep 2014 18:25:00 EDT Asegúrese que su familia tenga un punto de reunión para encontrarse en casos de emergencia. Visite N-Y-C punto gov diagonal ready N-Y o marque 311. Full Article Publicidad Editoriales Servicios de información Aviso de Contenido para Radio TV Responsabilidad Social Corporativa Seguridad Pública Nueva York