"If it ain't broken, don't fix it."Sound advice, but limited to situations where "fixing it" only entails restoring past performance. In contrast, innovations entail substantive improvements over the past. Innovations are not just corrections of past mistakes, but progress towards a better future.

However, innovations often present a challenging dilemma to decision makers. Many decisions require choosing between options, one of which is both potentially better in the outcome but markedly more uncertain. In these situations the decision maker faces an "innovation dilemma."

The innovation dilemma arises in many contexts. Here are a few examples.

Technology. New and innovative technologies are often advocated because of their purported improvements on existing products or methods. However, what is new is usually less well-known and less widely tested than what is old. The range of possible adverse (or favorable) surprises of an innovative technology may exceed the range of surprise for a tried-and-true technology. The analyst who must choose between innovation and convention faces an innovation dilemma.

Investment. The economic investor faces an innovation dilemma when choosing between investing in a promising but unknown new start-up and investing in a well-known existing firm.

Auction. "Nothing ventured, nothing gained" is the motto of the risk-taker, while the risk-avoider responds: "Nothing ventured, nothing lost". The innovation dilemma is embedded in the choice between these two strategies. Consider for example the "winner's curse" in auction theory. You can make a financial bid for a valuable piece of property, which will be sold to the highest bidder. You have limited information about the other bidders and about the true value of the property. If you bid high you might win the auction but you might also pay more than the property is worth. Not bidding is risk-free because it avoids the purchase. The choice between a high bid and no bid is an innovation dilemma.

Employer decision. An employer must decide whether or not to replace a current satisfactory employee with a new candidate whose score on a standardized test was high. A high score reflects great ability. However, the score also contains a random element, so a high score may result from chance, and not reflect true ability. The innovation dilemma is embedded in the employer's choice between the current adequate employee and a high-scoring new candidate.

Natural resource exploitation. Permitting the extraction of offshore petroleum resources may be productive in terms of petroleum yield but may also present officials with significant uncertainty about environmental consequences.

Public health. Implementation of a large-scale immunization program may present policy officials with worries about uncertain side effects.

Agricultural policy. New technologies promise improved production efficiency or new consumer choices, but with uncertain benefits and costs and potential unanticipated adverse effects resulting from use of manufactured inputs such as fertilizers, pesticides, and machinery, and, more recently, genetically engineered seed varieties and information technology. (I am indebted to L. Joe Moffitt and Craig Osteen for these examples in natural resources, public health and agriculture.)

An essay like this one should - according to custom - end with a practical prescription: What to do about the innovation dilemma? You need to make a decision - a choice between options - and you face an innovation dilemma. How to choose? All I'll say is that the first step is to identify what you need to achieve from this decision. Recognizing the vast uncertainties which accompany the decision, choose the option which achieves the required outcome over the largest range of uncertain contingencies.

If you want more of an answer than that, consult your favorite decision theory (like info-gap theory, for instance).

I will conclude by drawing a parallel between the innovation dilemma and one of the oldest quandaries in political philosophy. In The Evolution of Political Thought C. Northcote Parkinson explains the historically recurring tension between freedom and equality.

Freedom. People have widely varying interests and aptitudes. Hence a society that offers broad freedom for individuals to exploit their abilities, will also develop a wide spread of wealth, accomplishment, and status. Freedom enables individuals to explore, invent, discover, and create. Freedom is the recipe for innovation. Freedom induces both uncertainty and inequality.

Equality. People have widely varying interests and aptitudes. Hence a society that strives for equality among its members can achieve this by enforcing conformity and by transferring wealth from rich to poor. The promise of a measure of equality is a guarantee of a measure of security, a personal and social safety net. Equality reduces both uncertainty and freedom.

The dilemma is that a life without freedom is hardly human, but freedom without security is the jungle. And life in the jungle, as Hobbs explained, in "solitary, poor, nasty, brutish and short".

Decisions are made in order to achieve desirable outcomes. An innovation dilemma arises when a seemingly more attractive option is also more uncertain than other options. In this essay we explore the relation between the innovation dilemma and the robustness of a decision, and the relation between robustness and probability. A decision is robust to uncertainty if it achieves required outcomes despite adverse surprises. A robust decision may differ from the seemingly best option. Furthermore, robust decisions are not based on knowledge of probabilities, but can still be the most likely to succeed.

Imagine a squirrel nibbling acorns under an oak tree. They're pretty good acorns, though a bit dry. The good ones have already been taken. Over in the distance is a large stand of fine oaks. The acorns there are probably better. But then, other squirrels can also see those trees, and predators can too. The squirrel doesn't need to get fat, but a critical caloric intake is necessary before moving on to other activities. How long should the squirrel forage at this patch before moving to the more promising patch, if at all?

Imagine a hedge fund manager investing in South African diamonds, Australian Uranium, Norwegian Kroners and Singapore semi-conductors. The returns have been steady and good, but not very exciting. A new hi-tech start-up venture has just turned up. It looks promising, has solid backing, and could be very interesting. The manager doesn't need to earn boundless returns, but it is necessary to earn at least a tad more than the competition (who are also prowling around). How long should the manager hold the current portfolio before changing at least some of its components?

These are decision problems, and like many other examples, they share three traits: critical needs must be met; the current situation may or may not be adequate; other alternatives look much better but are much more uncertain. To change, or not to change? What strategy to use in making a decision? What choice is the best bet? Betting is a surprising concept, as we have seen before; can we bet without knowing probabilities?

The decision is easy in either of two extreme situations, and their analysis will reveal general conclusions.

One extreme is that the status quo is clearly insufficient. For the squirrel this means that these crinkled rotten acorns won't fill anybody's belly even if one nibbled here all day long. Survival requires trying the other patch regardless of the fact that there may be many other squirrels already there and predators just waiting to swoop down. Similarly, for the hedge fund manager, if other funds are making fantastic profits, then something has to change or the competition will attract all the business.

The other extreme is that the status quo is just fine, thank you. For the squirrel, just a little more nibbling and these acorns will get us through the night, so why run over to unfamiliar oak trees? For the hedge fund manager, profits are better than those of any credible competitor, so uncertain change is not called for.

From these two extremes we draw an important general conclusion: the right answer depends on what you need. To change, or not to change, depends on what is critical for survival. There is no universal answer, like, "Always try to improve" or "If it's working, don't fix it". This is a very general property of decisions under uncertainty, and we will call it preference reversal. The agent's preference between alternatives depends on what the agent needs in order to "survive".

The decision strategy that we have described is attuned to the needs of the agent. The strategy attempts to satisfy the agent's critical requirements. If the status quo would reliably do that, then stay put; if not, then move. Following the work of Nobel Laureate Herbert Simon, we will call this a satisficing decision strategy: one which satisfies a critical requirement.

"Prediction is always difficult, especially of the future." - Robert Storm Petersen

Now let's consider a different decision strategy that squirrels and hedge fund managers might be tempted to use. The agent has obtained information about the two alternatives by signals from the environment. (The squirrel sees grand verdant oaks in the distance, the fund manager hears of a new start up.) Given this information, a prediction can be made (though the squirrel may make this prediction based on instincts and without being aware of making it). Given the best available information, the agent predicts which alternative would yield the better outcome. Using this prediction, the decision strategy is to choose the alternative whose predicted outcome is best. We will call this decision strategy best-model optimization. Note that this decision strategy yields a single universal answer to the question facing the agent. This strategy uses the best information to find the choice that - if that information is correct - will yield the best outcome. Best-model optimization (usually) gives a single "best" decision, unlike the satisficing strategy that returns different answers depending on the agent's needs.

There is an attractive logic - and even perhaps a moral imperative - to use the best information to make the best choice. One should always try to do one's best. But the catch in the argument for best-model optimization is that the best information may actually be grievously wrong. Those fine oak trees might be swarming with insects who've devoured the acorns. Best-model optimization ignores the agent's central dilemma: stay with the relatively well known but modest alternative, or go for the more promising but more uncertain alternative.

"Tsk, tsk, tsk" says our hedge fund manager. "My information already accounts for the uncertainty. I have used a probabilistic asset pricing model to predict the likelihood that my profits will beat the competition for each of the two alternatives."

Probabilistic asset pricing models are good to have. And the squirrel similarly has evolved instincts that reflect likelihoods. But a best-probabilistic-model optimization is simply one type of best-model optimization, and is subject to the same vulnerability to error. The world is full of surprises. The probability functions that are used are quite likely wrong, especially in predicting the rare events that the manager is most concerned to avoid.

Now we come to the truly amazing part of the story. The satisficing strategy does not use any probabilistic information. Nonetheless, in many situations, the satisficing strategy is actually a better bet (or at least not a worse bet), probabilistically speaking, than any other strategy, including best-probabilistic-model optimization. We have no probabilistic information in these situations, but we can still maximize the probability of success (though we won't know the value of this maximum).

When the satisficing decision strategy is the best bet, this is, in part, because it is more robust to uncertainty than another other strategy. A decision is robust to uncertainty if it achieves required outcomes even if adverse surprises occur. In many important situations (though not invariably), more robustness to uncertainty is equivalent to being more likely to succeed or survive. When this is true we say that robustness is a proxy for probability.

A thorough analysis of the proxy property is rather technical. However, we can understand the gist of the idea by considering a simple special case.

Let's continue with the squirrel and hedge fund examples. Suppose we are completely confident about the future value (in calories or dollars) of not making any change (staying put). In contrast, the future value of moving is apparently better though uncertain. If staying put would satisfy our critical requirement, then we are absolutely certain of survival if we do not change. Staying put is completely robust to surprises so the probability of success equals 1 if we stay put, regardless of what happens with the other option. Likewise, if staying put would not satisfy our critical requirement, then we are absolutely certain of failure if we do not change; the probability of success equals 0 if we stay, and moving cannot be worse. Regardless of what probability distribution describes future outcomes if we move, we can always choose the option whose likelihood of success is greater (or at least not worse). This is because staying put is either sure to succeed or sure to fail, and we know which.

This argument can be extended to the more realistic case where the outcome of staying put is uncertain and the outcome of moving, while seemingly better than staying, is much more uncertain. The agent can know which option is more robust to uncertainty, without having to know probability distributions. This implies, in many situations, that the agent can choose the option that is a better bet for survival.

The skillful decision maker not only knows a lot, but is also able to deal with conflicting information. We have discussed the innovation dilemma: When choosing between two alternatives, the seemingly better one is also more uncertain.

Animals, people, organizations and societies have developed mechanisms for dealing with the innovation dilemma. The response hinges on tuning the decision to the agent's needs, and robustifying the choice against uncertainty. This choice may or may not coincide with the putative best choice. But what seems best depends on the available - though uncertain - information.

The commendable tendency to do one's best - and to demand the same of others - can lead to putatively optimal decisions that may be more vulnerable to surprise than other decisions that would have been satisfactory. In contrast, the strategy of robustly satisfying critical needs can be a better bet for survival. Consider the design of critical infrastructure: flood protection, nuclear power, communication networks, and so on. The design of such systems is based on vast knowledge and understanding, but also confronts bewildering uncertainties and endless surprises. We must continue to improve our knowledge and understanding, while also improving our ability to manage the uncertainties resulting from the expanding horizon of our efforts. We must identify the critical goals and seek responses that are immune to surprise. 

Often the biggest challenge for older cities and close-in suburbs is not a lack of affordable housing but a need to grow, hold, and attract middle-income households and to foster mixed-income neighborhoods. This creates a policy dilemma: While federal policymakers target limited federal housing assistance to persons with the greatest needs, doing so can create concentrations of poverty within already challenged cities and suburbs. This approach also can set limits that hinder efforts to create the middle-income and mixed-income areas needed for revitalization in older communities.

The metro program hosts and participates in a variety of public forums. To view a complete list of these events, please visit the metro program's Research and Commentary page which provides copies of major speeches, PowerPoint presentations, event transcripts, and event summaries.

Downloads

• Download

      

       

       

What a way to start the new year. Decades of Saudi-Iranian tensions reached a new high this past week. The cycle of reactions to Riyadh’s execution of prominent Shiite cleric Nimr al-Nimr on January 2 is a reminder of how the Saudis, and their Iranian rivals, have viewed and used sectarianism throughout the tumultuous period since 2011.

Al-Nimr was arrested in 2012 and subsequently sentenced to death for allegedly "seeking ‘foreign meddling’ in Saudi Arabia, ‘disobeying’ its rulers and taking up arms against the security forces." The arrest was meant not merely as a signal to Tehran, but at least as much to Saudi Arabia’s own Shiite minority. Shiites comprise as much as 20 percent of the Saudi population, and are concentrated in the oil-rich Eastern Province—and the community has regularly erupted in protests against its economic and political marginalization. In 2011, amid the Arab Spring uprisings in majority-Shiite Bahrain, Saudi Shiites also demonstrated for the release of long-held prisoners, and Saudi forces shot and killed several Shia in the streets.

Riyadh’s decision to carry out the death sentence was greeted with demonstrations in Iran and attacks on Saudi diplomatic facilities. This Iranian reaction must have been calculated, as al-Nimr has been on “death row” for a very long time. In response, Saudi Arabia quickly cut ties with its longtime geopolitical foe and urged fellow Sunni governments to follow suit. So far, Bahrain and Sudan have also cut off relations, and both Qatar and the UAE have downgraded them. 

Governments on both sides of the Sunni-Shiite divide found a sectarian narrative useful in rallying their populations and in justifying their actions in response to the 2011 Arab uprisings. The sectarian narrative has helped the parties in this larger regional power struggle mobilize support by playing up the sectarian dimension of protests in Bahrain, the Assad regime’s crackdown in Syria, and the breakdown of inclusive politics in Iraq. Likewise, many Sunni-led countries have found sectarian rhetoric an effective way to rally Sunni citizens, intimidate their own Shiite populations, and to justify crackdowns on dissent. 

Governments on both sides of the Sunni-Shiite divide found a sectarian narrative useful in rallying their populations and in justifying their actions in response to the 2011 Arab uprisings.

Last April, I wrote that Iran was likely to escalate its asymmetric efforts to destabilize Arab politics by exploiting the cracks within Arab societies. They have done so, and it is a form of escalation the Saudis are ill-equipped to match. Last summer, I suggested that the Sunni Arab states could defend best against this Iranian subversion by tamping down sectarian tensions and working to heal the rifts within their own societies through inclusive political and economic policies. So far, I have not seen much effort from the Arab Gulf states in that direction—instead, they have doubled down on divisive sectarianism in Yemen and elsewhere. As this escalatory spiral advances, civilians will pay the price. 

Some are portraying the decision to execute al-Nimr as a negative Saudi response to Iranian efforts at rapprochement over the last few weeks. I do not necessarily see it that way, because the Iranians have done as much as the Gulf Cooperation Council (GCC) states to provoke and exploit tensions between the two in recent times. That notwithstanding, there is no question this execution will inflame sectarian tensions in the Gulf and Iraq, as well as present the Islamic State with new opportunities. 

It has been clear for some time that the U.S. focus on the threat from the so-called Islamic State is simply not matched by the Saudis, who are far more concerned about Iran and Shiite expansionism than by this violent extremist Sunni group in their neighborhood. As such, the execution and ensuing crisis brings the clash of U.S. and Saudi interests into sharp relief and has the potential to become an inflection point in regional affairs – not necessarily because of the way the Saudi and Iranian governments choose to play, but because of how others might react.

For example, Iraqi Prime Minister Haider al-Abadi quickly and publicly condemned the execution. The execution—and the inevitable crackdown on Shiite protests in Qatif—might increase pressure on Abadi from Shiites in Iraq (and from Iran) to demonstrate sectarian preferences in his rhetoric and policy. That could prevent him from moving forward on steps Washington has been pushing to bring Iraqi Sunnis back into the political fold. This easily could threaten the anti-Islamic State campaign in Iraq, since it relies on Sunnis in Ramadi, Mosul, and elsewhere turning away from Islamic State and back toward the Iraqi state. Iraqi counterterrorism forces have taken much of Ramadi, but they cannot hold it without local Sunni support.

Increased Islamic State influence in the Arabian Peninsula would certainly challenge the Saudi government and prompt a renewed securitization of domestic policy.

The Islamic State worked hard to stoke sectarian tensions within the Gulf states over the past year, carrying out attacks on Shiite mosques in Saudi Arabia and Kuwait. The GCC leaders were not drawn in at that stage, instead expressing solidarity with their Shiite compatriots. But this time, a Sunni Gulf government is taking steps that exacerbate sectarian tensions—and that could very easily push the Islamic State to take up the issue again by attempting more such attacks. Increased Islamic State influence in the Arabian Peninsula would certainly challenge the Saudi government and prompt a renewed securitization of domestic policy. It would be an ironic outcome of a Saudi move—47 executions, mostly of Sunni extremists—that was intended to deter ISIS sympathizers. At a moment when low oil prices and a tightened financial future constrain their capacity to coopt a large, underemployed, youthful populace, this is not a recipe for stability.

The possibility that ISIS will gain from this crisis illustrates the problem with governments self-interestedly wielding that sectarian narrative is that it becomes a self-fulfilling prophecy, and it actually increases the incentive on both sides of the sectarian divide to escalate their real power competition, both directly and through proxies. Today, that narrative of sectarian conflict is far more than rhetoric in Iraq and Syria, where a true intercommunal conflict is underway. 

More immediately, the ripple effects of al-Nimr’s execution spotlight American policy dilemmas in the region. The escalation in sectarian conflict threatens the nascent Syrian peace process. It increases the Islamic State’s scope for action there, threatens the political dimension of the anti-Islamic State strategy in Iraq, and incentivizes Sunni extremism in the Arabian Peninsula. It pushes the Yemen war further from resolution as well, leaving al-Qaida in the Arabian Peninsula (AQAP) with room to grow and plan attacks against the American homeland. And it puts the United States into a very tight spot as it continues diplomatic dialogue with Iran in the wake of the nuclear agreement. Given this beginning, 2016 looks to be an even tougher year for the United States in the Middle East than 2015.

      

Today on Toothpaste For Dinner: the americans dilemma