Keeping with tradition, we start the year with a compendium of forecasts for 2016 from our guest bloggers and ourselves.  At the end of the year, we will assess how we did (for last year’s forecasting performance, click here).

The prevailing sentiment about economic developments during 2016 is decidedly mixed. There are positive and negative views, sometimes from the same source. Here is a sampling:

On the negative side, “emerging economies will continue to disappoint;” “ODA will be squeezed by refugee costs (and climate change financing commitments);” “geopolitical tensions will remain;” “the dollar will be stronger with a severe impact on emerging economies;” and a range of idiosyncratic, political risks: weak governance and terrorist threats in Kenya; declining investor confidence and rising social strife in South Africa; corruption scandals in Brazil; and low oil prices coupled with domestic and geopolitical tensions in Russia.

On the positive side, “oil prices will remain low;” “the Islamic State will be defeated;” “the effect of monetary policy normalization will be very limited;” “food prices will remain low or fall, helping reduce global hunger;” “African countries will improve cereal yields;” “OECD countries will accept a record number of refugees and migrants;” “oil exporters will reform their economies;” and “peace agreements to end the wars in Syria, Libya and Yemen will be signed.”

An emerging theme is whether the disappointments in developing country growth in 2015 stem from idiosyncratic factors in specific countries—especially the BRICS, Turkey, and Indonesia—or whether those idiosyncratic factors, often associated with domestic political developments, are symptomatic of a broader issue of a slowing down of global convergence. Indeed, this theme of whether convergence remains a strong force that will continue to dominate developing country prospects, or a weak force that is all too easily offset by other factors, will likely remain one of the critical unknowns of 2016.

In summary, it is fair to say that with views as diverse as those we received, the picture for 2016 is complicated to say the least.

There is no analytical clarity in the global economy, despite forecasts from most major organizations (e.g., the IMF) that growth will be better in 2016 than in 2015 in every region except perhaps East Asia (although Asia will still probably record higher growth than anywhere else).

The fears generated by a slowing of one of the main engines of the global economy over the past decade, namely China, are palpable. The big story of 2016 is perhaps that it is an emerging economy, China, which is the major source of uncertainty over this year’s global outlook. While prospects for the major advanced economies—the USA, Europe, and Japan—are relatively stable, it is the developing world where there is the least clarity over the short- term outlook. Certainly, the volatility in global stock markets in the first days of the year suggests that volatility, risk aversion, and differences of views over short-term developments are all high as 2016 begins.

But there is at least one bright note. Almost certainly, prospects will improve for almost 200 million people who were living in countries that last year remained outside the scope of a normally functioning global economy. In Myanmar, Argentina, Venezuela, Cuba, and Iran, economic conditions will improve as a result of recent political developments. In addition, in 2016 there will probably be at least 100 million more people joining the global middle class—those living in households with incomes of $10-100 a day (2005 PPP). Good news for them but a reminder that the task of moving towards a world with sustainable consumption and production patterns remains huge.

There was one consensus thread among our bloggers—all the Europeans appear consumed by the Euro 2016 soccer event (“Spain, France, or Germany will win”), while only one blogger dared to comment on the Olympics (that Brazil would do twice as well as in 2012). It seems that sports will be less complicated than economics in 2016.

Abstract

Most pollsters base their election projections off questions of voter intentions, which ask “If the election were held today, who would you vote for?” By contrast, we probe the value of questions probing voters’ expectations, which typically ask: “Regardless of who you plan to vote for, who do you think will win the upcoming election?” We demonstrate that polls of voter expectations consistently yield more accurate forecasts than polls of voter intentions. A small-scale structural model reveals that this is because we are polling from a broader information set, and voters respond as if they had polled twenty of their friends. This model also provides a rational interpretation for why respondents’ forecasts are correlated with their expectations. We also show that we can use expectations polls to extract accurate election forecasts even from extremely skewed samples.

I. Introduction

Since the advent of scientific polling in the 1930s, political pollsters have asked people whom they intend to vote for; occasionally, they have also asked who they think will win. Our task in this paper is long overdue: we ask which of these questions yields more accurate forecasts. That is, we evaluate the predictive power of the questions probing voters’ intentions with questions probing their expectations. Judging by the attention paid by pollsters, the press, and campaigns, the conventional wisdom appears to be that polls of voters’ intentions are more accurate than polls of their expectations.

Yet there are good reasons to believe that asking about expectations yields more greater insight. Survey respondents may possess much more information about the upcoming political race than that probed by the voting intention question. At a minimum, they know their own current voting intention, so the information set feeding into their expectations will be at least as rich as that captured by the voting intention question. Beyond this, they may also have information about the current voting intentions—both the preferred candidate and probability of voting—of their friends and family. So too, they have some sense of the likelihood that today’s expressed intention will be changed before it ultimately becomes an election-day vote. Our research is motivated by idea that the richer information embedded in these expectations data may yield more accurate forecasts.

We find robust evidence that polls probing voters’ expectations yield more accurate predictions of election outcomes than the usual questions asking about who they intend to vote for. By comparing the performance of these two questions only when they are asked of the exact same people in exactly the same survey, we effectively difference out the influence of all other factors. Our primary dataset consists of all the state-level electoral presidential college races from 1952 to 2008, where both the intention and expectation question are asked. In the 77 cases in which the intention and expectation question predict different candidates, the expectation question picks the winner 60 times, while the intention question only picked the winner 17 times. That is, 78% of the time that these two approaches disagree, the expectation data was correct. We can also assess the relative accuracy of the two methods by assessing the extent to which each can be informative in forecasting the final vote share; we find that relying on voters’ expectations rather than their intentions yield substantial and statistically significant increases in forecasting accuracy. An optimally-weighted average puts over 90% weight on the expectations-based forecasts. Once one knows the results of a poll of voters expectations, there is very little additional information left in the usual polls of voting intentions. Our findings remain robust to correcting for an array of known biases in voter intentions data.

The better performance of forecasts based on asking voters about their expectations rather than their intentions, varies somewhat, depending on the specific context. The expectations question performs particularly well when: voters are embedded in heterogeneous (and thus, informative) social networks; when they don’t rely too much on common information; when small samples are involved (when the extra information elicited by asking about intentions counters the large sampling error in polls of intentions); and at a point in the electoral cycle when voters are sufficiently engaged as to know what their friends and family are thinking.

Our findings also speak to several existing strands of research within election forecasting. A literature has emerged documenting that prediction markets tend to yield more accurate forecasts than polls (Wolfers and Zitzewitz, 2004; Berg, Nelson and Rietz, 2008). More recently, Rothschild (2009) has updated these findings in light of the 2008 Presidential and Senate races, showing that forecasts based on prediction markets yielded systematically more accurate forecasts of the likelihood of Obama winning each state than did the forecasts based on aggregated intention polls compiled by Nate Silver for the website FiveThirtyEight.com. One hypothesis for this superior performance is that because prediction markets ask traders to bet on outcomes, they effectively ask a different question, eliciting the expectations rather than intentions of participants. If correct, this suggests that much of the accuracy of prediction markets could be obtained simply by polling voters on their expectations, rather than intentions.

These results also speak to the possibility of producing useful forecasts from non-representative samples (Robinson, 1937), an issue of renewed significance in the era of expensive-to-reach cellphones and cheap online survey panels. Surveys of voting intentions depend critically on being able to poll representative cross-sections of the electorate. By contrast, we find that surveys of voter expectations can still be quite accurate, even when drawn from non-representative samples. The logic of this claim comes from the difference between asking about expectations, which may not systematically differ across demographic groups, and asking about intentions, which clearly do. Again, the connection to prediction markets is useful, as Berg and Rietz (2006) show that prediction markets have yielded accurate forecasts, despite drawing from an unrepresentative pool of overwhelmingly white, male, highly educated, high income, self-selected traders.

While questions probing voters’ expectations have been virtually ignored by political forecasters, they have received some interest from psychologists. In particular, Granberg and Brent (1983) document wishful thinking, in which people’s expectation about the likely outcome is positively correlated with what they want to happen. Thus, people who intend to vote Republican are also more likely to predict a Republican victory. This same correlation is also consistent with voters preferring the candidate they think will win, as in bandwagon effects, or gaining utility from being optimistic. We re-interpret this correlation through a rational lens, in which the respondents know their own voting intention with certainty and have knowledge about the voting intentions of their friends and family.

Our alternative approach to political forecasting also provides a new narrative of the ebb and flow of campaigns, which should inform ongoing political science research about which events really matter. For instance, through the 2004 campaign, polls of voter intentions suggested a volatile electorate as George W. Bush and John Kerry swapped the lead several times. By contrast, polls of voters’ expectations consistently showed the Bush was expected to win re-election. Likewise in 2008, despite volatility in the polls of voters’ intentions, Obama was expected to win in all of the last 17 expectations polls taken over the final months of the campaign. And in the 2012 Republican primary, polls of voters intentions at different points showed Mitt Romney trailing Donald Trump, then Rick Perry, then Herman Cain, then Newt Gingrich and then Rick Santorum, while polls of expectations showed him consistently as the likely winner.

We believe that our findings provide tantalizing hints that similar methods could be useful in other forecasting domains. Market researchers ask variants of the voter intention question in an array of contexts, asking questions that elicit your preference for one product, over another. Likewise, indices of consumer confidence are partly based on the stated purchasing intentions of consumers, rather than their expectations about the purchase conditions for their community. The same insight that motivated our study—that people also have information on the plans of others—is also likely relevant in these other contexts. Thus, it seems plausible that survey research in many other domains may also benefit from paying greater attention to people’s expectations than to their intentions.

The rest of this paper proceeds as follows, In Section II, we describe our first cut of the data, illustrating the relative success of the two approaches to predicting the winner of elections. In Sections III and IV, we focus on evaluating their respective forecasts of the two-party vote share. Initially, in Section III we provide what we call naïve forecasts, which follow current practice by major pollsters; in Section IV we product statistically efficient forecasts, taking account of the insights of sophisticated modern political scientists. Section V provides out-of-sample forecasts based on the 2008 election. Section VI extends the assessment to a secondary data source which required substantial archival research to compile. In Section VII, we provide a small structural model which helps explain the higher degree of accuracy obtained from surveys of voter expectations. Section VIII characterizes the type of information that is reflected in voters’ expectation, arguing that it is largely idiosyncratic, rather than the sort of common information that might come from the mass media. Section IX assesses why it is that people’s expectations are correlated with their intentions. Section VI uses this model to show how we can obtain surprisingly accurate expectation-based forecasts with non-representative samples. We then conclude. To be clear about the structure of the argument: In the first part of the paper (through section IV) we simply present two alternative forecasting technologies and evaluate them, showing that expectations-based forecasts outperform those based on traditional intentions-based polls. We present these data without taking a strong position on why. But then in later sections we turn to trying to assess what explains this better performance. Because this assessment is model-based, our explanations are necessarily based on auxiliary assumptions (which we spell out).

Right now, we begin with our simplest and most transparent comparison of the forecasting ability of our two competing approaches.

Download the full paper » (PDF)

Downloads

• Forecasting Elections: Voter Intentions versus Expectations

Editor's Note: A new academic study by David Rothschild and Justin Wolfers concludes that poll questions about expectations—which ask people whom they think will win—have historically been better guides to the outcome of presidential elections than traditional questions about people’s preferences. David Leonhardt of The New York Times conducted an interview with Wolfers by e-mail, focusing on the implications of the study for current presidential polls.

David Leonhardt:In the article, I discussed only briefly the expectations polls about the 2012 race, and some of the Twitter feedback was eager for more. By my count, there have been five recent major polls asking people whom they expect to win — by ABC/Washington Post, Gallup, Politico/George Washington University, New York Times/CBS News, and the University of Connecticut. There is also sixth from Rand asking people the percentage chances they place on each candidate winning. How consistent are the polls?

Justin Wolfers: There’s a striking consistency in how people are responding to these polls. The most recent data are from the Gallup poll conducted Oct. 27-28, and they found 54 percent of adults expect Obama to win, versus 34 percent for Romney. Around the same time (Oct. 25-28), there was a comparable New York Times/CBS poll in which 51 percent of likely voters expect Obama to win, versus 34 percent for Romney.

But these results aren’t just stable across pollsters, they’ve also been quite stable over the past few weeks, even as the race appeared to tighten for a while. Politico and George Washington University ran a poll of likely voters on Oct. 22-25, finding 54 percent expect Obama to win, versus 36 percent for Romney. The University of Connecticut/Hartford Courant poll of likely voters got a somewhat higher share not venturing an answer, with 47 percent expecting Obama to win versus 33 percent for Romney. Finally, the ABC/Washington Post poll of registered voters run Oct. 10-13 found 56 percent expect Obama to win, compared to 35 percent for Romney.

I’m rather surprised by the similarities here – across time, across pollsters, across how they word the question, and across different survey populations (likely voters, registered voters, or adults) – but I suspect that is part of the nature of the question. You just don’t see the noise here that you see in the barrage of polls of voter intentions, which are extremely sensitive to all of these factors.

I always throw out the folks who don’t have an opinion, and count the proportions as a share of only those who have an opinion. By this measure, the proportion who expect Obama to win is: 61 percent (Gallup), 60 percent (The New York Times), 60 percent (Politico), 59 percent (Hartford Courant), 62 percent (ABC). The corresponding proportions who expect Romney to win are: 39 percent, 40 percent, 40 percent, 41 percent and 38 percent. Taking an average across all these polls: 60.3 percent expect Obama to win. Or if you prefer that I focus only on the freshest two polls, 60.7 percent expect him to win.

DL: The results do seem have tightened somewhat since the first debate, which Romney was widely seen to have won, right? Do the patterns — or lack of patterns — in the numbers help solve the issue of what most people are thinking of when they answer the expectation question: Private information (their friends’ voting plans, yard signs in their neighborhood, etc.) or public information (media coverage, speeches, etc.)?

JW: The results of the polls of voter intentions seem to have tightened a bit since the first debate. There’s an interesting school of thought in political science that basically says: voters are pretty predictable. But they don’t think too hard about how they’re going to vote until right before the election. So what happens is that public opinion through time just converges to where it “should” be. And viewed through this lens, the first debate was just an opportunity for people who really should always have been in Romney’s camp to figure out that they’re in Romney’s camp.

So why did the expectations polls move less sharply than intentions polls? One possibility is that your expectations are explicitly forward-looking, and perhaps people saw the race tightening as they saw that some of the support for Obama was a bit soft. Let me put this another way: There are two problems with how we usually ask folks how they plan to vote. First, the question captures the state of public opinion today, while the expectations question effectively asks you where you think public opinion is going. And second, polls typically demand a yes or no answer, when the reality may be that we know that our support is pretty weak, and it may change, or we aren’t even sure whether we’ll turn up to the polls. The virtue of asking about expectations is that you can think about each of your friends, and think not just about who they’re supporting today, but also whether they may change their minds in the future.

I worry that it sounds a bit like I haven’t answered your question, but that’s because I don’t have a super-sharp answer. If I had to summarize, it would be: expectations questions allow you to think about how the dynamics of the race may change, and so they are less sensitive to that change when it happens.

DL: Based on your research and the current polls, what does the expectations question suggest is the most likely outcome on Tuesday?

JW: If a majority expects Obama to win, then right there, it says that I’m forecasting an Obama victory.

But by how much? Here’s where it gets tricky. The fact that 60 percent of people think that Obama is going to win doesn’t mean that he’s going to win 60 percent of the votes. And it doesn’t mean that he’s a 60 percent chance to win. Rather, it simply says that given the information they have, 60 percent of people believe that Obama is going to win. Can we use this to say anything about his likely winning margin?

Yes. I’ll spare you the details of the calculation, but it says that if 60.3 percent of people expect Obama to beat Romney, then we can forecast that he’ll win about 52.5 percent of the two-party vote. That would be a solid win, though not as impressive as his seven-point win in 2008.

The proportion who expect Obama to win right now looks awfully similar to the proportion who expected George W. Bush to win in a Gallup Poll at a similar point in 2004. Ultimately Bush won 51.2 percent of the two-party vote.

Right now, Nate Silver is predicting that Obama will win 50.5 percent of the popular vote, and Romney 48.6 percent. As a share of the two-party vote, this says he’s forecasting Obama to win 51 percent of the vote. Now Silver’s approach aggregates responses from hundreds of thousands of survey respondents, while I have far fewer, so his estimate still deserves a lot of respect. I don’t want to overstate the confidence with which I’m stating my forecast. So let me put it this way: My approach says that it’s likely that Obama will outperform the forecasts of poll-based analysts like Silver.

DL: We’ll find out soon enough. Thanks.